Two Rabbits Impact Evaluation Results
After two years of piloting, six weeks of data collection, and endless hours of work, Two Rabbits is proud and excited to report findings from our recent impact evaluation! Read on to find out what we have accomplished and what this means for our students.
Background. Two Rabbits conducted a small-scale randomized controlled trial of our model from September 2015 to June 2017. We created one year of preschool curriculum in the Baka language and cultural style, and uploaded it onto hand crank-powered mp3 devices. Project staff trained community-nominated lesson facilitators on child-centered pedagogy and mp3 device operation, who then facilitated daily learning sessions for a total of 250 Baka children age 3-6.
Data collection. We gathered baseline data from a sample of ten Baka villages, and implemented the program in two villages randomly selected from these ten, over two academic years. Our targeted outcomes were increased school readiness (measured through student literacy outcomes), and increased cultural pride (measured through interviews with parents). We measured these skills at baseline and endline among first graders, to test program benefits beyond the preschool year.
Results. Our data shows significant improvements in student literacy skills and community sense of cultural pride.
In literacy, Two Rabbits alumni have reading skills that are twice as high as their peers in the control group. The table here shows average performance on various emergent literacy tasks by students in the treatment and control groups. Across tasks, Two Rabbits students' scores are double their non-participating peers. For example, Two Rabbits participants could read six letters per minute from a list, compared to just three letters per minute among the control group. Asked to find a requested letter among a list of letters, Two Rabbits alumni could find an average of 4.5 per minute, compared to 2.5 per minute of control group children.
We also found that Two Rabbits villages had greater gender parity in reading outcomes than control villages. In control villages, girls performed just 21% as well as boys in reading letters from a list, and just 11% as well as boys in pointing out a requested letter from a list. In Two Rabbits villages, however, girls performed 91% as well and 65% as well as boys on these respective tasks. While boys still consistently outperform girls across all villages, it is promising that Two Rabbits has demonstrated success in closing this gap.
Regarding cultural pride, parents expressed greater pride in their culture and hope for the future in treatment villages than control villages. Parents in Two Rabbits villages felt that "Baka children are proud of their culture," and that as adults "we ourselves are satisfied, lead a fulfilled life, and have a culture that is rich in potential.” They felt deeply justified in their sense of pride, saying that "God created the Baka, and He makes no mistakes. He says that Baka children should not be ashamed to be Baka.” In control villages, parents agreed that Baka culture is important, but noted that Baka children tend to be ashamed and abandon cultural traditions. Parents were more optimistic about their children’s ability to learn in school than in control villages. While control village parents emphasized skills that their children lack (like the ability to speak French), parents in Two Rabbits villages noted their children’s strengths (such as curiosity, intellect, and playfulness) as effective learners.
Together, these findings indicate that Two Rabbits has positively impacted its target outcomes: boosting children's literacy skills, reducing gender gaps, increasing cultural pride, and increasing parents' confidence in their child's ability to learn. It is important to note that, while it was a randomized experiment, our sample size was small and so our result should be interpreted with caution. That said, with this preliminary evidence in hand, we are poised expand to more villages, and to update our content to be even more effective.
We are proud of what our learners have accomplished, and grateful to our community of supporters that have made this possible. We look forward to expanding to bring the joy of culturally adapted learning to more communities!