R4D’s Evaluation and Adaptive Learning team partnered with Rising Academy Network to design, test, and improve a low-cost, scalable literacy intervention for their growing network of schools.
The Challenge
We know literacy leads to better life outcomes. We also know that “the majority of pupils in Sierra Leone are not learning how to read in schools” (UNICEF, 2014). Rising Academy Network (RAN) is a growing chain of low-fee private schools in Sierra Leone and Liberia that aims to widen access to quality education. Baseline data from an independent evaluation by Oxford University shows that RAN students, like their peers in other schools, are enrolling in Junior Secondary School with literacy skills well below grade-level, despite having completed at least six years of primary school.
Many RAN students struggled with basic literacy skills, making it difficult for them to engage with content and material in other subject areas. Initial results after the first academic year of the study (abridged due to the Ebola crisis) show that RAN students were making significant progress in reading, writing and math beyond that of their peers in both government schools and other private schools in Sierra Leone.
However, because students were starting from such a low baseline, RAN staff identified an urgent need to find ways to accelerate improvement, and to monitor that progress over the students’ first year at RAN.
The Opportunity
In 2016 and 2017, RAN partnered with Results for Development (R4D) to develop, test, and improve a low-cost literacy intervention that could be scaled across their growing network of schools. RAN set the ambitious goal for this pilot of increasing student literacy by an average of two grade levels for every one year a student spends at RAN. This required regular monitoring of student growth, informing students and their teachers of that growth, and intervening to improve literacy outcomes, all in an affordable and scalable way.
Instead of waiting for endline results of RAN’s impact evaluation to find out how successful RAN has been in improving students’ learning outcomes, RAN and R4D partnered to embed experimentation and evaluation into RAN’s ongoing activities, with the goal of continuously adapting to maximize outcomes.
Our Work
During the 2016-17 school year, RAN and R4D piloted three strategies within three RAN schools, and evaluated these interventions throughout an experimentation phase:
- a peer-led assessment tool, administered eight times by second-year (JSS2) students with all JSS1 students;
- “Rising Reader Boards” which visualized progress for teachers and students alike;
- and an early-morning Reading Club where “helping readers” (stronger students at baseline) worked one-on-one with “rising readers” (lower scores at baseline) to give both groups more practice reading.
One teacher from each school was trained by RAN and R4D staff to serve as the Literacy Program Coordinator. This individual led implementation of the interventions at their school, including coordinating assessments, updating rising boards, leading reading club, an sharing data with RAN headquarters and R4D.
We held a series of four “Learning Checks” throughout the year with RAN, during which R4D reflected back our quantitative (assessment and attendance data) and qualitative (interviews and focus groups) findings to the RAN team, and facilitated discussions about how iterate to improve the three interventions.
Findings
The adaptive learning approach helped RAN “fail fast” and learn faster as we honed an assessment tool that was validated, a data visualization scheme that was easy to understand for teachers and students, and a club that maximized the limited free time available to engender a love of reading and build literacy skills. In the school year that followed the pilot, the RAN team used these learnings to scale a revised literacy program in all of their JSS schools in Sierra Leone.
R4D’s engagement with RAN also impacted RAN as an organization. While RAN had embraced the importance of evaluation and learning since its outset, R4D provided a new framework and set of tools to use throughout the course of their program. RAN has provided R4D with specific examples of how they were inspired by R4D’s approach to conduct small-scale lean tests, and more broadly embrace a culture of testing and experimentation as they innovate and make decisions as they grow.
