quasi experimental design
Adolescent Girls’ Education in Somalia (AGES) Endline evaluation
This evaluation of CARE International’s Adolescent Girls Education in Somalia (AGES) programme concludes a four-and-a-half-year effort to track the programme’s impact on various cohorts of girl learners. The evaluation is preceded by three previous evaluations: a baseline (BL) round in late 2019, a Midline 1 (ML1) round in early 2022, and a Midline 2 (ML2) round in early 2023. Three key outcomes for girls are measured through this study: 1) learning outcomes, as measured by numeracy and literacy skills; 2) transitions outcomes, measured by girls’ trajectories since first surveyed, such as whether a girl has progressed in grade levels, or secured gainful employment after their participation in the AGES programme; and 3) sustainable change emerging from shifts in social norms at the community and individual level, as well as strengthened institutional capacities to support inclusive education.
In addition, the programme’s theory of change (ToC) posits that several intermediate outcomes mediate the effect of programme interventions on the primary outcomes. As such, this study also examines progress on several intermediate outcomes: 1) attendance rates, 2) quality of teaching practices, 3) girls’ leadership and life skills, 4) school management and governance, 5) community support for girls’ education, 6) girl’s self-efficacy, 7) strengthened economic circumstances for female youth, and finally, 8) access to social support services.
Importantly, this study comprises evaluations for three separate groups of girls, all recruited into the study at various points in time. The study began with an original baseline cohort of girls participating in the formal education (FE), Alternative Basic Education (ABE), and Non-Formal Education (NFE) programmes. These girls were recruited from schools in the following states, or geographic zone: Banadir, South West State, and Jubaland. The ABE programme focused on accelerated education of basic skills to enable girls to transition into the formal education system, while the NFE programme aimed to equip girls with the skills – both hard and soft – needed to pursue livelihoods, though some NFE girls also transition to the formal education system. These initial groups – whom we collectively refer to as Cohort 1 (C1) or the baseline cohort – were first recruited at BL in 2019, and were funded by the UK’s Girls’ Education Challenge (GEC) fund.
Two further cohorts, funded by USAID, were later introduced. Cohort 4, or C4, NFE girls were introduced into the study at ML1, while Cohort 5, or C5, NFE girls were introduced to the study at ML2. As such, unlike the C1 girls, we take the ML1 and ML2 evaluation rounds as the cohort-specific baselines for C4 and C5 NFE girls, respectively, and compare changes in main and intermediate outcomes from those rounds to EL. The USAID expansion of the NFE programming also meant the C4 and C5 Read More...
In addition, the programme’s theory of change (ToC) posits that several intermediate outcomes mediate the effect of programme interventions on the primary outcomes. As such, this study also examines progress on several intermediate outcomes: 1) attendance rates, 2) quality of teaching practices, 3) girls’ leadership and life skills, 4) school management and governance, 5) community support for girls’ education, 6) girl’s self-efficacy, 7) strengthened economic circumstances for female youth, and finally, 8) access to social support services.
Importantly, this study comprises evaluations for three separate groups of girls, all recruited into the study at various points in time. The study began with an original baseline cohort of girls participating in the formal education (FE), Alternative Basic Education (ABE), and Non-Formal Education (NFE) programmes. These girls were recruited from schools in the following states, or geographic zone: Banadir, South West State, and Jubaland. The ABE programme focused on accelerated education of basic skills to enable girls to transition into the formal education system, while the NFE programme aimed to equip girls with the skills – both hard and soft – needed to pursue livelihoods, though some NFE girls also transition to the formal education system. These initial groups – whom we collectively refer to as Cohort 1 (C1) or the baseline cohort – were first recruited at BL in 2019, and were funded by the UK’s Girls’ Education Challenge (GEC) fund.
Two further cohorts, funded by USAID, were later introduced. Cohort 4, or C4, NFE girls were introduced into the study at ML1, while Cohort 5, or C5, NFE girls were introduced to the study at ML2. As such, unlike the C1 girls, we take the ML1 and ML2 evaluation rounds as the cohort-specific baselines for C4 and C5 NFE girls, respectively, and compare changes in main and intermediate outcomes from those rounds to EL. The USAID expansion of the NFE programming also meant the C4 and C5 Read More...
Adolescent Girls’ Education in Somalia (AGES) Midline evaluation
Original Baseline Cohorts
Learning outcomes among the original baseline cohorts are mixed. The FE and ABE cohorts showed substantial learning gains at ML1 and continue to perform above their baseline levels, but have experienced significant backsliding in both literacy and numeracy since ML1. This learning loss is concerning, and is coupled with a failure, even at ML1, to meet the learning benchmarks established at baseline.1 C1 NFE girls, on the other hand, have shown no learning improvements since baseline on either literacy or numeracy, in either the ML1 or ML2 rounds.
Learning gains among FE girls include an increase in literacy of 20.5 points since baseline, compared to an expected (benchmarked) achievement of 29.7 points. This cohort came closer to meeting the numeracy targets, gaining 15.3 points since baseline, compared to an expected improvement of 18.5 points. ABE girls have also failed to meet their benchmarks, but by larger margins, falling short of the literacy and numeracy benchmarks by 15.4 points and 11.5 points, respectively.
An important caveat when assessing learning outcomes concerns differential exposure to schooling among the cohorts. Girls who remained enrolled in FE consistently improved considerably, but the cohort’s overall scores are reduced by the number of girls who dropped out over time; the same is true of ABE and NFE girls.2 While even consistent attenders generally failed to meet improvement benchmarks, these differential gains underscore the importance of continuous enrolment to achieving the programme’s learning goals.
C4 NFE Cohort
Improving learning outcomes—numeracy and Somali literacy—are at the centre of the AGES programme’s goals. C4 NFE girls were enrolled before ML1 and completed the 10-month NFE programme. At the ML2 evaluation point, in the aggregate, we find that both numeracy and literacy scores improved significantly from a low base set of scores at ML1. The C4 NFE girls improved their average numeracy scores from 21.1 points to 49.6 points, and average literacy scores increased by 18.3 points to 34.9 points.
Examining changes in learning outcomes by region, C4 NFE girls in Banadir, Lower Shabelle, and Middle Shabelle improved their numeracy scores from 24.6 to 28.2 points on average while in Bay C4 NFE girls only improved by 2.3 points on average. With regard to literacy scores, C4 NFE girls in Banadir and Lower Shabelle improved their literacy scores 23.2 and 24.9 points, respectively, while scores in Bay stagnated with only a 1.8 point increase. Girls from Bay scored significantly higher than girls from all other regions at ML1, but by ML2 the numeracy and literacy gains were minimal and not statistically significant. Read More...
Learning outcomes among the original baseline cohorts are mixed. The FE and ABE cohorts showed substantial learning gains at ML1 and continue to perform above their baseline levels, but have experienced significant backsliding in both literacy and numeracy since ML1. This learning loss is concerning, and is coupled with a failure, even at ML1, to meet the learning benchmarks established at baseline.1 C1 NFE girls, on the other hand, have shown no learning improvements since baseline on either literacy or numeracy, in either the ML1 or ML2 rounds.
Learning gains among FE girls include an increase in literacy of 20.5 points since baseline, compared to an expected (benchmarked) achievement of 29.7 points. This cohort came closer to meeting the numeracy targets, gaining 15.3 points since baseline, compared to an expected improvement of 18.5 points. ABE girls have also failed to meet their benchmarks, but by larger margins, falling short of the literacy and numeracy benchmarks by 15.4 points and 11.5 points, respectively.
An important caveat when assessing learning outcomes concerns differential exposure to schooling among the cohorts. Girls who remained enrolled in FE consistently improved considerably, but the cohort’s overall scores are reduced by the number of girls who dropped out over time; the same is true of ABE and NFE girls.2 While even consistent attenders generally failed to meet improvement benchmarks, these differential gains underscore the importance of continuous enrolment to achieving the programme’s learning goals.
C4 NFE Cohort
Improving learning outcomes—numeracy and Somali literacy—are at the centre of the AGES programme’s goals. C4 NFE girls were enrolled before ML1 and completed the 10-month NFE programme. At the ML2 evaluation point, in the aggregate, we find that both numeracy and literacy scores improved significantly from a low base set of scores at ML1. The C4 NFE girls improved their average numeracy scores from 21.1 points to 49.6 points, and average literacy scores increased by 18.3 points to 34.9 points.
Examining changes in learning outcomes by region, C4 NFE girls in Banadir, Lower Shabelle, and Middle Shabelle improved their numeracy scores from 24.6 to 28.2 points on average while in Bay C4 NFE girls only improved by 2.3 points on average. With regard to literacy scores, C4 NFE girls in Banadir and Lower Shabelle improved their literacy scores 23.2 and 24.9 points, respectively, while scores in Bay stagnated with only a 1.8 point increase. Girls from Bay scored significantly higher than girls from all other regions at ML1, but by ML2 the numeracy and literacy gains were minimal and not statistically significant. Read More...