Words can change worlds: An impact evaluation of Shine Literacy

May 2018
By Charlotte-Kathrin Stollberg

Reading literacy is a key problem among South African learners. The latest results from the 2016 Progress in International Reading Literacy Study (PIRLS) on Grade 4 reading achievements across 50 participating developed and developing countries indicated that eight out of ten Grade 4 learners in South Africa are unable to read for meaning (Mullis et al. 2017). Learners cannot locate clearly listed information in text or make inferences about stated events or actions (Mullis et al. 2017). The 2011 PIRLS, which considered South African Grade 5 learners, showed that even in higher grades the picture is dismal: 29% of those students did not reach the pre-PIRLS international benchmark for literacy (Pretorius and Spaull 2016). Similarly, the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ) reports have indicated that 27% of South African Grade 6 learners are functionally illiterate,1 with only very little improvement between 2000 and 2007 (Moloi and Chetty 2011). With fewer than 40 correct words per minute, learners are essentially rendered as ‘non-readers’ (Pretorius and Spaull 2016: 1456). 

The results from the SACMEQ and PIRLS reports need to be seen in the context of the extensive public spending on education in South Africa, which has been 5–6% of GDP since 1994 (World Bank 2016). This is relatively high, internationally speaking (World Bank 2016). Given that about 99% of learners are enrolled, school enrolment itself does not seem to be the issue (World Bank 2016). Instead, South Africa’s education crisis can be linked to the lack of quality inputs, not access – a phenomenon encountered repeatedly in developing countries (Pretorius and Spaull 2016). As a consequence of a poorly performing education sector, the proportion of unemployable low-skilled youth is increasing, ultimately leading to a widening of the already stark income and wealth inequality, and to a further halt on the move towards a knowledge-based South African economy (McCarthy and Oliphant 2013). 

Universally, as one of the primary skills acquired at school, reading literacy is defined not only as the ability to understand, use and reflect on text to develop reading knowledge and skills, but also, ultimately, to participate actively and successfully in society (Howie et al. 2008; Linnakylä et al. 2004; Pretorius and Spaull 2016). Illiterate individuals are at risk of being excluded from working and studying, but also from cultural or social life (Linnakylä et al. 2004). To further emphasise the importance of this topic, Overett and Donald
(1998) highlight literacy’s ability to equip people with cultural power and identity, which may – particularly for those with marginalised identities – give individuals the necessary skill set to transform their life. There is a large body of research confirming how early reading ability holds predictive validity for later child development and academic outcome (Du Plessis et al. 2003; Howie et al. 2008; The National Institute for Literacy 2008; Spaull 2011). It has been shown that interventions targeted at lower primary school level can be very effective in serving as relatively swift corrective measures for those who struggle to read (He et al., 2009; The National Institute for Literacy 2008). 

Undeniably, the current South African educational system has failed to deliver. Sub-standard academic performance and other education metrics, such as high dropout rates and grade retention, indicate a rather dire situation.2 This is in large part due to the structural and systemic issues that are deeply entrenched in the post-apartheid South African schooling environment. Apartheid left its footprint on the South African educational system, with unequal access to education and varying levels of educational quality – when it comes to both private and social returns on schooling, disparities along racial lines still exist. Even though the lives of African and Coloured people have generally improved, returns to education remain much higher for white people, who earn 40% more than Africans and 20% more than Coloured individuals per added year of schooling (Salisbury 2016).

The Shine Literacy programme offers an important, high-quality input to resource-poor schools, with the goal of addressing some of these educational shortfalls and thus redressing injustices of the past early on. Shine Literacy is a South African non-profit organisation (NPO) that provides reading assistance to learners who are identified to be at literacy risk by the end of Grade 1. Assistance takes place during Grade 2, in the form of bi-weekly one-on-one sessions between volunteers and at-risk learners, teaching phonetics, word recognition and decoding, and the shared reading of whole paragraphs. If the learner does not improve sufficiently after the completion of Grade 2, they will be taken on for another year during Grade 3. Shine’s programme is embedded in the mutual understanding between government, business and NPOs that South Africa’s literacy problem requires strong collaboration between a variety of actors, calling for a holistic approach. Shine Literacy aims to ensure that learners are equipped with their developmental age/grade adequate literacy level. Their ethos makes it clear that early interventions imply large gains in terms of academic self-esteem in later years, keeping the goal of promoting a culture of learning in mind. This mission follows the notion emphasized in the work on the returns to education by Heckman (2006; 2011) and Heckman and Masterov (2004), namely, that the earlier the intervention, the more persistent and socially just the returns will be. For instance, Heckman and Masterov (2004) estimated that the rate of return of an early intervention program designed for low-ability children is 16%. 

This study evaluates the impact of Shine Literacy by use of a difference-in-differences approach.3 The analysis can be placed in the international discussion on which inputs to education are required to overcome the persistent lack of quality. With a potential lookout for externality effects, this study did not only aim to measure the programme impact on learners’ literacy, but also to provide some evidence on why Shine Literacy should be scaled-up across South Africa. Large standard deviation literacy improvements of up to 1.9 were found, with the most conservative estimate starting from 0.6. This amounts to educationally meaningful results. Further, it appeared that those who were most deficient with regards to their literacy and thereby later schooling were assisted the most, and experienced the greatest literacy improvements. No heterogeneous effects on gender were found when assigned to the programme, boy learners do just as well as girl learners – this is an important insight into closing the gender gap that exists in literacy. 

By way of introduction, the following section discusses the available data and methods used to evaluate the Shine programme, as well as the results of the analysis and their contextual implications. By way of conclusion, the discussion section aims to provide some linkages to the international evidence and deliver not only some remarks on the limitations of the analysis, but also some suggestions as to future work. Read more

Download SA-TIED Working Paper 16/2018