Abstract

The Quality Preschool for Ghana Impact Evaluation 2016, Midline survey (QP4G-ML 2016) was approved by the Strategic Impact Evaluation Fund (SIEF) of the World Bank on August 2015 in the Great Accra Region of Ghana. The official project name is called "Testing and scaling-up supply- and demand-side interventions to improve kindergarten educational quality in Ghana”, known as “Quality Preschool for Ghana (QP4G)”.

The project seeks to increase the quality of preschool education during the two years of universal Kindergarten (KG) in Ghana through intervening in the supply-side (i.e., teacher in-service training) and the demand side (i.e., increasing parental awareness for developmentally appropriate quality early education).

The primary goal of the impact evaluation is to test the efficacy of a potentially scalable (8-day) in-service teacher training to improve the quality of KG teacher practices and interactions with children and to improve children’s development, school readiness and learning in both private and public preschools in the Greater Accra Region of Ghana. Additional goals of this evaluation are: to test the added value of combining a scalable (low-cost) parental awareness intervention with teacher in-service training; to compare implementation challenges in public and private schools; and to examine several important sources of potential heterogeneity of impact, primarily impacts in public vs. private schools.

The current submission is for the Midline Survey, conducted with 3 types of respondents across two phases – School survey and Caregiver [household] surveys. The school survey was conducted from May to July 2016 and consisted of collecting the following data: (a) direct assessments of children’s school readiness, (b) surveys of KG teachers, (c) direct observation of inventory of facilities within KG classrooms [environmental scan]; videotaping of KG classroom processes, teaching, and learning (not being submitted); as well as video coding of KG classroom video recordings using Teacher Instructional Practices and Processes Systems (instrument not being submitted). The caregiver survey was conducted via phone from August to September 2016 on primary caregivers of KG children. The caregiver survey sought information on caregivers’ background, poverty status, involvement or participation in school and home activities, and perception about ECD. Overall, the Midline Survey was conducted from May to September 2016 for all respondents.

Geographic coverage

Urban and Peri-Urban Districts, Greater Accra Region

Analysis unit

Units of analysis include individuals (KG teachers, children, caregivers), KG classrooms and preschools.

Universe

The survey universe is 6 poor districts in the Greater Accra Region. We sampled 240 schools, 108 public (Govt.) schools and 132 private schools. The population of interest is KG teachers and children in KG 1 and KG 2 classrooms in these schools, as well as the caregivers of sampled students.

Kind of data

Sample survey data [ssd]

Sampling procedure

This impact evaluation applies a cluster-randomized design. Eligible schools were randomly selected to participate in the study. The eligible population was schools with KG 1 and KG 2 classrooms (the two years of universal preprimary education) in six districts in the Greater Accra Region. In these six districts, we have sampled 240 schools; 108 public schools and 132 private schools in total.

The unit of randomization for this randomized control trial (RCT) is schools, whereby eligible schools (stratified by public and private sector schools) are randomly assigned to: (1) in-service teacher-training program only; (2) in-service teacher-training program plus parental awareness program; or (3) control (current standard operating) condition.

The sampling frame for this study was based on data in the Education Management Information System (EMIS) from the Ghana Education Service. This data was verified in a 'school listing exercise' conducted in May 2015.

Sample selection was done in four stages: The first stage involved purposive selection of six districts within the region based on two criteria: (a) most disadvantaged (using UNICEF's District League Table scores, out of sixteen total districts); and (b) close proximity to Accra Metropolitan for travel for the training of the KG teachers. The six selected municipals were La Nkwantanang-Madina Municipal, Ga Central Municipal, Ledzokuku-Krowor Municipal, Adentan Municipal, Ga South Municipal and Ga East Municipal.

The second stage involved the selection of public and private schools from each of the selected districts in the Accra region. We found 678 public and private schools (schools with kindergarten) in the EMIS database. Of these 361 schools were sampled randomly (stratified by district and school type) for the school listing exercise, done in May 2015. This was made up of 118 public schools and 243 private schools. The sampling method used for the school listing exercise was based on two approaches depending on the type of school. For the public schools, the full universe of public schools (i.e., 118) were included in the school listing exercise. However, private schools were randomly sampled using probability proportional to the size of the private schools in each district. Specifically, the private schools were sampled in each district proportionate to the total number of district private schools relative to the total number of private schools. In so doing, one school from the Ga South Municipal was removed and added to Ga Central so that all districts have a number of private schools divisible by three. This approach yielded 122 private schools. Additionally, 20 private schools were randomly selected from each of the districts (i.e., based on the remaining list of private schools in each district following from the first selection) to serve as replacement lists. The replacement list was necessary given the potential refusals from the private schools. There were no replacement lists for the public schools since all public schools would automatically qualify for participation.

The third stage involved selecting the final sample for the evaluation using the sampling frame obtained through the listing exercise. A total of 240 schools were randomly selected, distributed by district and sector. Schools were randomized into treatment groups after the first round of baseline data collection was completed.

The survey respondents were sampled using different sampling techniques: a. KG teachers: The research team sampled two KG teachers from each school; one from KG1 and KG2. KG teachers were sampled using purposive sampling method. In schools where there were more than two KG classes, the KG teachers from the "A" stream were selected. For the treatment schools, all KG teachers were invited to participate in the teacher training program. b. KG child-caregiver pair: The research team sampled KG children and their respective caregivers using simple random sampling method. Fifteen KG children-caregivers pair were sampled from each school. For schools with less than 15 KG children (8 from KG1, 7 from KG2 where possible), all KG children were included in the survey. KG children were selected from the same class as the selected KG teacher. The survey team used the class register to randomly select KG children who were present on the day of the school visit. Sampling was not stratified by gender or age. The caregivers of these selected child respondents were invited to participate in the survey. The research team sought informed consent from the school head teacher, caregivers, as well as child respondents.

Mode of data collection

Other [oth]

Research instrument

Data were collected at Midline Survey using structured questionnaires or forms.

Child Direct Assessment: The KG Child Assessment was conducted using the International Development and Early Learning Assessment (IDELA) tool designed by Save the Children. IDELA was adapted based on extensive pre-testing and piloting by different members of the evaluation team. The adapted version measured five indicators of ECD. The indicators were early numeracy skills, language/literacy skills and development, physical well-being and motor development, socio-emotional development, and approaches to learning. IDELA contained 28 items. In addition, one task was added – the Pencil Tap – to assess executive function skills. Apart from the English language, IDELA was translated and administered into three local languages, namely, Twi, Ga, and Ewe. These local language versions had gone through rigorous processes of translation and back translation. The IDELA tool has not been shared as Save the Children have proprietary rights over this.

KG Class Environmental Scan: The KG classroom observation involved taking inventories of the KG classrooms [environmental scan] and conducting video recordings of the classroom processes. The KG Class Environmental Scan tool was designed to take inventories of the facilities in the KG classrooms. The classroom video recordings have not been shared as they contain PIIs.

TIPPS: The video recordings taken during the classroom observations were coded using an early childhood education adapted version of Teacher Instructional Practices and Processes Systems (TIPPS). Seidman, Raza, Kim, and McCoy (2014) of New York University developed the TIPPS instrument. TIPPS observes nineteen key concepts of teacher practices and classroom processes that influence children’s cognitive and social-emotional development. The concept sheet was used to code the kindergarten classroom videos. The TIPPS tool has not been shared as

Abstract

The Quality Preschool for Ghana Impact Evaluation 2017, Endline survey (QP4G-EL 2017) was approved by the Strategic Impact Evaluation Fund (SIEF) of the World Bank on August 2015 in the Great Accra Region of Ghana. The official project name is called "Testing and scaling-up supply- and demand-side interventions to improve kindergarten educational quality in Ghana”, known as “Quality Preschool for Ghana (QP4G)”.

The project seeks to increase the quality of preschool education during the two years of universal Kindergarten (KG) in Ghana through intervening in the supply-side (i.e., teacher in-service training) and the demand side (i.e., increasing parental awareness for developmentally appropriate quality early education).

The primary goal of the impact evaluation is to test the efficacy of a potentially scalable (8-day) in-service teacher training to improve the quality of KG teacher practices and interactions with children and to improve children’s development, school readiness and learning in both private and public preschools in the Greater Accra Region of Ghana. Additional goals of this evaluation are: to test the added value of combining a scalable (low-cost) parental awareness intervention with teacher in-service training; to compare implementation challenges in public and private schools; and to examine several important sources of potential heterogeneity of impact, primarily impacts in public vs. private schools.

The current submission is for the Endline Survey, conducted with 3 types of respondents across two phases – School survey and Caregiver [household] surveys. The school survey was conducted from February to March 2017 and consisted of collecting the following data: (a) direct assessments of children’s school readiness, (b) surveys of KG teachers, (c) direct observation of inventory of facilities within KG classrooms [environmental scan]; videotaping of KG classroom processes, teaching, and learning (not being submitted); as well as video coding of KG classroom video recordings using Teacher Instructional Practices and Processes Systems (instrument not being submitted). The caregiver survey was conducted via phone from May to July August 2017 on the primary caregivers of the KG children. The caregiver survey sought information on caregivers’ background, poverty status, involvement or participation in school and home activities, and perception about ECD.

Geographic coverage

Urban and Peri-Urban Districts, Greater Accra Region

Analysis unit

Units of analysis include individuals (KG teachers, children, caregivers), KG classrooms and preschools.

Universe

The survey universe is 6 poor districts in the Greater Accra Region. We sampled 240 schools, 108 public (Govt.) schools and 132 private schools. The population of interest is KG teachers and children in KG 1 and KG 2 classrooms in these schools, as well as the caregivers of sampled students.

Kind of data

Sample survey data [ssd]

Sampling procedure

This impact evaluation applies a cluster-randomized design. Eligible schools were randomly selected to participate in the study. The eligible population was schools with KG 1 and KG 2 classrooms (the two years of universal preprimary education) in six districts in the Greater Accra Region. In these six districts, we have sampled 240 schools; 108 public schools and 132 private schools in total.

The unit of randomization for this randomized control trial (RCT) is schools, whereby eligible schools (stratified by public and private sector schools) are randomly assigned to: (1) in-service teacher-training program only; (2) in-service teacher-training program plus parental awareness program; or (3) control (current standard operating) condition.

The sampling frame for this study was based on data in the Education Management Information System (EMIS) from the Ghana Education Service. This data was verified in a 'school listing exercise' conducted in May 2015.

Sample selection was done in four stages: The first stage involved purposive selection of six districts within the region based on two criteria: (a) most disadvantaged (using UNICEF's District League Table scores, out of sixteen total districts); and (b) close proximity to Accra Metropolitan for travel for the training of the KG teachers. The six selected municipals were La Nkwantanang-Madina Municipal, Ga Central Municipal, Ledzokuku-Krowor Municipal, Adentan Municipal, Ga South Municipal and Ga East Municipal.

The second stage involved the selection of public and private schools from each of the selected districts in the Accra region. We found 678 public and private schools (schools with kindergarten) in the EMIS database. Of these 361 schools were sampled randomly (stratified by district and school type) for the school listing exercise, done in May 2015. This was made up of 118 public schools and 243 private schools. The sampling method used for the school listing exercise was based on two approaches depending on the type of school. For the public schools, the full universe of public schools (i.e., 118) were included in the school listing exercise. However, private schools were randomly sampled using probability proportional to the size of the private schools in each district. Specifically, the private schools were sampled in each district proportionate to the total number of district private schools relative to the total number of private schools. In so doing, one school from the Ga South Municipal was removed and added to Ga Central so that all districts have a number of private schools divisible by three. This approach yielded 122 private schools. Additionally, 20 private schools were randomly selected from each of the districts (i.e., based on the remaining list of private schools in each district following from the first selection) to serve as replacement lists. The replacement list was necessary given the potential refusals from the private schools. There were no replacement lists for the public schools since all public schools would automatically qualify for participation.

The third stage involved selecting the final sample for the evaluation using the sampling frame obtained through the listing exercise. A total of 240 schools were randomly selected, distributed by district and sector. Schools were randomized into treatment groups after the first round of baseline data collection was completed.

In the final stage, the survey respondents were sampled using different sampling techniques: a. KG teachers: The research team sampled two KG teachers from each school; one from KG1 and KG2. KG teachers were sampled using purposive sampling method. In schools where there were more than two KG classes, the KG teachers from the "A" stream were selected. For the treatment schools, all KG teachers were invited to participate in the teacher training program.

b. KG child-caregiver pair: The research team sampled KG children and their respective caregivers using simple random sampling method. Fifteen KG children-caregivers pair were sampled from each school. For schools with less than 15 KG children (8 from KG1, 7 from KG2 where possible), all KG children were included in the survey. KG children were selected from the same class as the selected KG teacher. The survey team used the class register to randomly select KG children who were present on the day of the school visit. Sampling was not stratified by gender or age. The caregivers of these selected child respondents were invited to participate in the survey.

The research team sought informed consent from the school head teacher, caregivers, as well as child respondents.

Mode of data collection

Face-to-face [f2f]

Research instrument

Data were collected at Endline Survey using structured questionnaires or forms.

Child Assessment: Child Assessment was conducted using International Development and Early Learning Assessment [IDELA] tool designed by Save the Children. IDELA was adapted based on extensive pre-testing and piloting by different members of the evaluation team. The adapted version measured five indicators of ECD. The indicators were early numeracy skills, language/literacy skills and development, physical well-being and motor development, socio-emotional development, and approaches to learning. IDELA contained 28 items. In addition, one task was added – the Pencil Tap – to assess executive function skills. IDELA was translated into three local languages, namely, Twi, Ga, and Ewe. These local language versions had gone through rigorous processes of translation and back translation. No change was made to the IDELA used.

Environmental Scan: The Environmental Scan tool was designed to take inventories of the facilities in the KG classrooms. No changes have been made to the Endline version of the KG Class Environmental Scan. The class environmental scan also included a video recording of the KG classroom processes and systems. TIPPS: The video recordings were coded using an early childhood education adapted version of TIPPS. Seidman, Raza, Kim, and McCoy (2014) of New York University developed the TIPPS instrument. TIPPS observes nineteen key concepts of teacher practices and classroom processes that influence children’s cognitive and social-emotional development. The concept sheet was used to code the kindergarten classroom videos.

Teacher Survey: The Teacher Survey measured KG teachers’ attitudes, behaviors, and perceptions on their background, nature and work conditions, depression and anxiety, external control, motivation, job satisfaction, burnout, and perceptions of early childhood education.

School Attendance Records: The School Attendance Records Form was designed to record

Abstract

The Ghana Quality for Preschool Impact Evaluation 2015, Baseline survey (QPIE-BL 2015) was approved by the Strategic Impact Evaluation Fund (SIEF) of the World Bank on August 2015 in the Great Accra Region of Ghana. The official project name is called "Testing and scaling-up supply- and demand-side interventions to improve kindergarten educational quality in Ghana”, known as “Quality Preschool for Ghana (QP4G)”.

The project seeks to increase the quality of preschool education during the two years of universal Kindergarten (KG) in Ghana through intervening in the supply-side (i.e., teacher in-service training) and the demand side (i.e., increasing parental awareness for developmentally appropriate quality early education).

The primary goal of the impact evaluation is to test the efficacy of a potentially scalable (8-day) in-service teacher training to improve the quality of KG teacher practices and interactions with children and to improve children’s development, school readiness and learning in both private and public preschools in the Greater Accra Region of Ghana. Additional goals of this evaluation are: to test the added value of combining a scalable (low-cost) parental awareness intervention with teacher in-service training; to compare implementation challenges in public and private schools; and to examine several important sources of potential heterogeneity of impact, primarily impacts in public vs. private schools.

The current submission is for the Baseline Survey, conducted with 5 types of respondents in two phases - Baseline I and Baseline II. Baseline I consisted of interviews with school head teachers and school proprietors (for private schools) and was conducted in June 2015. Baseline II consisted of collecting the following data: (a) direct assessments of children’s school readiness at school entry, (b) surveys of teacher well-being and demographics, (c) video recordings for classroom observations of teachers (not being submitted), and (d) caregiver surveys. This data collection was conducted from Sep-Nov 2015.

Geographic coverage

Urban and Peri-Urban Districts, Greater Accra Region

Analysis unit

Units of analysis include individuals (head teachers, teachers, children, caregivers) and schools.

Universe

The survey universe is 6 poor districts in the Greater Accra Region. We sampled 240 schools, 108 public (Govt.) schools and 132 private schools. The population of interest is KG teachers and students in Kindergarten (KG) 1 and KG 2 classrooms in these schools, as well as the caregivers of sampled students. It also includes school head teachers and owners/proprietors.

Kind of data

Sample survey data [ssd]

Sampling procedure

This impact evaluation applies a cluster-randomized design. Eligible schools were randomly selected to participate in the study. The eligible population was schools with KG 1 and KG 2 classrooms (the two years of universal preprimary education) in six districts in the Greater Accra Region. In these six districts we have sampled 240 schools; 108 public schools and 132 private schools in total.

The unit of randomization for this randomized control trial (RCT) is schools, whereby eligible schools (stratified by public and private sector schools) are randomly assigned to: (1) in-service teacher-training program only; (2) in-service teacher-training program plus parental awareness program; or (3) control (current standard operating) condition.

The sampling frame for this study was based on data in the Education Management Information System (EMIS) from the Ghana Education Service. This data was verified in a 'school listing exercise' conducted in May 2015.

Sample selection was done in multiple stages as shown in Figure 1. The first stage involved purposive selection of six districts within the region based on two criteria: (a) most disadvantaged (using UNICEF's District League Table scores, out of sixteen total districts); and (b) close proximity to Accra Metropolitan for travel for the training of the KG teachers. The six selected municipals were La Nkwantanang-Madina Municipal, Ga Central Municipal, Ledzokuku-Krowor Municipal, Adentan Municipal, Ga South Municipal and Ga East Municipal.

The second stage involved the selection of public and private schools from each of the selected districts in the Accra region. We found 678 public and private schools (schools with kindergarten) in the EMIS database. Of these 361 schools were sampled randomly (stratified by district and school type) for the school listing exercise, done in May 2015. This was made up of 118 public schools and 243 private schools.

The sampling method used for the school listing exercise was based on two approaches depending on the type of school. For the public schools, the full universe of public schools (i.e., 118) were included in the school listing exercise. However, private schools were randomly sampled using probability proportional to the size of the private schools in each district. Specifically, the private schools were sampled in each district proportionate to the total number of district private schools relative to the total number of private schools. In so doing, one school from the Ga South Municipal was removed and added to Ga Central so that all districts have a number of private schools divisible by three. This approach yielded 122 private schools. Additionally, 20 private schools were randomly selected from each of the districts (i.e., based on the remaining list of private schools in each district following from the first selection) to serve as replacement lists. The replacement list was necessary given the potential refusals from the private schools. There were no replacement lists for the public schools since all public schools would automatically qualify for participation.

The third stage involved selecting the final sample for the evaluation using the sampling frame obtained through the listing exercise. A total of 240 schools were randomly selected, distributed by district and sector. Schools were randomized into treatment groups after the first round of baseline data collection was completed.

The survey respondents were sampled using different sampling techniques: a. KG teachers: The research team sampled two KG teachers from each school; one from KG1 and KG2. KG teachers were sampled using purposive sampling method. In schools where there were more than two KG classes, the KG teachers from the "A" stream were selected. For the treatment schools, all KG teachers were invited to participate in the teacher training program. b. KG child-caregiver pair: The research team sampled KG children and their respective caregivers using simple random sampling method. Fifteen KG children-caregivers pair were sampled from each school. For schools with less than 15 KG children (8 from KG1, 7 from KG2 where possible), all KG children were included in the survey. KG children were selected from the same class as the selected KG teacher. The survey team used the class register to randomly select KG children who were present on the day of the school visit. Sampling was not stratified by gender or age. The caregivers of these selected child respondents were invited to participate in the survey.

The research team sought informed consent from the school head teacher, caregivers, as well as child respondents.

Mode of data collection

Face-to-face [f2f]

Research instrument

See attached questionnaires. All instruments have been shared except for IDELA (child assessment) as Save the Children have proprietary rights over this. Please contact the project Task Team Leader Deborah Newitter Mikesell dmikesell@worldbank.org for more information.

Cleaning operations

Data consistency checks (or High Frequency Checks) and back checks (audits) were conducted for all surveys remotely. Corrections were made during and after data collection after errors were reconciled.

All checks and cleaning was done using STATA and IPA possesses all the relevant code.

Response rate

Baseline I Out of the 276 schools that were selected for the Baseline I, 269 schools were surveyed (remember that potential replacement schools were also surveyed during Baseline I). This represents a response rate of 97%. It must, however, be emphasized that there were incomplete surveys in some of the schools, especially for the private schools. Incomplete surveys mean that only one of the surveys (instead of the two) was administered.

Baseline II All the surveys/assessment [with the exception of the Caregiver Survey] reported more than 90% response rate. The response rate for the Caregiver Survey was 60.0%.