Abstract

Abstract

Background: Adolescent girls in Kenya are disproportionately affected by early and unintended pregnancies, unsafe abortion and HIV infection. The In Their Hands (ITH) programme in Kenya aims to increase adolescents' use of high-quality sexual and reproductive health (SRH) services through targeted interventions. ITH Programme aims to promote use of contraception and testing for sexually transmitted infections (STIs) including HIV or pregnancy, for sexually active adolescent girls, 2) provide information, products and services on the adolescent girl's terms; and 3) promote communities support for girls and boys to access SRH services.

Objectives: The objectives of the evaluation are to assess: a) to what extent and how the new Adolescent Reproductive Health (ARH) partnership model and integrated system of delivery is working to meet its intended objectives and the needs of adolescents; b) adolescent user experiences across key quality dimensions and outcomes; c) how ITH programme has influenced adolescent voice, decision-making autonomy, power dynamics and provider accountability; d) how community support for adolescent reproductive and sexual health initiatives has changed as a result of this programme.

Methodology ITH programme is being implemented in two phases, a formative planning and experimentation in the first year from April 2017 to March 2018, and a national roll out and implementation from April 2018 to March 2020. This second phase is informed by an Annual Programme Review and thorough benchmarking and assessment which informed critical changes to performance and capacity so that ITH is fit for scale. It is expected that ITH will cover approximately 250,000 adolescent girls aged 15-19 in Kenya by April 2020. The programme is implemented by a consortium of Marie Stopes Kenya (MSK), Well Told Story, and Triggerise. ITH's key implementation strategies seek to increase adolescent motivation for service use, create a user-defined ecosystem and platform to provide girls with a network of accessible subsidized and discreet SRH services; and launch and sustain a national discourse campaign around adolescent sexuality and rights. The 3-year study will employ a mixed-methods approach with multiple data sources including secondary data, and qualitative and quantitative primary data with various stakeholders to explore their perceptions and attitudes towards adolescents SRH services. Quantitative data analysis will be done using STATA to provide descriptive statistics and statistical associations / correlations on key variables. All qualitative data will be analyzed using NVIVO software.

Study Duration: 36 months - between 2018 and 2020.

Geographic coverage

Homabay,Kakamega,Nakuru and Nairobi counties

Analysis unit

Private health facilities that provide T-safe services under the In Their Hands(ITH) Program.

Universe

1.Adolescent girls aged 15-19 who enrolled on the T-safe platform and received services and those who enrolled but did not receive services from the ITH facilities. 2.Service providers incharge of provision of T-safe services in the ITH facilities. 3.Mobilisers incharge of adolescent girls aged 15-19 recruitment into the T-safe program.

Sampling procedure

Qualitative Sampling

IDI participants were selected purposively from ITH intervention areas and facilities located in the four ITH intervention counties; Homa Bay, Nakuru, Kakamega and Nairobi respectively which were selected for the midline survey. Study participants were identified from selected intervention facilities. We interviewed one service provider of adolescent friendly ITH services per facility. Additionally, we conducted IDI's with adolescent girls' who were enrolled and using/had used the ITH platform to access reproductive health services or enrolled but may not have accessed the services for other reasons.

Sample coverage We successfully conducted a total of 122 In-depth Interviews with 54 adolescents enrolled on the T-Safe platform, including those who received services and those who were enrolled but did not receive services, 39 IDIS with service providers and 29 IDIs with mobilizers. The distribution per county included 51 IDI's in Nairobi City County (24 with adolescent girls, 17 with service providers and 10 with mobilisers), 15 IDI's in Nakuru County (2 with adolescent girls,8 with service providers and 5 with mobilisers), 34 IDI's in Homa Bay County (18 with adolescent girls,8 with service providers and 8 with mobilisers) and 22 IDI's in Kakamega County (10 with adolescent girls,6 with service providers and another 6 with mobilisers.)

Sampling deviation

N/A

Mode of data collection

Face-to-face [f2f]

Research instrument

The midline evaluation included qualitative in-depth interviews with adolescent T-Safe users, adolescents enrolled in the platform but did not use the services, providers and mobilizers to assess the adolescent user experience and quality of services as well as provider accountability under the T-Safe program. Generally,the aim of the qualitative study was to assess adolescents' T-Safe users experience across quality dimensions as well as provider's experiences and accountability. The dimensions assessed include adolescent's journey with the platforms, experience with the platform, perceptions of quality of services and how the ITH platforms changed provider behavior and accountability.

Adolescent in-depth interview included:Adolescent journey,Barriers to adolescents access to SRH services,Community attitudes towards adolescent use of contraceptives,Decision making,Factors influencing decision to visit a clinic,Motivating factors for girls to join ITH,Notable changes since the introduction of ITH,Parental support ,and Perceptions about T-Safe.

Service providers in-depth interview included;Personal and professional background,Provider's experience with ITH/T-safe platform,Notable changes/influences since the introduction of ITH/T-safe,Influence/Impact on the preference of adolescent service users and health care providers as a result of the program,Impact/influence of ITH on quality of care,Facilitators and barriers for adolescents to access SRH services,Mechanisms to address the barriers,Challenges related to the facility,Feedback about facility from adolescents,Types of support needed to improve SRH services provided to adolescents Scenarios of different clients accessing SRH services,and Free node.

Mobilisers in-depth interview included;Mobilizer responsibilities and designation,Job description,Motivation for joining ITH,Personal and professional background,Training,Mobilizer roles in ITH,Mobilization process ,Experience with ITH platform,Key messages shared with adolescent about ITH/ Tsafe during enrollment,Motivating factors for adolescents to join ITH/Tsafe,Community's attitude towards ITH/Tsafe,Challenges faced by mobilizers when mobilizing adolescents for Tsafe,Adolescents view regarding platform,Addressing the challenges ,andFree node

Cleaning operations

Qualitative interviews were audio-recorded and the audio recordings were transmitted to APHRC study team by uploading the audios to google drive which was only accessible to the team. Related interview notes, participant's description forms and Informed consent forms were transported to APHRC offices in Nairobi at the end of data collection where the data transcription and coding was conducted. Audio recordings from qualitative interviews were transcribed and saved in MS Word format. The transcripts were stored electronically in password protected computers and were only accessible to the evaluation team working on the project. A qualitative software analysis program (NVIVO) was used to assist in coding and analyzing the data. A “thematic analysis” approach was used to organize and analyze the data, and to assist in the development of a codebook and coding scheme. Data was analyzed by first reading the full IDI transcripts, becoming familiar with the data and noting the themes and concepts that emerged. A thematic framework was developed from the identified themes and sub-themes and this was then used to create codes and code the raw data.

Response rate

N/A

Sampling error estimates

N/A

Abstract

Background: Adolescent girls in Kenya are disproportionately affected by early and unintended pregnancies, unsafe abortion and HIV infection. The In Their Hands (ITH) programme in Kenya aims to increase adolescents' use of high-quality sexual and reproductive health (SRH) services through targeted interventions. ITH Programme aims to promote use of contraception and testing for sexually transmitted infections (STIs) including HIV or pregnancy, for sexually active adolescent girls, 2) provide information, products and services on the adolescent girl's terms; and 3) promote communities support for girls and boys to access SRH services.

Objectives: The objectives of the evaluation are to assess: a) to what extent and how the new Adolescent Reproductive Health (ARH) partnership model and integrated system of delivery is working to meet its intended objectives and the needs of adolescents; b) adolescent user experiences across key quality dimensions and outcomes; c) how ITH programme has influenced adolescent voice, decision-making autonomy, power dynamics and provider accountability; d) how community support for adolescent reproductive and sexual health initiatives has changed as a result of this programme.

Methodology ITH programme is being implemented in two phases, a formative planning and experimentation in the first year from April 2017 to March 2018, and a national roll out and implementation from April 2018 to March 2020. This second phase is informed by an Annual Programme Review and thorough benchmarking and assessment which informed critical changes to performance and capacity so that ITH is fit for scale. It is expected that ITH will cover approximately 250,000 adolescent girls aged 15-19 in Kenya by April 2020. The programme is implemented by a consortium of Marie Stopes Kenya (MSK), Well Told Story, and Triggerise. ITH's key implementation strategies seek to increase adolescent motivation for service use, create a user-defined ecosystem and platform to provide girls with a network of accessible subsidized and discreet SRH services; and launch and sustain a national discourse campaign around adolescent sexuality and rights. The 3-year study will employ a mixed-methods approach with multiple data sources including secondary data, and qualitative and quantitative primary data with various stakeholders to explore their perceptions and attitudes towards adolescents SRH services. Quantitative data analysis will be done using STATA to provide descriptive statistics and statistical associations / correlations on key variables. All qualitative data will be analyzed using NVIVO software.

Study Duration: 36 months - between 2018 and 2020.

Geographic coverage

Narok and Homabay counties

Analysis unit

Households

Universe

All adolescent girls aged 15-19 years resident in the household.

Sampling procedure

The sampling of adolescents for the household survey was based on expected changes in adolescent's intention to use contraception in future. According to the Kenya Demographic and Health Survey 2014, 23.8% of adolescents and young women reported not intending to use contraception in future. This was used as a baseline proportion for the intervention as it aimed to increase demand and reduce the proportion of sexually active adolescents who did not intend to use contraception in the future. Assuming that the project was to achieve an impact of at least 2.4 percentage points in the intervention counties (i.e. a reduction by 10%), a design effect of 1.5 and a non- response rate of 10%, a sample size of 1885 was estimated using Cochran's sample size formula for categorical data was adequate to detect this difference between baseline and end line time points. Based on data from the 2009 Kenya census, there were approximately 0.46 adolescents girls per a household, which meant that the study was to include approximately 4876 households from the two counties at both baseline and end line surveys.

We collected data among a representative sample of adolescent girls living in both urban and rural ITH areas to understand adolescents' access to information, use of SRH services and SRH-related decision making autonomy before the implementation of the intervention. Depending on the number of ITH health facilities in the two study counties, Homa Bay and Narok that, we sampled 3 sub-Counties in Homa Bay: West Kasipul, Ndhiwa and Kasipul; and 3 sub-Counties in Narok, Narok Town, Narok South and Narok East purposively. In each of the ITH intervention counties, there were sub-counties that had been prioritized for the project and our data collection focused on these sub-counties selected for intervention. A stratified sampling procedure was used to select wards with in the sub-counties and villages from the wards. Then households were selected from each village after all households in the villages were listed. The purposive selection of sub-counties closer to ITH intervention facilities meant that urban and semi-urban areas were oversampled due to the concentration of health facilities in urban areas.

Qualitative Sampling

Focus Group Discussion participants were recruited from the villages where the ITH adolescent household survey was conducted in both counties. A convenience sample of consenting adults living in the villages were invited to participate in the FGDS. The discussion was conducted in local languages. A facilitator and note-taker trained on how to use the focus group guide, how to facilitate the group to elicit the information sought, and how to take detailed notes. All focus group discussions took place in the local language and were tape-recorded, and the consent process included permission to tape-record the session. Participants were identified only by their first names and participants were asked not to share what was discussed outside of the focus group. Participants were read an informed consent form and asked to give written consent. In-depth interviews were conducted with purposively selected sample of consenting adolescent girls who participated in the adolescent survey. We conducted a total of 45 In-depth interviews with adolescent girls (20 in Homa Bay County and 25 in Narok County respectively). In addition, 8 FGDs (4 each per county) were conducted with mothers of adolescent girls who are usual residents of the villages which had been identified for the interviews and another 4 FGDs (2 each per county) with CHVs.

Sampling deviation

N/A

Mode of data collection

Face-to-face [f2f] for quantitative data collection and Focus Group Discussions and In Depth Interviews for qualitative data collection

Research instrument

The questionnaire covered; socio-demographic and household information, SRH knowledge and sources of information, sexual activity and relationships, family planning knowledge, access, choice and use when needed, exposure to family planning messages and voice and decision making autonomy and quality of care for those who visited health facilities in the 12 months before the survey. The questionnaire was piloted before the data collection and the questions reviewed for appropriateness, comprehension and flow. The questionnaire was piloted among a sample of 42 adolescent girls (two each per field interviewer) 15-19 from a community outside the study counties.

The questionnaire was originally developed in English and later translated into Kiswahili. The questionnaire was programmed using ODK-based Survey CTO platform for data collection and management and was administered through face-to-face interview.

Cleaning operations

The survey tools were programmed using the ODK-based SurveyCTO platform for data collection and management. During programming, consistency checks were in-built into the data capture software which ensured that there were no cases of missing or implausible information/values entered into the database by the field interviewers. For example, the application included controls for variables ranges, skip patterns, duplicated individuals, and intra- and inter-module consistency checks. This reduced or eliminated errors usually introduced at the data capture stage. Once programmed, the survey tools were tested by the programming team who in conjunction with the project team conducted further testing on the application's usability, in-built consistency checks (skips, variable ranges, duplicating individuals etc.), and inter-module consistency checks. Any issues raised were documented and tracked on the Issue Tracker and followed up to full and timely resolution. After internal testing was done, the tools were availed to the project and field teams to perform user acceptance testing (UAT) so as to verify and validate that the electronic platform worked exactly as expected, in terms of usability, questions design, checks and skips etc.

Data cleaning was performed to ensure that data were free of errors and that indicators generated from these data were accurate and consistent. This process begun on the first day of data collection as the first records were uploaded into the database. The data manager used data collected during pilot testing to begin writing scripts in Stata 14 to check the variables in the data in 'real-time'. This ensured the resolutions of any inconsistencies that could be addressed by the data collection teams during the fieldwork activities. The Stata 14 scripts that perform real-time checks and clean data also wrote to a .rtf file that detailed every check performed against each variable, any inconsistencies encountered, and all steps that were taken to address these inconsistencies. The .rtf files also reported when a variable was

Abstract

Abstract

Background: Adolescent girls in Kenya are disproportionately affected by early and unintended pregnancies, unsafe abortion and HIV infection. The In Their Hands (ITH) programme in Kenya aims to increase adolescents' use of high-quality sexual and reproductive health (SRH) services through targeted interventions. ITH Programme aims to promote use of contraception and testing for sexually transmitted infections (STIs) including HIV or pregnancy, for sexually active adolescent girls, 2) provide information, products and services on the adolescent girl's terms; and 3) promote communities support for girls and boys to access SRH services.

Objectives: The objectives of the evaluation are to assess: a) to what extent and how the new Adolescent Reproductive Health (ARH) partnership model and integrated system of delivery is working to meet its intended objectives and the needs of adolescents; b) adolescent user experiences across key quality dimensions and outcomes; c) how ITH programme has influenced adolescent voice, decision-making autonomy, power dynamics and provider accountability; d) how community support for adolescent reproductive and sexual health initiatives has changed as a result of this programme.

Methodology ITH programme is being implemented in two phases, a formative planning and experimentation in the first year from April 2017 to March 2018, and a national roll out and implementation from April 2018 to March 2020. This second phase is informed by an Annual Programme Review and thorough benchmarking and assessment which informed critical changes to performance and capacity so that ITH is fit for scale. It is expected that ITH will cover approximately 250,000 adolescent girls aged 15-19 in Kenya by April 2020. The programme is implemented by a consortium of Marie Stopes Kenya (MSK), Well Told Story, and Triggerise. ITH's key implementation strategies seek to increase adolescent motivation for service use, create a user-defined ecosystem and platform to provide girls with a network of accessible subsidized and discreet SRH services; and launch and sustain a national discourse campaign around adolescent sexuality and rights. The 3-year study will employ a mixed-methods approach with multiple data sources including secondary data, and qualitative and quantitative primary data with various stakeholders to explore their perceptions and attitudes towards adolescents SRH services. Quantitative data analysis will be done using STATA to provide descriptive statistics and statistical associations / correlations on key variables. All qualitative data will be analyzed using NVIVO software.

Study Duration: 36 months - between 2018 and 2020.

Geographic coverage

Homabay county

Analysis unit

Households

Universe

Adolescent girls aged 15-19 years, parents and the community health volunteers

Sampling procedure

Quantitative Sampling

We estimated a sample size of 1,918 to detect a five percentage-point difference in the use of long term methods between baseline and endline time points at 80% power.As baseline, 23% of the adolescent girls reported that they were using long term methods in Homa Bay county. We sampled three sub counties—Ndhiwa, Homa Bay town and Kasipul for the endline survey. However, as fieldwork was interrupted due to the COVID-19 pandemic, we added one sub county—Karachuonyo sub county—when data collection resumed in September 2020. Sub counties and wards were purposively selected from sub counties that had been prioritized for the ITH program based on availability of ITH affiliated health facilities. The purposive selection of sub counties based on presence of ITH intervention affiliated health facilities meant that urban and peri-urban areas were oversampled due to the concentration of the health facilities in urban/peri-urban areas. In each ward, eight villages that formed the immediate catchment area for each ITH program affiliated health facilities were then selected for the study. We conducted a household listing of all households in each sampled village to identify households with an adolescent girl who met the study's inclusion criteria. Households were then randomly sampled from the list of households with eligible adolescents of age 15-19 years. To be eligible, an adolescent girl had to be aged 15-19 years, resident in the study area for at least six months preceding the study. Accordingly, students who stayed in boarding schools away from their parents were excluded from the study.

Qualitative Sampling

The qualitative component involved in-depth interviews (IDIs) with adolescent girls ages 15-19 years and focus group discussions (FGDs) with parents/adults and CHVs. We conducted IDIs with adolescent girls who had enrolled in the program but dropped out for various reasons, as well as girls who were enrolled and still using t-safe services. In addition, we conducted FGDs with CHVs and parents/adult caretakers of adolescents aged 15-19 years from the program areas. Participants were purposively selected from the villages included in the evaluation study. For the endline study, we conducted 17 IDIs with adolescents who had been enrolled in the ITH program and were receiving services or had dropped from the program. We also conducted two FGDs with CHVs and four FGDs with parents/adultcaretakers of adolescents aged 15-19 years.

Sampling deviation

N/A

Mode of data collection

Face-to-face [f2f] for quantitative data collection and Focus Group Discussions and In Depth Interviews for qualitative data collection

Research instrument

An interviewer-administered questionnaire was used to collect data from adolescent girls. The questionnaire included questions on socio-demographic and household characteristics; SRH knowledge and sources of information; sexual activity and relationships; contraceptive knowledge, access, choice and use; and exposure to family planning messages and contraceptive decision making. To assess adolescents’ exposure to the t-safe program we included a series of questions drawn from similar project evaluation surveys as well as t-safe project program monitoring indicators. The questions assessed whether adolescents had ever heard the t-safe program, whether they have ever been contacted by mobilizers, whether they participated in any community event organized by the t-safe mobilizers, whether they received information about SRH through t-safe affiliated organizations Facebook or website, and whether they received SMS or WhatsApp messages focused on SRH from tsafe. For those who responded positively, the survey asked further questions on the sources; from which site on internet or Facebook’ or ‘which person or organization sent you these messages’ and ‘how many times have you received information’. Adolescents were also asked whether they had ever registered to a t-safe or Triggerise platform using a mobile phone after discussing with a mobilizer, after discussing with their peers or family members or by themselves after hearing from some other places. The questionnaire was developed in English and then translated into Kiswahili. Data were collected on android tablets programmed using the Open Data Kit (ODK)-based SurveyCTO platform.

For the qualitative component ;Semi-structured interview guides were developed by experienced researchers in consultation with the program partners for the qualitative interviews (with adolescent girls) and FGDs (with parents/adult caretakers of adolescents and CHVs). The guides included probes to explore adolescents' exposure to the ITH program; their experiences with program's SRH services; their perceptions on quality of services; as well as challenges and barriers to access of SRH services. The guides also included probes on the community’s "support" for adolescents' sexual and reproductive health services and; their perspectives on the effects of the program. The guides were developed in English and then translated into Kiswahili for data collection. The guides were pre-tested during the pilot study.

Cleaning operations

Quantitative data was collected on android tablets programmed using the Open Data Kit (ODK)-based SurveyCTO platform while qualitative data was collected using a recorder.Once quantitative data were confirmed to be complete, the data was approved for synchronization. Data were electronically transmitted to a secure password protected SurveyCTO server at the APHRC office. Backup versions of the data remained in the encrypted and password-protected tablets until the end of field activities when all the data were considered to have been synchronized. Subsequently, tablet was securely and permanently cleaned. Data on the server were retrieved by the data manager and then downloaded for use. For qualitative data, audio recordings from qualitative interviews were transcribed and saved in MS Word format. The transcripts were stored electronically in password protected computers and were only accessible to the evaluation team working on the project.

Response rate

N/A

Sampling error estimates

N/A