Introduction
Adolescent girls and young women (AGYW) in South Africa are disproportionately impacted by HIV. They account for 35% of new HIV infections in the country [1], and their incidence is three times higher than that of adolescent boys and young men [Reference Zuma, Simbayi and Zungu2]. Multiple social and structural factors drive high infection rates among AGYW, such as a high prevalence of age-disparate sex (estimated at 45% in a recent study) [Reference Magni, Hatcher, Gibbs, Wamoyi, Dunkle and Christofides3] and intimate partner violence (up to 45% in some regions of South Africa) [Reference Jewkes, Dunkle, Nduna and Shai4,Reference Mthembu, Mabaso, Reis, Zuma and Zungu5]. There is increasing recognition that reducing HIV incidence among AGYW requires prevention approaches that tackle these social and structural determinants of risk, going beyond traditional biomedical approaches [6].
IMARA-SA (Informed, Motivated, Aware, and Responsible Adolescents and Adults- South Africa) is a culturally tailored intervention for AGYW and their female caregivers (FC) (e.g., mothers, sisters, aunts, grandmothers) designed to reduce HIV-related disparities by empowering AGYW with sexual risk prevention skills [Reference Donenberg, Atujuna and Merrill7]. The intervention draws on a relatively underutilized resource for HIV prevention in South Africa – namely, the engagement of families, which play a central role in the growth and sexual decision-making of AGYW [Reference Soon, Kaida and Nkala8,Reference Adu-Mireku9]. Some studies have sought to engage parents in HIV prevention [Reference Arthur, Christofides and Nelson10–Reference Vandenhoudt, Miller and Ochura12] and to support youth living with HIV [Reference Bhana, Mellins and Petersen13–Reference Denison, Burke and Miti15]. However, we are unaware of other programs specifically engaging AGYW and their FC in HIV/STI prevention in the region despite the important part that mothers, in particular, play in promoting safer sexual behavior of their adolescents [Reference Widman, Choukas-Bradley, Noar, Nesi and Garrett16]. Our team found promising early findings on the intervention’s reach, feasibility, acceptability, fidelity, and effectiveness in a pilot study with 59 AGYW-FC dyads [Reference Merrill, Atujuna and Emerson17]. A randomized controlled trial (RCT) was subsequently conducted to assess the impact of IMARA-SA on incident sexually transmitted infections (STIs), HIV testing and counseling (HTC), and pre-exposure prophylaxis (PrEP) uptake [Reference Donenberg, Atujuna and Merrill7]. Analyses of the RCT data (currently in progress) will shed light on the intervention’s effectiveness in achieving its desired health outcomes.
Beyond the assessment of intervention health outcomes, there is growing recognition of the need to bridge the gap between research and practice by scientifically studying questions about implementation [Reference Peters, Adam, Alonge, Agyepong and Tran18]. Implementation outcomes – i.e., the effects of deliberate actions to implement an intervention [Reference Proctor, Silmere and Raghavan19] – are key ingredients in implementation success. They are necessary precursors to desired health outcomes, given that an intervention cannot achieve effectiveness if not implemented well [Reference Proctor, Silmere and Raghavan19]. Proctor et al. proposed a taxonomy of eight implementation outcomes [Reference Proctor, Silmere and Raghavan19], comprising the implementation outcomes framework, which sparked notable advancements in the implementation science field [Reference Proctor, Bunger and Lengnick-Hall20]. Increasingly, researchers are assessing effectiveness and implementation outcomes within the same study through hybrid effectiveness-implementation designs [Reference Curran, Bauer, Mittman, Pyne and Stetler21]. Concurrent examination of both types of outcomes can accelerate the speed at which research findings are put into practice [Reference Landes, McBain and Curran22]. The literature examining implementation experiences of HIV prevention programming for AGYW in the region is growing [Reference Rosen, Musheke and Mulenga23,Reference McClinton Appollis, Duby and Jonas24], but there remains a need for further understanding of the implementation outcomes to strengthen program delivery and increase the potential for health impacts.
This study examined the implementation outcomes of IMARA-SA as a sub-study within the ongoing RCT. We focused on the perspectives of IMARA-SA study staff, investigators, and collaborators. Study staff and other individuals supporting delivery are well-suited to sharing perspectives on implementation outcomes, given their unique views on organizational and systems factors that affect delivery. Our study aim was to examine six implementation outcomes of IMARA-SA – i.e., acceptability, appropriateness, feasibility, reach, adoption, and sustainability – from the perspective of IMARA-SA staff, investigators, and collaborators, drawing on the implementation outcomes framework [Reference Proctor, Silmere and Raghavan19].
Materials and methods
Study design
This study used an sequential explanatory mixed methods design. Quantitative data were collected first, followed by qualitative data, with equal value placed on each to value the strengths and limitations of each method (i.e., QUANT → QUAL) [Reference Creswell and Plano Clark25,Reference Palinkas, Aarons, Horwitz, Chamberlain, Hurlburt and Landsverk26]. Both types of methods were used to address the same study aim (i.e., a convergence function) [Reference Palinkas, Aarons, Horwitz, Chamberlain, Hurlburt and Landsverk26].
Parent study
This study was a sub-study within a larger RCT conducted with AGYW (15–19 years old) and their FC (24+ years) from Phillippi and surrounding townships (e.g., Nyanga, Gugulethu, Khayelitsha) in the Cape Town metropolitan area (details reported elsewhere [Reference Donenberg, Atujuna and Merrill7]). Dyads of AGYW and their FC were recruited through street outreach, word-of-mouth referral, flyers, clinic contacts by study personnel, and walk-ins to the clinic. They were randomized 1:1 to the IMARA-SA treatment arm or a health promotion control group. Female interventionists of similar backgrounds to the participants delivered the intervention (roughly 10 hours long) over two days at the DTHF offices in Philippi Village. Participants completed assessments – including surveys, STI testing, HIV testing, and counseling – at baseline, 6 months, and 12 months. At the end of each workshop day, participants also completed surveys about their perceptions of the acceptability, appropriateness, and feasibility of the intervention (findings to be reported elsewhere). Participants were reimbursed 250 rand cash or the equivalent in vouchers/gift cards for each study visit.
Positionality statement
The study was carried out through a collaboration between the Desmond Tutu Health Foundation (DTHF) in South Africa and the Center for Dissemination and Implementation Science in Health Disparities (CDIS) at the University of Illinois Chicago (UIC). Our authorship team includes 10 individuals affiliated with DTHF and UIC. All co-authors identify as female – aligning with the focus of the intervention on women and girls – and contributed expertise in HIV/STI risk among AGYW in South Africa, family-based interventions, implementation science, and mixed-methods studies. While carrying out this study, we acknowledged how our identities, nationalities, and backgrounds shaped our approach to the research and our interpretation of findings. We used extensive discussion and analytical memos [Reference Saldana27] to critically reflect on our positionalities and ensure that our findings would achieve goodness – i.e., where emerging themes authentically represent the data [Reference Tobin and Begley28].
Intervention overview
IMARA-SA is an HIV/STI prevention intervention for AGYW and their FC, adapted from the evidence-based IMARA program for Black adolescent girls and their mothers in the U.S. [Reference Donenberg, Kendall, Emerson, Fletcher, Bray and McCabe29,Reference Donenberg30]. AGYW and FC participate in separate and joint activities, which are interactive and experiential, over two days. The curriculum addresses individual factors (e.g., knowledge and attitudes about HIV/STIs), social factors (e.g., the family context), and structural factors (e.g., gender dynamics) that drive HIV risk (additional detail is presented elsewhere [Reference Donenberg, Atujuna and Merrill7]). The program aims to enhance the credibility of the FC as a resource for AGYW in HIV/STI prevention. The group format is designed to facilitate structural change by building community norms for practicing prevention (e.g., HIV testing and counseling and PrEP uptake) and reducing HIV stigma. The goals and motto emphasize strong relationships between AGYW and FCs, as well as sisterhood, empowerment, and HIV/STI prevention motivation. Participants are given homework at the end of Day 1 to complete before Day 2 to facilitate reflection on learnings.
Participants and procedures
Data were collected between 30/8/2021 and 11/4/2024. Individuals were eligible for the study if they were a study staff member, investigator, or collaborator on the IMARA-SA study or if they were a staff member or administrator from DTHF, 18 years or older, and spoke English. Each group offered different but valuable perspectives on the project. IMARA-SA study staff were directly involved in the day-to-day running of the study. Investigators were either directly supporting the study’s implementation or heavily involved in decision-making throughout the study through bi-weekly calls hosted between the SA and US teams. “Collaborators” included individuals who were not part of the study team but had advised on the conception and/or implementation of IMARA-SA. Finally, DTHF staff or administrators were less intimately involved but were regularly informed about the project and thus well-positioned to offer insights into how the project fit into the landscape of DTHF and other similar programs in the area.
We invited all South Africa-based IMARA-SA study staff to complete surveys. We also invited select DTHF finance and grants staff, as well as administrators, who were involved in implementing study budgets and procuring project resources. Baseline surveys were administered prior to the start of the RCT’s launch or when joining the study (for study staff). Follow-up surveys were administered roughly 18 months after baseline surveys or prior to a study staff member transition off the study (mean = 14.3 months, SD = 6.2 months). No reimbursement was given for surveys completed. This study reports on demographic data collected at baseline and implementation outcome data collected at follow-up.
About 2.5 years after the RCT’s launch, we conducted three focus group discussions and seven key informant interviews with study team members in South Africa and the U.S., using the same eligibility criteria. All invited individuals from the IMARA-SA study team in South Africa had completed the quantitative surveys except for one; U.S.-based IMARA-SA study team members had not been invited to complete the quantitative surveys. The first focus group was conducted with IMARA-SA study staff (n = 5), including facilitators of the IMARA-SA program and those involved in recruiting and transporting participants. A second focus group was conducted with clinical staff who conducted clinical assessments for the IMARA-SA study (i.e., HIV and STI testing and administration of PrEP) (n = 3). A third focus group was conducted with members of the DTHF Community Advisory Board (CAB) (n = 5), whose input on the project was first sought over five years ago. The CAB members were intimately familiar with the project, having provided input on the adaptation and testing of IMARA-SA through both the piloting and RCT phases. Focus groups were hosted in person at DTHF offices and lasted about 90 minutes. Key informant interviews (n = 7) were conducted with the study co-principal investigators (co-PIs), co-investigators, SA and U.S.-based project directors, and the study physician. Interviews were hosted in person or over Zoom and lasted about 30 minutes. CAB members were reimbursed 200 rand for their time and transport for participating in a focus group discussion. No reimbursement was given for study or clinic staff participating in focus groups or for key informants.
KM moderated the focus groups with IMARA-SA study and clinic staff and led the key informant interviews in English. KM and MA co-moderated the focus group with CAB members, which was hosted in English and Xhosa with simultaneous translation by MA. Focus groups and interviews were audio-recorded and transcribed verbatim. KM wrote analytical memos after focus groups and interviews to capture reflections on the data collection tools, interpret preliminary findings, and document the research process [Reference Saldana27,Reference Tobin and Begley28].
Frameworks and outcomes
Proctor’s implementation outcomes framework [Reference Proctor, Silmere and Raghavan19] informed our decision to examine six implementation outcomes: acceptability, appropriateness, feasibility, reach, adoption, and sustainability (Table 1). Data were also collected on fidelity and will be reported elsewhere.
Table 1. Implementation outcomes examined, with definitions
*Note: we drew on Glasgow et al.’s definition for reach [Reference Glasgow, Harden and Gaglio31] instead of Proctor’s outcome of penetration [Reference Proctor, Silmere and Raghavan19], given its greater applicability to delivery in a community versus service setting.
Quantitative measures and qualitative guides
Surveys included demographic questions and a question about the respondent’s familiarity with the IMARA-SA curriculum. Participants completed items from the mental health implementation science tools (mhIST), a valid and reliable instrument of implementation outcomes used across diverse populations in low- and middle-income countries [Reference Aldridge, Kemp and Bass32]. To maintain a reasonable survey length, we selected five of the most relevant items from the acceptability sub-scale (the sample size was too small to generate a Cronbach’s alpha), appropriateness sub-scale (Cronbach’s alpha = 0.93), feasibility subscale (Cronbach’s alpha = 0.68), and reach sub-scale (Cronbach’s alpha = 0.95) of the mhIST. We also included 11 items from the reliable Antiretroviral Treatment Access Study (ARTAS) [Reference Norton33] to measure the likelihood of adoption (Cronbach’s alpha = 0.79). Finally, we selected 17 items from the valid and reliable Program Sustainability Assessment Tool (PSAT) [Reference Luke, Calhoun, Robichaux, Elliott and Moreland-Russell34] to measure sustainability, including 5 items measuring environmental support, 3 items measuring partnerships, 5 items measuring organizational capacity, and 4 items measuring strategic planning (Cronbach’s alpha = 0.94).
Focus group and interview guides were tailored to participants and included open-ended questions prompting participants to expand on the implementation outcomes addressed in questionnaires (e.g., “What did you like about the IMARA sessions? What did you dislike about the sessions?” “To what extent did the recruitment team reach adolescent girls and their caregivers who could benefit from the program?”). The focus group with IMARA-SA facilitators probed further into views on the curriculum, while the key informant interviews with investigators probed more into perceptions of adoption and potential sustainability.
Analyses
Quantitative data were analyzed using descriptive statistics (e.g., means, standard deviations). To support interpretation, we generated a combined mean for several items from the same scale that addressed similar topics – i.e., for safety in the feasibility scale and financial implications in the adoption scale. We ran t-tests to test if mean differences across the outcomes varied by whether an individual had been supporting IMARA for less than one year compared to one year or more.
Qualitative data were analyzed using a rapid analysis approach [Reference Hamilton35,Reference Taylor, Henshall, Kenyon, Litchfield and Greenfield36]. A framework matrix was generated, based on the focus group and interview guides [Reference Hamilton35,Reference St. George, Harkness, Rodriguez-Diaz, Weinstein, Pavia and Hamilton37]. SA extracted data into the framework matrix, which was checked for accuracy by KM. Emerging themes were identified through extensive discussions between KM, SA, and MA and additional discussion with the full authorship team. We used data triangulation, merging the quantitative and qualitative data by implementation outcome [Reference Palinkas, Aarons, Horwitz, Chamberlain, Hurlburt and Landsverk26,Reference Tobin and Begley28]. During this process, quantitative and qualitative methods were equally valued, and both consistencies and inconsistencies across the methods were identified. Our goal was for completeness [Reference Tobin and Begley28], where the possibility of multiple realities was recognized in an effort to generate a holistic understanding of findings.
Ethical considerations
Written informed consent was obtained from all study participants. This research was approved by the Institutional Review Boards of the University of Cape Town (077/2019) and the University of Illinois Chicago (2018–0709).
Results
Participant characteristics and overall trends
Surveys were completed by 27 staff and collaborators (Table 2). Half of study staff (non-facilitators) (50%, n = 9), all of IMARA facilitators (n = 5), and 25% of DTHF staff/administrators (n = 1) reported knowing the curriculum “very well” with the remainder reporting knowing the curriculum “somewhat well” or “not well.”
Table 2. Characteristics of survey participants (n = 27)
Mean scores were highest for acceptability, appropriateness, and reach, followed by feasibility, adoption, and sustainability. Across quantitative findings, mean scores were highest among IMARA-SA staff (non-facilitators) and IMARA-SA facilitators compared to DTHF staff and administrators. We did not find evidence that mean scores across outcomes different by whether participants had supported IMARA for less than a year compared to one year or more. We proceed to unpack these results by implementation outcome.
Acceptability
Acceptability scores were high across roles, averaging 2.8/3.0 (SD = 0.6) (Table 3). Focus group participants and interviewees alike reported that AGYW and FC loved the program. The clinical staff focus group depicted participants as “feeling happy” and “laughing” during and after sessions. Focus group participants especially liked the program’s inclusion of separate and joint sessions for the AGYW and FC to give protected time for open discussion in each group, as well as the flexibility for AGYW to choose which caregiver they wanted to invite to the workshop. A participant from the CAB focus group said of IMARA-SA, “I wish it could continue for a lifetime.”
Table 3. Ratings of acceptability, appropriateness, feasibility, and reach in the full sample and by role
Abbreviations: IMARA = IMARA-South Africa.
Note: Items were adapted from the mental health implementation science tools (mhIST) [Reference Aldridge, Kemp and Bass32].
Scores were similarly high among IMARA-SA facilitators regarding the extent to which they liked and felt satisfied with the intervention and its delivery, the skills they learned, and their comfort answering participants’ questions (2.96/3.0, SD = 0.1). In their focus group discussion, IMARA-SA facilitators and study staff described five key components of the intervention based on their involvement with the adaptation of IMARA to a South African context: 1) basic information about STIs and HIV (e.g., a poster of common STIs, activities on risky behavior); 2) how to use condoms; 3) assertive communication between mothers and daughters; 4) relationship content addressing healthy versus unhealthy relationships, gender-based violence, and partner communication; and 5) content on self-esteem and culture, including the integration of poems and songs. These participants viewed these components of the intervention as highly acceptable and essential to the intervention.
Appropriateness
Appropriateness scores were also high across roles, averaging 2.7/3.0 overall (SD = 0.5) and 3.0/3.0 among IMARA-SA facilitators (Table 3). Based on survey responses, IMARA-SA was perceived to fit well with respondents’ cultural values, personal values, and the female culture in South Africa and was thought to address problems common in the area. Interviewees who were part of the two-year adaptation and piloting of IMARA-SA spoke about the systematic process through which the intervention was adapted to ensure its appropriateness among South African families. One interviewee described how the curriculum manages to walk a fine line between encouraging AGYW to express their needs and wants to their FC without undermining FC: “We tried to make sure that the curriculum can empower the mother but also enable the child to communicate well so that the two can build a good relationship.”
Scores were similarly high for perceptions that the site where IMARA-SA was delivered is a good place for participants to attend the program (2.9/3.0, SD = 0.5). Qualitative findings underscored the critical importance of delivering IMARA-SA in a community location accessible to the population of interest. The delivery site was thus perceived as highly appropriate for achieving reach (see further details below). An interviewee explained the reasons for why a community location was so critical: “It is a very, very vulnerable community here. We have very high STI rates, and the HIV rate is high.”
Feasibility
On the feasibility scale, scores were high across roles for the confidentiality of the space where IMARA-SA was delivered (2.7/3.0, SD = 0.5) and the perception that women in the community could seek HIV care from IMARA-SA without fear of how others would view them (2.6/3.0, SD = 0.6) (Table 3). Qualitative findings on recruitment and retention were positive. Recruitment conducted through DTHF community officers, non-governmental organizations (NGOs), other DTHF research studies, and participant referrals was considered highly successful. An interviewee praised IMARA-SA for its high retention, which the interviewee largely attributed to the participants’ positive experiences in the intervention and through interactions with the study team.
Compared to the highest-scoring feasibility items, scores were lower for having enough time to deliver the intervention (2.2/3.0, SD = 0.9). Echoing these quantitative results, the qualitative data revealed challenges with time management due to the research procedures completed prior to the intervention on Day 1 of the intervention, including enrollment, baseline surveys, and clinical procedures (i.e., HIV/STI testing). Numerous participants recommended separating the research procedures from Day 1 of the intervention because they affected participants’ engagement in the intervention:
Surveys, HIV testing, STI testing – it was a lot. In terms of the program itself, sitting for so long was a challenge. Because it is long, we would feel the girls and their caregivers would lose interest. By the end, they were so tired. (Interviewee)
Scores were comparatively lower for perceptions of safety (overall mean = 1.4/3.0, SD = 0.7), including perceived safety of the intervention site (1.3/3.0, SD = 0.9) and travel to/from the site (1.4/3.0, SD = 0.7) – themes which were echoed in the qualitative data. Focus group participants and interviewees, for instance, described the added pressure of completing all the activities on Day 1 before dark, due to safety concerns. Furthermore, the team had to find a workaround for the research procedures since the DTHF clinical team does not come to the site on the weekends, which is when many IMARA-SA sessions were delivered. A clinic team member explained, “Because of safety, we don’t come in on Saturdays. It’s a very, very unsafe area here.” The IMARA-SA facilitators and recruiters’ focus groups additionally noted that intervention delivery would be improved if the curricula and workbooks were in Xhosa and English and extra facilitators were trained as back-ups.
Reach
Scores for the intervention’s reach were high (2.7/3.0, SD = 0.5) (Table 3). In qualitative data, reach was considered one of the project’s greatest successes. Focus group participants and interviewees described very few challenges with enrolling over 640 dyads, particularly after the recruitment procedures were ironed out. A CAB member attributed high reach to the high need for the program offerings: “It means that IMARA-SA was a necessity.” Scores were similarly high for reaching the “poorest women” in the mhIST tool (2.8/3.0, SD = 0.5). Indeed, qualitative participants explained that the delivery site and the broad eligibility criteria facilitated access to a population at high risk.
Adoption
Mean scores for the adoption of IMARA-SA were 3.8/5 (SD = 0.3) (Table 4). IMARA-SA and its clinical procedures were viewed as filling a gap in the services offered in the area by addressing both behavioral and biobehavioral aspects of HIV/STI prevention. Interviewees highlighted the intervention’s family focus as a unique feature compared to other HIV interventions in the area, which have emphasized a biomedical focus in recent years. An interviewee explained, “The distinguishing or critical factor for IMARA is the involvement of families. IMARA wasn’t looking at a person from an individual perspective but actually from a structural, societal, family perspective.”
Table 4. Ratings of adoption in the full sample and by role
Abbreviations: IMARA = IMARA-South Africa; STI = sexually transmitted infection, HTC = HIV testing and counselling; PrEP= pre-exposure prophylaxis. Note: Items were adapted from the Antiretroviral Treatment Access Study (ARTAS) [Reference Norton33]. *Indicates that an item was reverse-coded.
The lowest mean scores for adoption concerned the complexity of the intervention (2.9/5, SD = 1.3) and financial implications of the intervention (overall mean = 2.8/5, SD = 0.9), including perceptions that IMARA is too expensive (3.5/5, SD = 1.0), requires too many human resources (2.9/5, SD = 1.1), and that funding would need to be provided for the respondent to adopt the intervention (2.1/5, SD = 1.2). Focus group participants and interviewees suggested that these perceived challenges concerned the complexity and cost of the research procedures for the project rather than the intervention itself. They highlighted the challenges with the clinical procedures, which coincided with Day 1 of the intervention, building on the feasibility challenges (described above).
Sustainability
Mean scores for the sustainability of IMARA-SA were 5.9/7 (SD = 0.8) (Table 5). The highest scores were for the perceived organizational capacity of DTHF, including existing leadership support and organizational systems to support ongoing intervention delivery (items 9–13, mean range = 6.0–6.2). Interviewees explained that DTHF was an ideal site to test the effectiveness of IMARA-SA in a community setting. However, DTHF is a research entity, and thus, the goal was to identify another venue for future delivery and sustainment. Qualitative data revealed numerous possibilities for the future delivery of IMARA-SA, including in schools, clinics, community organizations or NGOs, churches, or community halls. Participants promoted including the clinical procedures as part of the intervention going forward, since the intervention, STI and HIV testing, and provision of PrEP would go “hand-in-hand” (interviewee). Participants recognized, however, that integrating clinical testing and PrEP provision would increase the complexity and cost of the intervention (see the adoption section above) and may be less viable in school or community settings compared to clinic settings. Several participants proposed developing a mobile app for IMARA-SA to reach a wider audience.
Table 5. Ratings of sustainability in the full sample and by role
Abbreviations: IMARA = IMARA-South Africa. Note: Items were adapted from the Program Sustainability Assessment Tool (PSAT) [Reference Luke, Calhoun, Robichaux, Elliott and Moreland-Russell34].
Qualitative data also revealed interest in expanding IMARA-SA to other groups, including boys, men, and mixed-gender populations. An interviewee recommended involving the male partners of AGYW because many AGYW participants tested positive for STIs at follow-up even though they were treated at baseline. “It’s just so hard for our participants to have that discussion with the partner despite all the confidence and building that’s happening between the adolescent and caregiver.” Some described the importance of including male caregivers to support adolescent girls. As a participant in the CAB focus group explained:
You get men who play the mother’s role and they do not know how to speak to their daughter about certain things. It will create a conflict. If you can, also include fathers – a father and daughter thing. (Focus group participant).
Another interviewee expanded on the need to engage men and boys generally:
We target females for issues around STIs and HIV, but I feel like sometimes we neglect the men. They’re the ones with so much power in the relationships. Maybe if they can also be taught around assertive communication, the importance of using the condom, and not give that responsibility all the time to the women. (Interviewee).
Discussion
This study reports the perspectives of study staff, investigators, and collaborators of IMARA-SA, who are well-suited to reflect on the implementation outcomes of interest. We assessed a wide range of implementation outcomes, including reach and sustainability (which are often absent from the literature [Reference Proctor, Bunger and Lengnick-Hall20]), from a range of invested groups. We found that IMARA-SA addresses widespread health concerns facing AGYW, a hard-to-reach population. Mean scores were high across outcomes, particularly among IMARA-SA team members (i.e., facilitators and non-facilitators), and study participants discussed the value of the intervention in the community. These findings provide strong support for the implementability of IMARA-SA in the South African context and bode well for the program’s future delivery. We proceed to discuss three key takeaways while addressing the implications of our findings for implementation science researchers and practitioners.
First, our findings highlight tensions between achieving feasibility, delivery site appropriateness, and reach. Safety concerns posed a challenge for feasibility in that staff perceived barriers to participants attending the intervention given concerns about safety – both at the delivery site and in the surrounding area. Yet, at the same time, the delivery site was seen as appropriate and the program’s reach was one of its greatest strengths; by implementing in a community setting, the project was able to meet the population of interest where they live and recruit and retain AGYW living in highly vulnerable areas. Indeed, reaching South African AGYW is a recognized priority due to their vulnerability to HIV [Reference Chimbindi, Birdthistle and Shahmanesh38,Reference Gourlay, Birdthistle and Mthiyane39]. These findings have implications for future implementation science researchers in showing how the pursuit of different implementation outcomes might have trade-offs. In this study, the downside of safety concerns was outweighed by the perceived importance of implementing the intervention in a community setting and reaching AGYW and FC most in need. These findings expand on other examples in the literature of the trade-offs that implementation scientists must navigate [Reference Olswang and Prelock40,Reference Stensland, Sales, Damschroder and Skolarus41] (e.g., between fidelity and adaptation [Reference Allen, Shelton, Emmons, Linnan, Brownson, Colditz and Proctor42]). Indeed, achieving high scores across all implementation outcomes may not be the desired outcome for a project based on contextual realities. By grappling with these trade-offs, implementation science researchers and practitioners can advance not only implementation success but also the science of implementation [Reference Olswang and Prelock40].
Second, our findings illustrate the difficulty of isolating implementation experiences of an intervention when delivered in tandem with extensive research procedures. Compared to the highest scoring items, we observed lower scores for complexity and cost (measures of adoption) and time management (a measure of feasibility). However, the focus groups and interviews revealed that the lower scores on quantitative surveys were primarily a reflection of the research activities and should not be attributed to deficiencies in the intervention itself. Implementation researchers should be aware of the challenges among study participants of distinguishing between intervention and research activities and should strive to find ways of clarifying these distinctions, particularly in surveys. This challenge has been identified by researchers before [Reference Reynolds, DiLiberto and Mangham-Jefferies43] but is not often addressed in the implementation science literature. These findings clearly illustrate the value of using mixed methods in implementation research; they show how the qualitative data we collected provided a deeper understanding of the topics of interest than would have been achieved with quantitative data alone [Reference Palinkas, Aarons, Horwitz, Chamberlain, Hurlburt and Landsverk26].
Third, our findings provide important insight into the sustainability and future delivery potential of the intervention. Regarding sustainability, DTHF was perceived as compatible for evaluating IMARA-SA in a rigorous clinical trial, with strong organizational skills and mission alignment. Delivering the intervention through DTHF provided valuable insights into the intervention’s delivery potential over time given DTHF’s presence in the community as a trusted biomedical provider. However, DTHF is, by nature, a research entity. Hence, a new implementation avenue is needed to achieve IMARA-SA sustainability. Qualitative findings revealed multiple possible future directions for IMARA-SA, including through schools, clinics, NGOs, churches, or community halls. Most immediately, the intervention has been adapted to a Zambian population through the ZAIMARA project [Reference Mudenda44], building on the findings presented in this paper [Reference Chimbindi, Birdthistle and Shahmanesh38,Reference Merrill, Merrill and Hershow45]. While the intervention was perceived as empowering AGYW and their FC, there were suggestions to involve boys and men – and specifically the partners and male caregivers of the AGYW – given attitudes and norms that may impede using learned skills with partners to improve their sexual health. This finding echoes literature highlighting the importance of engaging men and boys in HIV prevention and gender-based violence programming to support women and girls [Reference Colpitts46–Reference Crooks, Yates and Sosina48]. In the U.S., an adaptation of IMARA for Black male caregivers and their girls is currently undergoing assessment [Reference Crooks, Debra and Coleman49], which could offer an opportunity for addressing the requests from participants in this study to engage male caregivers.
Study limitations must be noted. Five IMARA-SA staff did not complete follow-up surveys prior to leaving the study. These staff may have reported different implementation experiences than those who remained on the project, but they were not facilitators and only represent a small proportion of the study sample. We did not use the full scales for the implementation outcome measures. While the items we selected generally had acceptable to excellent Cronbach’s alpha, the alpha was lower for feasibility (0.67) and unable to be generated for acceptability given the small sample size. This may have limited the reliability of results for feasibility and acceptability, though this was mitigated in part thanks to the collection of qualitative data which provided a more complete understanding of findings. Additionally, the first author who led the qualitative data collection was the former U.S.-based project director for the pilot work leading up to the RCT, thus closely connected to the project and team and invested in the project’s success. Analytical memoing [Reference Saldana27] and extensive discussions within the study team, along with the involvement of a team member external to the SA study team (SA), helped ensure that the findings authentically represent the data [Reference Tobin and Begley28].
Conclusions
IMARA-SA staff, investigators, and collaborators reported positive perceptions of the implementation outcomes the intervention. Even in the presence of safety concerns related to delivery site, IMARA-SA was able to reach South African AGYW, a key population vulnerable to HIV. These findings reveal tensions between achieving high reach while maintaining feasibility and site appropriateness due to safety concerns. Researchers must grapple with potential trade-offs in implementing interventions for vulnerable populations. Moreover, our findings illustrate the difficulty of isolating implementation experiences of the intervention itself when delivered in tandem with extensive research procedures; our use of mixed methods proved essential to disentangling feedback on the intervention versus the research procedures. Our findings have already begun to inform future iterations of IMARA-SA and provide valuable insights that can strengthen the implementation of other HIV/STI prevention programs in the region.
Acknowledgements
We wish to thank the IMARA-SA staff and collaborators for their openness in sharing their perspectives in this study. We also thank the Center for Dissemination and Implementation Science at the University of Illinois Chicago (CDIS-UIC) for its support with carrying out this study.
Author contributions
Katherine G. Merrill: Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Visualization, Writing-original draft, Writing-review & editing; Millicent Atujuna: Conceptualization, Data curation, Methodology, Writing-review & editing; Saba Ahmed: Formal analysis, Visualization, Writing-review & editing; Erin Emerson: Data curation, Writing-review & editing; Anelisiwe Ngcuka: Data curation, Writing-review & editing; Erin Jaworski: Conceptualization, Data curation, Methodology, Writing-review & editing; Linda-Gail Bekker: Conceptualization, Writing-review & editing; Natasha Crooks: Writing-review & editing; Alyssa Debra: Writing-review & editing; Geri Donenberg: Conceptualization, Formal analysis, Methodology, Writing-review & editing.
Funding statement
The IMARA-SA study is funded by the National Institute of Child Health and Human Development and the Office of Behavioral and Social Sciences Research (UG3HD096875; UH3HD096875). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Qualitative data collection was funded using start-up funds (KM).
Competing interests
The authors declare that they have no conflicts of interests.
Open access
