STEM mentorship programs are designed to support women from career selection through to professional advancement and empowerment. Evaluating their impact is essential to assess effectiveness and guide evidence-based improvements. This study presents a systematic review of publications from the Latin American Women in Computing Congress (LAWCC) between 2020 and 2024, aiming to identify and classify the impact indicators used in university-level mentorship programs for women in STEM. A combined methodological approach was applied, integrating the PICO approach to formulate the research question and the PRISMA methodology to ensure transparency and rigor in the selection process. Of 62 papers reviewed, 5 fulfilled the inclusion criteria and were analyzed qualitatively using Atlas.ti. The analysis identified 17 impact indicators, categorized by evaluation type (qualitative, quantitative, or mixed) and impact dimension (psychological, academic, institutional, vocational, social, or attitudinal). The most frequently reported indicators were participation, motivation, student satisfaction, and vocational interest. To structure the identified indicators, the logical model by Wolf and Brenning was employed and adapted, organizing them into three levels: immediate outcomes, medium-term results, and long-term impact. This systematization proposes a new evaluation framework designed to assess the impact of STEM mentorship initiatives within the context of gender equity.
the Caribbean report [
Including more women in STEM is not only an ethical imperative but also a strategic necessity
for the region’s sustainable development. Multiple studies indicate that closing the gender gap in
STEM could boost regional GDP (Gross Domestic Product) while also enhancing innovation and
organizational competitiveness [
However, the effectiveness of many of these programs remains under-documented. Wolf and
Brenning [
In Latin America, the Latin American Women in Computing Congress (LAWCC), co-located with
the Latin American Conference on Informatics (CLEI), is one of the region’s most important
academic forums for presenting and discussing initiatives aimed at reducing the gender gap in
computing and STEM. Since 2020, LAWCC proceedings have been indexed in CEUR and Scopus,
reinforcing their value as a source of information on university-level mentorship programs across
diverse Latin American contexts [
The article is organized into four main sections. Section 1 presents the research problem and the context of university mentorship programs for women in STEM in Latin America. Section 2 outlines the methodology used, including the combined application of the PICO framework and the PRISMA methodology, as well as the use of Atlas.ti for qualitative analysis. Section 3 presents the results obtained. Finally, Section 4 discusses the findings, proposes recommendations for future evaluations of mentorship programs, and identifies gaps in the literature and opportunities to improve impact measurement in these types of initiatives.
For this systematic literature review of LAWCC articles published between 2020 and 2024, the
PRISMA methodology [
To structure the research question and guide data extraction, the PICO approach (Population, Intervention, Comparator, Outcome) was implemented, enabling a clear delineation of the analytical scope and precise definition of the core components of the study. This methodological approach also guided the coding and thematic extraction processes and facilitated the establishment of a common taxonomy for comparative analysis across the selected documents.
The research question formulated was:
RQ1: What impact indicators are reported in university mentoring programs for women in STEM, as presented in the LAWCC proceedings from 2020 to 2024?
The following components were defined: • P (Population): Women students in STEM fields involved in training, mentoring, or support interventions presented within the context of LAWCC. • I (Intervention): Mentoring programs, workshops, training initiatives, and institutional actions aimed at supporting academic and professional development to strengthen female participation in STEM. • C (Comparator): Not always applicable; in some cases, implicit comparisons were identified between cohorts, program phases, or baseline data in relation to the reported outcomes. • O (Outcome): Reported impact indicators, both quantitative and qualitative, such as academic retention, sense of belonging, leadership, empowerment, satisfaction, performance, participation, among others.
In addition, this research question gives rise to other relevant associated questions to be explored in this study, such as:
RQ2: Which university mentoring programs propose or measure the identified impact indicators in the selected articles?
RQ3: How can the identified impact indicators in the selected articles be classified to standardize and promote their future use?
RQ4: Are there any other relevant findings related to the impact indicators identified in the selected articles?
Specific parameters were defined to delimit the search and selection of studies:
Data source: The open-access CEUR-WS repository was used exclusively, specifically the proceedings of LAWCC from 2020 to 2024. This conference is a regional forum specialized in gender and computing, providing a representative sample of Latin American initiatives.
Inclusion criteria: Articles were selected if they fulfilled all of the following criteria: • Proposed or implemented mentoring programs, workshops, or interventions with a gender focus in STEM. • Included evaluative components or proposed measurable impact indicators. • Focused on higher education (university-level) populations. • Published between 2020 and 2024. • The mentoring programs, workshops, or interventions were developed within the Latin
American context.
Exclusion criteria: Articles were excluded if they: • Did not present evaluative components or failed to propose measurable indicators. • Focused on primary or secondary education. • Focused on social or community-based mentoring. • Were developed outside the Latin American context.
• Described technological development projects.
A total of 62 articles from LAWCC (2020–2024) were extracted based on the following criteria: year, country, type of intervention, evaluation instruments, evaluation indicators (proposed or used), and reported outcomes. The resulting PRISMA diagram is presented in Figure 1. Additional details of the PRISMA Checklist can be found in Appendix A.
Identified papers (n = 62)
Excluded after title/abstract (n = 35)
Evaluated full-text (n = 27) Excluded (n = 22)
Included in qualitative review (n = 5)
According to the application of the PRISMA protocol (Figure 1), the following process was done: a) Identification. A systematic search was carried out in the CEUR-WS.org repository, the official and comprehensive source for LAWCC publications, which meets the criteria of reliability and open access. All available articles published between 2020 and 2024 were retrieved: 13 in 2020, 9 in 2021, 11 in 2022, 19 in 2023, and 10 in 2024, totaling 62 unique records. b) Selection. Following a review of titles and abstracts, 35 articles were excluded for not meeting the inclusion criteria outlined in Section 2.2, resulting in a preliminary selection of 27 articles. c) Eligibility. The full texts of the remaining 27 articles were evaluated, and 22 were excluded for various reasons. Common reasons for exclusion included the absence of evaluation of specific mentoring programs; a focus on primary or secondary rather than higher education; reflective analyses or essays without evaluative components; bibliometric studies; literature reviews; and exploratory studies on motivation lacking concrete program evaluation. Additional exclusions were made for studies focused on professional entrepreneurship, technological development, or mobile application use that did not constitute university-level mentoring programs. Table 1 shows the 27 selected full-text papers organized by year of publication. More details about them in appendix B. d) Inclusion. Ultimately, 5 articles fulfilled all inclusion criteria and were incorporated into the qualitative analysis using Atlas.ti. Table 2 provides a summary of the selected articles and indicates that no articles from 2020 or 2021 met the established inclusion criteria. The number of identified indicators is also presented in the table.
Definition and Implementation of W-STEM Mentoring Network
[
Experience in the creation of the JUMI Community [33].
Challenging the gender gap in STEM with Python and Data Science [40].
Hack4women: In search of a framework [42]. 3. Results Country
Brazil Several Countries This section presents the results obtained in response to the primary research question (RQ1), as well as additional questions that emerged in the course of the analysis.
RQ1: What impact indicators are reported in university mentoring programs for women in STEM, as presented in the LAWCC proceedings from 2020 to 2024?
An analysis of the five selected articles revealed a total of 17 proposed or evaluated impact indicators. These include active participation in offered activities, sustained motivation toward STEM-related tasks or careers, and the development of a sense of belonging in traditionally maledominated spaces. Key elements also include the presence of support networks through mentoring, the strengthening of personal empowerment, and increased vocational interest in scientific and technological disciplines. Other relevant indicators include the availability of institutional resources, organizational awareness of gender gaps, and the consolidation of a scientific identity among participants. Additionally, it is considered important to assess indicators such as organizational climate, perceived academic self-efficacy, and student satisfaction—all fundamental to understanding the effectiveness of the implemented interventions.
At the quantitative level, identified indicators include academic performance, enrollment rates in STEM programs, development of leadership skills, academic program retention, and completion rates of the interventions. Together, these indicators provide a comprehensive view of the impact of mentoring programs and serve as a basis for guiding future evaluation and improvement strategies. A detailed definition of each identified indicator, the type of evaluation (qualitative, quantitative, or mixed), the corresponding impact dimensions (psychological/emotional, academic/educational, vocational/professional, institutional/contextual, participatory/social, and perceptual/attitudinal), and the related articles for each indicator can be found in Appendix C. This classification enables an integrated and multidimensional analysis of the impact of mentoring programs.
The frequency of use of the identified evaluation indicators across the five articles is presented in Figure 2. The most measured indicators were participation, motivation, student satisfaction, and vocational interest. In contrast, the least considered indicators included academic performance, leadership development, and completion rate.
s itr o a c d n I
Figure 4 displays the distribution of impact indicators by year. It is noteworthy that most of the indicators were proposed or measured in the selected works from 2022, except for academic performance and completion rate, which were not addressed that year. The analysis of the results gives rise to additional questions which will now be addressed.
RQ2: Which university mentoring programs propose or measure the identified impact indicators in the selected articles?
Although some articles [
Description
University initiative promoting female STEAM
participation through educational workshops, outreach
programs, and collaborative research in computing and
technical fields [
Three-way mentoring network improving women's STEM
retention in Latin American universities through peer
support and gender-sensitive training with adaptable
institutional guidelines [
Student-led community promoting women's technical careers through outreach, partnerships, and mentorship to inspire engineering choices and reduce gender disparities [33].
Competitive event addressing gender inequities in technical fields through collaborative research, problemsolving, and solution development to tackle women's career barriers [42].
Virtual inter-institutional program training women in programming and environmental data analysis through hands-on projects connecting computing with conservation and sustainability challenges [40].
Country
Brazil
Multi-country initiative spanning Latin America and
Europe
Type Evaluation Evaluation proposal Colombia
Evaluation Chile
Evaluation proposal Costa Rica
Evaluation
RQ3: How can the identified impact indicators in the selected articles be classified to standardize and promote their future use?
To analyze and categorize the impact indicators identified in the selected articles, the logic model
proposed by Wolf and Brenning [
In relation to immediate outcome indicators assess the initial implementation and reception of the mentoring program, forming the foundation for subsequent effects, these indicators measure operational aspects and the quality of the initial experience. With respect to medium-term result indicators, these assess psychological and attitudinal changes that occur during or immediately after the mentoring program. These changes function as mediators of long-term impact. Finally, the long-term impact indicators evaluate sustained changes in behavior, professional decisions, and identity that emerge after the completion of the program. More details about our proposal for an evaluation framework according to all these effects are described in Table 4, Table 5 and Table 6.
This proposed evaluation framework enables a comprehensive evaluation of mentoring
programs, recognizing that impact occurs through multiple causal pathways rather than in a linear
or monocausal manner [
Figure 5 illustrates how the 17 identified impact indicators align with the logic model proposed by Wolf & Brenning, highlighting which indicators fully align, partially align, or do not align at all based on whether they are explicitly considered in the evaluation framework proposed in this study. Additionally, Figure 6 presents how the 17 impact indicators could be classified according to our proposed evaluation framework, although not all are included, as determined by our qualitative analysis among indicators in the selected papers and with similar studies via atlas.ti. ! =MatchesWolf&Brenning " =Notmatched !*=Partiallymatched
RQ4: Are there any other relevant findings related to the impact indicators identified in the selected articles?
Based on the qualitative analysis conducted using Atlas.ti, several relevant findings emerged regarding the impact indicators identified in the selected papers. The key insights are as follows: Of the 17 impact evaluation indicators identified in the articles analyzed from LAWCC between 2020 and 2024, a total of 10 are included in the proposed evaluative framework adapted from Wolf and Brenning. These include: participation, empowerment, self-efficacy, motivation, satisfaction, sense of belonging, and leadership development, as well as conceptual equivalents of mentoring support networks (included as network building), scientific identity (as professional identity in STEM), and vocational interest (as STEM career intention). This alignment demonstrates a strong coherence between the proposed evaluation framework and the recent empirical evidence, supporting its relevance as a tool for assessing the impact of mentorship programs for women in STEM. The relationships among these 10 impact indicators identified in LAWCC 2020–2024 are illustrated in Figure 7.
In addition to the 10 indicators aligned with those identified in LAWCC (2020–2024), the proposed framework incorporates four additional indicators to strengthen the comprehensiveness of the evaluation (Table 3, Table 4 and Table 5). These include: frequency of contact and mentorship quality, which capture operational and relational aspects of program implementation; achievement expectations, which reflects students projected academic or career aspirations; and persistence in STEM, which assesses long-term engagement beyond initial intent. These indicators were added to address dimensions commonly emphasized in educational program evaluations and to ensure that both immediate-term interactions and long-term outcomes are adequately measured, enhancing the analytical depth of our proposal. 4. Discussion
To reduce selection bias in the article review process, predefined inclusion and exclusion criteria were applied according to the PICO framework. The search and preselection were guided by specific keywords present in titles, abstracts, and full texts, including: mentoring, women in STEM, impact, indicators, evaluation, training, leadership, empowerment, and related terms. A manual and exhaustive review of the complete corpus of LAWCC publications from 2020 to 2024 was conducted, ensuring full coverage of the analyzed universe. No automated filters were applied, as the corpus was limited and thoroughly reviewed manually. A standardized protocol for reading, coding, and data extraction was employed, supported by the Atlas.ti v9 software. This ensured objectivity in identifying impact indicators and consistency across documents. Additionally, the PRISMA flow diagram was used to document each phase of the process (identification, selection, eligibility, and inclusion), ensuring the traceability and replicability of the study. All exclusion decisions were fully justified based on the defined criteria, and methodological transparency was maintained throughout the analysis.
The results of this systematic review reveal a growing interest in evaluating the impact of mentoring programs targeting women in STEM within the Latin American context, particularly in the LAWCC articles published between 2020 and 2024. However, there is a notable lack of standardization in the indicators used, which hinders comparisons across initiatives and the consolidation of long-term learning. Unlike international frameworks such as that of Wolf and Brenning (2023), which propose structured and scalable logic models, the studies analyzed tend to rely on qualitative or mixed-methods approaches with limited longitudinal continuity or cohort tracking. This constrains the ability to measure sustainable outcomes over time. Moreover, the review highlights an urgent need to develop comparable evaluation frameworks across countries and institutions that incorporate validated indicators and facilitate the sharing of tools, methodologies, and results. Establishing a common taxonomy such as the one proposed in this study could support the design, monitoring, and continuous improvement of interventions, contributing to the reduction of gender gaps in the scientific and technological fields in Latin America.
To enhance replicability, this study outlines the adaptation of the Wolf and Brenning logic model by organizing 17 identified impact indicators into three levels: immediate outcomes (e.g., participation, satisfaction), medium-term results (e.g., motivation, self-efficacy), and long-term impacts (e.g., leadership development, STEM persistence). These indicators were systematically mapped through thematic coding using Atlas.ti and aligned with dimensions proposed in existing evaluation models for mentoring programs. While the framework is theoretically grounded, its practical relevance would be strengthened by the inclusion of applied case examples demonstrating its implementation in real institutional settings. Furthermore, measuring long-term impacts particularly in attitudinal and institutional dimensions presents specific challenges, including delayed effect manifestation, difficulties in longitudinal participant tracking, and the influence of contextual factors such as cultural or policy changes. In the absence of sustained follow-up strategies, longitudinal data remain limited, constraining the robustness of long-term impact assessments.
While this study applied rigorous inclusion criteria to ensure methodological consistency, the resulting sample size was necessarily limited, which may affect the generalizability of the findings. The focus on LAWCC proceedings chosen for their regional relevance and thematic alignment provided valuable insights but may have restricted exposure to a broader range of initiatives. Additionally, although the proposed evaluation framework is grounded in established theoretical models, it has yet to be tested in empirical contexts. The exclusion of non-university programs, while deliberate for analytical clarity, may have overlooked important data on indicators such as motivation and identity, which often emerge during earlier educational stages. These considerations highlight valuable opportunities for future research to validate the framework in practice and to expand the scope of analysis across educational levels and sources. 5. Conclusions and recommendations It is essential to establish indicators that enable the measurement of progress and/or outcomes of mentoring programs before, during, and after their implementation. These results should be periodically analyzed, reviewed, and communicated to all stakeholders to continuously improve the effectiveness of such programs. This systematic review identified a relevant set of 17 impact indicators used in university-level mentoring programs for women in STEM, with particular emphasis on those related to participation, motivation, student satisfaction, and vocational interest. While these findings demonstrate a growing effort to measure the effects of these interventions, they also reveal limited standardization in evaluation approaches and a lack of longitudinal continuity in the analyzed studies.
The adaptation of the logic model proposed by Wolf and Brenning [
Future research should focus on empirically testing and refining the proposed evaluation framework by applying it in active university-level mentoring programs and integrating it into the design of new initiatives. This work should also broaden the scope of analysis by incorporating institutional reports, national policy documents, and other relevant sources to strengthen the representativeness and relevance of the findings. Finally, expanding the framework to include primary and secondary education would allow for the creation of a regionally adaptable impact model that spans the entire educational continuum, providing a solid foundation for long-term strategies to advance gender equity in STEM fields across Latin America.
Declaration on Generative AI GenAI is used to reformulate sentences and improve clarity. Workshop Proc, vol. 3321, pp. 1–12, 2022, Accessed: May 26, 2025. [Online]. Available: https://ceur-ws.org/Vol-3321/paper1.pdf [30] V. Souza, M. Holanda, C. Koike, and A. Araujo, “An educational intervention for inclusion of deaf students in computing,” CEUR Workshop Proc, vol. 3607, pp. 1–10, 2023, Accessed: May 26, 2025. [Online]. Available: https://ceur-ws.org/Vol-3607/paper1.pdf [31] G. Espinoza, G. Barrios, S. Miguel, and E. Salvador, “Analysis of the inclusion of female students in STEM/ICT majors at the Gerardo Barrios University of El Salvador: a gender perspective,” CEUR Workshop Proc, vol. 3607, pp. 2–13, 2023, Accessed: May 26, 2025. [Online]. Available: https://ceur-ws.org/Vol-3607/paper2.pdf [32] R. García, A. Armendariz, J. Umpierrez, and C. Rattaro, “Code Huntresses: promoting gender equity in ICTs,” CEUR Workshop Proc, vol. 3607, 2023, Accessed: May 26, 2025. [Online].
Available: https://ceur-ws.org/Vol-3607/paper5.pdf [33] E. Hernández-Leal, G. P. Gasca-Hurtado, and M. Clara Gómez-Álvarez, “Experience in the creation of the JUMI Community: Achievement after two years,” CEUR Workshop Proc, vol. 3607, 2023, Accessed: May 27, 2025. [Online]. Available: https://ceur-ws.org/Vol3607/paper6.pdf [34] A. García-Holgado, S. Verdugo-Castro, V. V Ojeda-Caicedo, and S. H. Contreras-Ortiz, “International mobility for enhancing leadership in women engineering students: a case study connecting Colombia and Spain,” CEUR Workshop Proc, vol. 3607, 2023, Accessed: May 27, 2025. [Online]. Available: https://ceur-ws.org/Vol-3607/paper10.pdf [35] G. Cuascota, A. Guevara-Vega, L. T. Subía, and J. Caraguay, “Mobile application as a technological alternative for the inclusion of women in STEM areas: UTN Case Study,” CEUR Workshop Proc, vol. 3607, 2023, Accessed: May 27, 2025. [Online]. Available: https://ceurws.org/Vol-3607/paper12.pdf [36] [36] G. Reynosa and M. Aguilar, “Participation of university women under regulations and strategies promoting gender equity in STEM/ICT careers in El Salvador,” CEUR Workshop Proc, vol. 3607, 2023, Accessed: May 27, 2025. [Online]. Available: https://ceur-ws.org/Vol3607/paper13.pdf [37] J. Chicaiza, S. Segarra-Morales, G. Rodríguez, and F. Cevallos, “Perceptions of the gender gap in higher education and how to improve female students’ integration: A case study in two engineering majors of UTPL,” CEUR Workshop Proc, vol. 3607, 2023, Accessed: May 27, 2025. [Online]. Available: https://ceur-ws.org/Vol-3607/paper14.pdf [38] R. Fernández, E. Kremer, and A. Delgado, “‘ProgramADAmente’: educational and social platform to bring programming closer to girls and teenage girls,” CEUR Workshop Proc, vol. 3607, 2023, Accessed: May 27, 2025. [Online]. Available: https://ceur-ws.org/Vol3607/paper15.pdf [39] S. Luiz De Farias et al., “The Brazilian professional, scientific, and technological education system as an instrument for promoting gender equality in computing,” CEUR Workshop Proc, vol. 3607, 2023, Accessed: May 27, 2025. [Online]. Available: https://ceur-ws.org/Vol3607/paper18.pdf [40] M. B. Portillo, J. Castro Ramírez, M. M. Cross, K. R. Brenes, and T. C. Huang, “Challenging the gender gap in STEM with Python and Data Science: case of the National Technical University de Costa Rica,” CEUR Workshop Proc, vol. 3872, 2024, Accessed: May 27, 2025. [Online].
Available: https://ceur-ws.org/Vol-3872/paper9.pdf [41] L. San-Martín, A. García-Holgado, A. Dominguez, and U. Andres Bello, “Initiatives to Reduce the Gender Gap in Senior Management: Actions Developed by Organizations in Chile,” CEUR Workshop Proc, vol. 3672, 2024, Accessed: May 27, 2025. [Online]. Available: https://ceurws.org/Vol-3872/paper3.pdf [42] G. Costa, A. García-Holgado, and P. P. Alvarez, “Hack4women: In search of a framework,” CEUR Workshop Proc, vol. 3872, 2024, Accessed: May 27, 2025. [Online]. Available: https://ceurws.org/Vol-3872/paper4.pdf A. PRISMA 2020 Checklist
The complete PRISMA 2020 checklist can be accessed at: link B. Full-text reviewed articles
The detailed list of reviewed articles is available at: link C. Classification and definition of mentoring impact indicators by dimensions and evaluation type For the complete classification of impact indicators, see: link