Computing Transfer Students Through Near-Peer Mentoring: Insights from Co-Design Studies for a Peer-Mentoring Support System

Abstract

A diverse STEM workforce is crucial for national competitiveness. To broaden participation of students from diverse backgrounds in computing education and improve learning outcomes for transfer students-specifically in terms of sense of belonging, self-eficacy, and STEM persistence-we are piloting a peer mentoring program leveraging near-peer mentoring and horizontal peer supports. The mentoring program involves triads comprised of one mentor and two mentees. To support participants' academic, social, and program engagement, we are designing and developing a peer-mentoring support system based on student-centered learning analytics dashboards with features that promote the regulation of learning. In this paper, we report the results from two sequential co-design workshops with 10 peer-mentoring program participants from diverse backgrounds, where the second workshop built upon the findings of the first. The study revealed unique insights from both mentors and mentees regarding goal tracking and management, data privacy and transparency, social functions, and accessibility. This research contributes to the limited representation of human-centered design studies aimed at supporting peer mentoring programs with diverse student populations within complex co-regulation social structures.

Keywords

• Peer-mentoring
• co-regulation
• learning analytics dashboard

1. Introduction

For a nation to remain competitive in science and technology, it is essential to have a diverse STEM workforce. However, in the United States, women and racial/ethnic minority groups continue to be underrepresented in computing majors at two- and four-year institutions [ 1 ]. In the United States, nearly half of all STEM undergraduates start their studies at 2-year community colleges [ 2, 3 ] due to their afordability, convenient locations, and flexible class schedules and delivery options [ 3, 2, 4 ]. The transition to four-year institutions is often met with challenges. Transfer students need to adapt to new academic and social environments while handling the demands from advanced upperlevel coursework. Emerging evidence suggests that mentoring can act as a protective factor via increasing confidence, a sense of belonging, and retention rates in STEM fields among underrepresented students [ 5, 6, 7 ]. The context of this study is a pilot mentoring program for new transfer students in computing majors (Information Systems) in a four-year, mid-sized minority-serving institution (MSI) in the Northeastern US. This program explores a novel mentoring structure that leverages near-peer mentoring and horizontal peer support [ 8 ]. A triad of transfer students forms a small cohort, which includes a near-peer mentor who is a transfer student who has completed at least one semester posttransfer and at least two semesters from graduation, and two mentees who are new transfer students in their first semester or newly switched to the computing majors. In addition, students are supported by

CEUR

ceur-ws.org instructional and program staf throughout the program via programming to enhance academic success and social engagement. The program started in spring 2023 and has since enrolled a total of 10 mentors and 16 mentees in two semesters, with the majority belonging to underrepresented minority groups. This diverse pool of participants in the peer mentoring program forms the basis for the recruitment of subjects for our design studies.

The complexity of the mentoring program’s social structure involving multiple stakeholders and the desire of the program to enhance students’ experience calls for a robust online infrastructure that supports students’ social and academic engagement in their new environment to complement the existing program structure. In this paper, we report a design study towards a peer mentoring support system that features a Learning Analytics Dashboard to support the enactment, tracking, and management of goals essential for mentees’ successful integration into their new environment and persistence in computing majors. Unlike most existing LA systems that focus on self-regulated learning [ 9 ] for individual students, our work explores learning analytics within a social context involving multiple stakeholders, including peer mentors and mentees. To our knowledge, this is one of the first human-centered design studies towards a learning analytics system that supports a peer mentoring program using participatory and co-design approaches. The study surfaced desired features from mentors’ and mentees’ perspectives in a data-driven co-regulation context.

Contribution This work contributes to the growing body of literature on student-centered learning analytics (LA) systems developed through a human-centered approach [ 10 ]. Specifically, it addresses the relatively limited research that actively involves student stakeholders in shaping the design and development of LA systems, rather than restricting their role to providing usability feedback [ 11 ]. By inspiring students in co-design activity involving personalized goal setting, and management, this work also contributes to the broader discussion of emerging inquiries in strength- and growth-based learning analytics [ 12 ].

2. Methods

2.1. Participants and Setting

The design workshop participants were recruited via email from those enrolled in the peer-mentoring program. To participate, both mentors and mentees were required to be undergraduate students, 18 years of age or older, who transferred to the university from a 2-year institution. Additionally, mentors were required to have been in a computing major for at least one semester, in good academic standing, and at least 2 semesters from graduation. Mentees were required to have been either newly transferred to the university or newly switched from a diferent computing major. Table 1 in the Appendix gives an overview of the recruited workshop participants and the overall participation pool of those enrolled in the peer mentoring program. The workshops were conducted in a lab equipped with a Smart Board1 on which the prototypes were shown to participants. Before beginning the workshop, the room was set up with paper, markers, colored pencils, pens, and stickers for dot voting. The study was approved by the university’s institutional review board.

2.2. Overview of the Design Process and Data Collection

The primary objective of developing a peer system to empower mentors and mentees to collaboratively monitor the mentee’s academic progress while strengthening the mentor-mentee relationship to enhance social engagement. The proposed dashboard is designed to support the articulation and management of goals related to academic, social, and peer mentoring programs. The dashboard design is grounded in the theoretical frameworks of self-regulated learning [ 13 ] and socially shared regulation of learning [ 14 ]. These frameworks guide the platform to support learning outcomes that encompass both cognitive 1SMART Technologies. ”Education.” Accessed September 22, 2024. https://www.smarttech.com/en/education. goals, such as persistence in computing, and non-cognitive factors, such as self-eficacy and a sense of belonging within computing majors.

We began the design process by developing the first iteration of the dashboard with input from mentoring program staf who are actively involved in supporting mentoring activities throughout the program. The initial prototype also drew on prior work [ 15, 16 ]. A key element of mentoring is the mentor’s role as an accountability partner, supporting the mentee in goal setting, tracking, and management which are critical self-regulation activities while adapting to the new social and academic environment post-transfer. The dashboard design is tailored to the roles of both mentors and mentees, with most features shared between the two views to support the setting, tracking, and monitoring of academic, social, and program engagement goals.

We conducted the first workshop based on the initial prototype, recruiting 2 mentors and 1 mentee from a small pool of peer mentoring program participants in Spring 2023. During the workshop, participants engaged in a co-design process, where they created paper sketches of their “ideal” system and identified additional needs through consensus voting.

Insights from this workshop informed the development of a second prototype iteration, which was tested with a larger group of participants in the second workshop. In addition to prototype testing, participants engaged in a co-design activity where they designed creative visual representations of goal-setting and tracking mechanisms, moving beyond common progress bars as assumed in the first workshop. This activity was inspired by observations from the first workshop, which highlight the central role of goal tracking in the peer-mentoring platform, which functions as a third “voice” [ 17 ] between those providing support (mentors) and those receiving it (mentees). Due to space constraints, for the rest of the paper we focus on the second workshop. Interested readers may refer to this paper [ 18 ] for details on the first workshop.

Table 2 in the Appendix provides an overview of the 2nd workshop activities, duration and participation mode, and data collected. The workshop was audio and video recorded using two GoPro Hero7 cameras. One camera was placed on the table to capture design artifacts, and the second camera was placed near the corner of the room to capture all the participants as they engaged in the various workshop activities. After participants signed informed consent forms, the ice-breaker session began with introductions, followed by an explanation of the study, and an ice-breaker design activity. Participants then engaged in prototype testing in which they completed a concept testing survey tailored to evaluate the wireframe developed based on feedback collected from the first workshop. In the subsequent visualization creation session, participants developed creative visualizations to represent goal tracking and management, using a think-pair-share approach to generate, discuss, and refine their ideas, with visual aids provided to inspire out-of-the-box thinking. They sketched their ideal visualization after which participants voted on the preferred features in their paper sketches. The workshop ended with a debrief where participants shared reflections from the workshop.

3. Data Analysis

We employed a basic qualitative analysis to identify themes and patterns in the data. To begin, speech from the GoPro videos was transcribed using Otter.ai and error-corrected by the first three authors. Researcher notes were hand-coded to annotate and complement the transcribed data. The first 3 authors met virtually several times throughout the analysis process. To begin, each researcher independently coded the prototype testing transcripts, focusing on the research questions. The team reconvened approximately one week later to discuss, refine, rename, and collapse codes as necessary. The team then individually re-coded the prototype testing transcripts according to the developed codebook. One researcher reviewed the re-coded transcripts to ensure consistency. The same codebook was then used to code the transcripts from the remaining workshop sessions. The team then met again to come to consensus on the codes and to identify the overarching themes: Mentors’ and Mentees’ perspectives on goal tracking and management, visual representation of goal tracking, feature feedback and suggestions, and accessibility consideration. Videos were selectively reviewed for the referenced artifacts and to add

4. Summary of Results

In this section, we summarize the findings on goal tracking and management and additional functions of the mentoring support system from mentors and mentees’ perspective.

Mentor’s perspective Mentors noted the need for specific data to efectively track and support their mentees’ progress. The majority of them wanted to achieve this by visualizing trends in their mentee’s performance. Seeing trends like changes in academic performance and levels of social and program engagement over time would help mentors quickly spot both strengths and areas of concern. This would enable them to intervene, if required, at an early stage, and provide targeted support. Access to the list of goals was another feature recommended by mentors. Once goals are created for the semester, mentors should have the ability to modify or add new goals, including detailed descriptions and task breakdowns. To foster meaningful mentor-mentee engagement, the participants noted that the dashboard should be updated regularly to reflect the mentee’s progress, notifying the mentors of the same. Notifications should be clearly visible and designed with strong visual cues to ensure mentors receive timely updates on their mentees’ progress.

Mentees’ perspective Diverse views on goal tracking and management mechanisms emerged from mentees. Some voiced specific needs for multiple views of goals and preferred them integrated with progress tracking and deadlines with personal to-do lists. We also note that mentors and mentees need to negotiate the data-sharing protocol between the two parties. While mentors would prefer a high level of transparency regarding mentees’ progress in their goals, mentees would like to be granted lfexibility and control to choose whether to share private information details with their mentors, such as course grades and academic progress.

Sense-making of Goals and Goal Management: At the outset of the prototype testing session, although the initial reaction to the dashboard was mostly positive, some mentees and mentors were unclear about the utility and meaning of some of the platform’s elements, suggesting confusion over how academic versus goal progress was displayed. Some were also unsure of the specific purpose of the progress bars due to the lack of suficient textual cues.

Accessibility Considerations One of the mentees, who self-identified as a neurodivergent student, voiced the challenges faced by users with ADHD in interpreting the data and provided viable solutions, for example, providing both a snapshot view and a more detailed section for goal management so that both mentors and mentees can stay on track.

5. Discussion

In this paper, we reported insights from a human-centered design for a peer-mentoring support system using co-design methods. The two design workshops engaged a total of 10 participants recruited from a small but growing near-peer mentoring program piloted in a minority serving institute, aiming to promote the academic, social and program engagements and support the successful transition to 4-year college for students from 2-year institutes in computing majors.

To our knowledge, this study is one of the first co-design studies for peer-mentoring support systems in the learning analytics community, with specific focus on regulation tasks such as goal tracking and management as well as the social support functions that are aligned with the program goals. In this section, we highlight a few lines of insights, hoping to shed light on the specific LA design considerations for peer-mentoring programs, a complex context involving multiple stakeholders. Value of Human-Centered Design Unlike other methods, such as surveys, that are used to elicit users’ perspectives, human-centered design methods like co-design engage students with lived experiences. From our experience, this approach has the advantage of efectively capturing students’ authentic voices. For example, we observed that design features often emerged from frictions arising in mentor-mentee interactions, such as missed scheduled meetings, as well as from a desire to leaverage an efective shared third “voice” [ 17 ] to facilitate goal tracking and management. The design workshops also highlighted the need for intuitive, engaging, and sometimes gamified visual representations of goal-tracking mechanisms, which may improve students’ engagement with the system, thus increasing the likelihood of enhanced academic, social and program engagement. In addition, the study highlights the importance of features that turn goal-tracking data into concrete steps, which enable students to act on the current goal status, if needed. This includes tools that support the articulation and display of subgoals and explicit task completion aids, such as to-do lists.

Value of Personalization and Customization Consistent with the learning analytics literature on the design of student-centered systems, which cannot adopt a ”one size fits all” approach [ 19 ], we found that students prefer dashboards that are customizable, for example, using widget-based frameworks. Additionally, we observed slight diferences in focus on features between mentors and mentees regarding goal tracking and management. Mentors often take a proactive approach, seeking to be informed of mentees’ progress and receiving system-pushed notifications. In contrast, mentees desire more control and flexibility over data sharing, including transparency about what information is shared and with whom. Rather than a single-party decision, customization or personalization requires negotiation of control distribution among multiple stakeholders, particularly between mentors and mentees. These groups often have difering views on the trade-ofs between the benefits of data sharing and the risks associated with the loss of privacy regarding sensitive information, such as grades. Important Roles of Supporting Social Functions For social structures like peer-mentoring programs, academic, social, and program engagement are intertwined and rely on robust online peermentoring support systems. This is especially critical for commuter students, who often have limited opportunities for in-person interactions. Online social functions thus play an important role in supplementing these limited interactions. Features that support direct mentor-mentee interactions, which were not part of the initial design of the learning analytics dashboard, frequently emerged in discussions, highlighting their practical importance. These features help weave together a social fabric that supports new transfer students as they navigate a new academic and social environment.

Scafolding and Supports for Mentors From the design studies, we identified an implicit need to provide scafolding support for mentors. Although the program ofers training for mentors, translating this knowledge into efective practice can be challenging, particularly when mentees’ goals are of track. Since mentors are only one semester ahead of their mentees, their self-regulation profiles may difer, which can create additional obstacles. Ideally, the platform could provide just-in-time support, such as ofering tips on the types of conversations that are beneficial or should be avoided when a mentee appears disengaged. With new technologies like generative AI, the development of a contextualized coaching support tool is becoming increasingly feasible.

6. Limitations and Future Work

Due to the relatively small scale of the pilot peer mentoring program, we were unable to recruit real triads of participants, which include one mentor and two mentees. Consequently, the design focus has primarily been on mentor-mentee interactions, missing the important dynamics between mentees themselves. Additionally, an unexpected accessibility issue raised by a self-identified neurodiverse student highlights a new dimension of design requirements. For an information-rich dashboard that demands frequent engagement, it is crucial to deliberately consider accessibility aspects, such as adopting LA design principles for everyone, including neurodivergent students, as suggested in co-design studies such as [ 20 ].

7. Conclusion

A diverse STEM workforce is essential for a nation’s competitiveness. Near-peer mentoring has shown promise in engaging students from diverse backgrounds and enhancing long-term learning outcomes, such as a sense of belonging, self-eficacy, and persistence. A robust peer mentoring support system that features goal tracking and management, along with additional functions for academic, social, and program engagement, is crucial to the efectiveness of peer mentoring programs. The insights generated from this human-centered design study not only provide a foundation for future development but also contribute to the emerging literature on designing student-centered, regulation-oriented learning analytics dashboards. When fully-developed and validated, we envision such dashboard systems can support students’ success in peer mentoring contexts and beyond.

Acknowledgments

This material is based upon work supported by the National Science Foundation under Grant No. 2216633.

Generative AI Declaration

During the preparation of this work, the authors used ChatGPT, Grammarly in order to: Grammar and spelling check, Paraphrase and reword. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content.

A. Appendix 1: Additional details on participants

Work

shop #1 #2

Semester

Spring 2023 Spring 2024 # of Peer Mentoring Pro- # of Recruited Participants in Design gram Participants Workshops 2 (Mentor), 2 (Mentee)

Mentor: 2 (2M), Mentee: 1 (1F) 8 (Mentor), 14 (Mentee)

Mentor: 4 (2M + 2F), Mentee: 3 (2M + 1F)

B. Appendix 2: Additional details on design workshop activities

Workshop Activi- Duration ties

Ice-breaker 15 Minutes Prototype Testing 30 Minutes Visualization Cre- 40 Minutes ation Dotmocracy 5 Minutes

Data collection

Whole group Individual Think-pair-share [ 21 ] Individual Paper Artifacts and video recording Video recording Paper Artifacts, Video recording Paper artifacts, Video recording

References

[1] P. C. of Advisors on Science, Technology, Engage to excel: Producing one million additional college graduates with degrees in science, technology , engineering, and mathematics, 2012 .

[2] J. B. Labov , Changing and evolving relationships between two-and four-year colleges and universities: They're not your parents' community colleges anymore , CBE-Life Sciences Education 11 ( 2012 ) 121 - 128 .

[3] S. J. Handel , Improving student transfer from community colleges to four-year institutions: The perspectives of leaders from baccalaureate-granting institutions ., 2011 .

[4] S. Freeman , P. Pape-Lindstrom , A. Casper , S. Eddy , Community college students rise to the challenge-meeting the time demands of highly structured courses , Journal of College Science Teaching 49 ( 2020 ) 7 - 16 .

[5] B. London , L. Rosenthal , S. R. Levy , M. Lobel , The influences of perceived identity compatibility and social support on women in nontraditional fields during the college transition , 2011 .

[6] R. Robnett , The role of peer support for girls and women in stem: Implications for identity and anticipated retention ., 2013 .

[7] A. M. Zaniewski , D. Reinholz , Increasing stem success: a near-peer mentoring program in the physical sciences , International Journal of STEM Education 3 ( 2016 ) 1 - 12 .

[8] E. Spiridon , L. K. Kaye , R. I. Nicolson , H. J. Ransom , A. J. Tan , B. W. Tang , Integrated learning communities as a peer support initiative for first year university students , Journal of Applied Social Psychology 50 ( 2020 ) 394 - 405 .

[9] B. J. Zimmerman , Self-regulated learning and academic achievement: An overview , Educational psychologist 25 ( 1990 ) 3 - 17 .

[10] S. Buckingham Shum , R. Ferguson , R. Martinez-Maldonado , Human-centred learning analytics , Journal of Learning Analytics 6 ( 2019 ) 1 - 9 .

[11] S. Buckingham Shum , R. Martínez-Maldonado , Y. Dimitriadis , P. Santos , Human-centred learning analytics: 2019 - 24 , 2024 .

[12] A. Wise , F. Li , Exploring strength-and growth-based learning analytics: Possibilities and challenges for new data, models and tools ., in: LAK Workshops , 2024 , pp. 295 - 301 .

[13] A. van Leeuwen , S. D. Teasley , A. F. Wise , Teacher and student facing learning analytics, Handbook of learning analytics ( 2022 ) 130 - 140 .

[14] A. Hadwin , S. Järvelä , M. Miller , Self-regulation, co-regulation, and shared regulation in collaborative learning environments, in: Handbook of self-regulation of learning and performance , Routledge , 2017 , pp. 83 - 106 .

[15] J. P. Sarmiento , F. Campos , A. Wise , Engaging students as co-designers of learning analytics, in: Companion proceedings of the 10th international learning analytics and knowledge conference , SoLAR Frankfurt , 2020 , pp. 29 - 32 .

[16] E. de Quincey , C. Briggs, T. Kyriacou , R. Waller , Student centred design of a learning analytics system , in: Proceedings of the 9th international conference on learning analytics & knowledge , 2019 , pp. 353 - 362 .

[17] A. F. Wise , Designing pedagogical interventions to support student use of learning analytics , in: Proceedings of the fourth international conference on learning analytics and knowledge , 2014 , pp. 203 - 211 .

[18] N. Anthraper , P. Javiya , S. Iluru , L. K. Chen , A. Kleinsmith , Peerconnect: Co-designing a peermentoring support system with computing transfer students , in: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems , 2024 , pp. 1 - 7 .

[19] S. D. Teasley , Student facing dashboards: One size fits all?, Technology, Knowledge and Learning 22 ( 2017 ) 377 - 384 .

[20] V. Bayer , P. Mulholland , M. Hlosta , T. Farrell , C. Herodotou , M. Fernandez , Co-creating an equality diversity and inclusion learning analytics dashboard for addressing awarding gaps in higher education , British Journal of Educational Technology 55 ( 2024 ) 2058 - 2074 .

[21] M. Kaddoura , Think pair share: A teaching learning strategy to enhance students' critical thinking ., Educational Research Quarterly 36 ( 2013 ) 3 - 24 .