In the dynamic landscape of education, the integration of Artificial Intelligence (AI) has become increasingly apparent. This experience report explores how students can be guided to use AI in the context of writing requirements, all within a bachelor module on Requirements Engineering at a Higher Education Institute (HEI). The study subject was a student assignment, graded using an evaluation rubric. Further insights were gathered through a questionnaire. Students critically evaluated AI-generated requirements, assessing their quality and alignment with boilerplates and quality criteria. The findings revealed diverse student approaches to AI interaction in requirement writing. Students underscored the pivotal role of prior Requirements Engineering knowledge acquired in the classroom, which served as a foundation for guiding and instructing the AI. They recognized AI as an invaluable supplementary tool rather than a complete replacement, emphasizing its role as a sparring partner. The questionnaire corroborated that AI already occupies a significant space in students' personal and academic lives, underscoring the necessity of guiding its utilization within higher education. This experience report presents a comprehensive perspective on integrating AI into Requirements Engineering Education (REE), particularly concerning writing requirements. While acknowledging its limitations, including the small sample size and the absence of a control group, it prompts future research endeavors to refine AI integration strategies and explore the broader implications of AI within REE.
In the dynamic landscape of education, the integration of Artificial Intelligence (AI) has become increasingly apparent. Also, within Software Engineering Education, it has emerged as a vital topic. Daun & Brings [1], who examined the impacts of generative AI, particularly ChatGPT, on software engineering education, address concerns about its potential for cheating while also highlighting the opportunities AI offers. They advocate for a balanced approach that leverages AI for educational enhancement while providing necessary guidance and oversight.
A year ago, in the spring semester of 2023, I suspected some of my Requirements Engineering module students had already used AI to solve their assignments. Centered on a case study involving a smart home system, the course required students to write requirements that conformed to specific boilerplates for user interaction and fulfilled quality criteria such as atomicity. Some student submissions contained phrases and sentence structures that deviated from the conventional boilerplate structures taught in class but included unusual sentence constructs. The origin of this anomaly became apparent after I participated in a lecturer's workshop and experimented afterward on my own. When trying out ChatGPT, I encountered the exact "odd" phrases I noticed in my students' submissions. Although I never had proof, I assume the students were leveraging ChatGPT while unreflectively accepting the AI-generated outputs without consulting the provided learning materials.
This pivotal shift in the educational process of the RE module led to a significant change. Instead of imposing a ban, the focus shifted to integrating AI into the curriculum. The goal was to enlighten students about the benefits and potential pitfalls of AI assistance in requirement engineering, echoing the sentiments expressed by Daun & Brings [1] regarding the necessity of incorporating these evolving tools in a controlled, educational context. Adding a requested reflection component to the assignment should foster the students' critical thinking, an essential 21st-century skill [2].
This experience report explores this integration, examining how students can be guided to use AI to enhance their learning experiences in Requirements Engineering while remaining critically aware of its limitations. The purpose is to prepare students for a tech-driven future by fostering reflected, guided usage of AI and cultivating a nuanced understanding of AI's current role, benefits, and limitations in different areas of RE. This report delves into the educational design of an example of AI integration in REE, its results, and the broader implications for instructional and curricular designs.
The integration of AI in the course took place in the last semester (autumn 2023). The "Requirements Engineering" course module is offered every semester as a mandatory course within the Business Informatics bachelor's program. The students typically take this course module during their third semester if they are enrolled full-time (6 semesters) or during their fourth semester if they are enrolled part-time (8 semesters). By this point, students have already completed a programming course but have not received specific prompt engineering classes. Alongside adjustments to the instructional design of the student's assignment, the evaluation rubric was also revised, and a questionnaire was created. The questionnaire was designed to gather feedback on students' previous experiences with AI and their perspectives on its application in this educational context.
Before the AI-assisted requirement writing assignment, students were introduced to requirement templates in class. They had opportunities to independently practice using these templates on various cases, followed by class discussions to share and critique their solutions. This phase aimed to provide students with a solid foundation in requirement writing using templates. In a later class session, I demonstrated AI prompting using ChatGPT 3.5 with a different case study as an example.
The study focused on a mandatory, graded practical assignment in which students were tasked with employing AI software to develop a requirement sentence describing an independent system activity for a smart home system. The activity should include a conditional trigger (e.g., If / As soon as / While). An essential aspect of the task was the requirement for students to document their prompts with the AI, critically assess the AI-generated requirement, and reflect on its quality regarding adherence to a predefined sentence template and specific quality criteria. Each student was required to complete the assignment individually and was graded accordingly.
Central to this assignment was the application of the 'MAsTER' (Mustergültige Anforderungen -die TEmplates für Requirements) boilerplates [3], a set of templates designed for drafting various types of requirements. Students were expected to critically evaluate the AIgenerated requirement against these templates and assess its alignment. Following this evaluation, students identified areas for improvement and were challenged to refine and rework the AI-produced requirement. This process necessitated a robust understanding of the principles of effective requirement writing and the skill to integrate AI-generated content with these established standards.
The final deliverable should be a polished requirement sentence, showcasing the students' ability to merge AI tools with critical evaluation skills. This assignment served not only as an introduction to the use of AI in Requirements Engineering but also highlighted the importance of adhering to proven templates and guidelines within the field. While the demonstration in the class utilized ChatGPT 3.5, students could choose any AI software they deemed appropriate for the task.
The evaluation of student submissions for the AI-assisted Requirements Engineering assignment was guided by a rubric. This rubric was designed to assess the effectiveness of AI integration in creating a requirement sentence for a smart home system scenario. The main evaluation criteria included adherence to the 'MASTER' framework and the quality criteria atomicity and clarity. The rubric graded submissions on a scale from 0 to 50 points. Higher scores required clear, focused documentation of AI dialogues, a critical reflection identifying any shortcomings in the AI-generated requirement, and an optimized final requirement that met all specified criteria. Lower scores indicated varying levels of proficiency and completeness. This concise rubric should allow an objective yet focused assessment of each student's submission, emphasizing the final product and the critical reflection.
Supplementing the practical assignment, a questionnaire was administered to the students. This questionnaire aimed to capture their experiences and perspectives regarding AI use in their academic life. It inquired about their prior experience with AI, their opinions on the utility of AI in writing requirements, and their reflections on the integration of AI into Requirements Engineering education. This methodological approach provided specific insights into how AI tools can be effectively utilized in teaching students to write requirements, emphasizing the importance of critical reflection in their learning process.
Different AI tools were utilized by students for the assignment, with ChatGPT emerging as the most popular choice, reflecting their diverse expertise (see Table 1).
Distribution of AI Tools Among Students While some students provided the AI with clear instructions on the specific type of requirement needed, others used it first as a brainstorming partner, allowing it to suggest potential requirements for a smart home system. The majority attempted to teach the AI, feeding it templates and criteria, even though the AI's understanding of frameworks like the 'event master template' was not always accurate. A standard critique involved the structure of the sentences, where conditions were often placed at the end instead of the beginning, and incorrect phraseology used by the AI. However, the students were perceptive in recognizing these issues and added their refinements to the final requirement sentence.
In choosing topics for the requirements, seven students (38.9%) proactively suggested topics for their requirements. Conversely, the remaining students relied on AI to provide them with topic suggestions, with the condition that the topic should be within the context of a Smart Home.
Student prompting behavior varied significantly, with the majority employing a single prompt (44.4%) and a few opting for multiple prompts, as outlined in Table 2.
In summary, the students' interactions with AI revealed diverse approaches to engaging with AI to fulfill their assignment of writing requirements. These findings provide valuable insights into how students utilize AI as a supplementary tool in requirement writing, underscoring the importance of clear guidance and prior knowledge.
Interestingly, while some students found that writing requirements with AI, although supplemented by teaching and personal final touches, was practical, others expressed frustration. One student candidly remarked on this experience: 'It is more laborious to write the many prompts required to guide the AI towards a useful answer than to write the requirement directly myself.' Moreover, one student shared, 'ChatGPT could not initially create a formulation according to the template. It was only after I explained more details about the template that ChatGPT provided an acceptable answer. The formulation was still flawed. Therefore, I will refrain from using ChatGPT in the Requirements Engineering module in the future.' Another student highlighted the importance of prior experience, stating, 'If you have no experience with the templates, this would certainly not work. In my case, I could guide ChatGPT, which is why it worked well.' Furthermore, students' reflections underscored the role of AI as a helpful tool but not a substitute for domain knowledge. One student observed, 'It worked very well for me, but you must 'spoon-feed' everything. As in most cases, ChatGPT is more helpful as inspiration and less for spitting out ready-made solutions.' These insights suggest that while AI tools like ChatGPT can assist in requirement writing, they are most effective when combined with domain expertise and clear guidance.
Frequency Distribution of Student Prompts The questionnaire responses from 15 out of 18 students provided valuable insights into their experiences with and perspectives on using AI in their academic life (see table 3). Most students (14 out of 15) had prior experience with AI in their free time. The responses varied regarding the likelihood of having used an AI for writing requirements without being asked. Some students considered it unlikely (6 out of 15), while others were neutral (5 out of 15) or leaned towards likely (3 out of 15). A single student expressed a very high likelihood that he would have used the AI to solve the assignment even if it was not explicitly asked for. Overall, the results collectively reflect students' diverse experiences, perceptions, and intentions regarding the integration of AI into their requirements writing process. They also serve as evidence that AI plays a significant role in their lives, both personally and academically. This underscores the significance of providing guidance for the use of AI, making students reflect on its application, rather than adopting a banning approach.
The synthesis of findings from this experience report provides valuable insights into integrating AI tools in Requirements Engineering education, particularly within higher education settings.
Students (%)
Students' interactions with AI tools showcased diverse approaches, from utilizing AI as a brainstorming partner to grappling with challenges in refining AI-generated requirements. The importance of prior familiarity with requirement templates and the need for clear guidance emerged as pivotal factors influencing students' critical evaluation of AI-supported requirements generation. Notably, through the requested reflection, the students showcased a heightened cognitive level within Bloom's taxonomy [4], demonstrating that they can not only apply writing requirements but also analyze and evaluate the AI's generated requirements. This suggests an enhanced learning outcome. Thus, it was not solely the usage of AI that led to a higher learning outcome but rather the inclusion of a reflection component within the assignment. Consequently, students emphasized AI's role as a valuable supplementary tool rather than a complete replacement of their skills. This synthesis underscores the nuanced nature of AI's involvement in Requirements Engineering education, emphasizing its optimal performance when complemented by human expertise and clear direction.
While this study acknowledges its limitations, including a relatively small sample size and the absence of a control group, it underscores the necessity for future research efforts. In the upcoming semester (Spring 2024), the assignment will be conducted with two different classes within the bachelor module of Information Science, following the same approach described in this experience report. Information Science students, who possess not only prior programming experience and but also information retrieval knowledge, might adopt different prompting approaches compared to the first observed student group. The emphasis of student reflections should also focus on the way of prompting, recognizing its impact on the outcomes. This endeavor seeks to refine AI integration strategies further and deepen our understanding of its implications in education.