=Paper=
{{Paper
|id=Vol-3672/EuT-paper3
|storemode=property
|title=Advancing RE Education: Towards a Mapping Scheme for Benchmarking Students' Specification Skills
|pdfUrl=https://ceur-ws.org/Vol-3672/EuT-paper3.pdf
|volume=Vol-3672
|authors=Anthea Moravánszky,Samuel A. Fricker
|dblpUrl=https://dblp.org/rec/conf/refsq/MoravanszkyF24
}}
==Advancing RE Education: Towards a Mapping Scheme for Benchmarking Students' Specification Skills==
https://ceur-ws.org/Vol-3672/EuT-paper3.pdf
Advancing RE Education: Towards a Mapping Scheme
for Benchmarking Students’ Specification Skills
Anthea Moravánszky1 and Samuel A. Fricker, 2
1
University of Szeged, Doctoral School of Education, Hungary & University of Applied Sciences of the Grisons,
Pulvermühlestrasse 57, 7000 Chur, Switzerland
2
University of Applied Sciences and Arts Northwestern Switzerland, Bahnhofstrasse 5, 5210 Windisch, Switzerland
Abstract
In Requirements Engineering (RE) Education, aligning students' requirements engineering skills with
industry demands is one of the primary aims for educators. RE curricula vary between higher
education institutes, but how they differ is difficult to describe. This paper proposes a novel mapping
scheme to determine students' specification skills and, consequently, their industry readiness. This
paper describes and motivates the scheme based on established bodies of knowledge in requirements
engineering, syllabi for education and certification, and requirements specification standards. A
preliminary validation of the scheme is provided by applying it to results produced by students of
two RE courses taught at two universities of applied sciences, with the respective lecturer reviewing
the accuracy and precision of its application. This paper describes the scheme and its development,
outlines the preliminary validation process, and discusses its potential use for mapping RE education
practices.
Keywords
Requirements Specification, Specification Skills, Mapping Scheme1
1. Introduction
The significance of preparing students for industry becomes evident in the requirement
specification skills discourse [1]. Given the diverse expectations on RE skills across various
industries and ever-new challenges addressed by changing technologies for designing solutions,
educators must continuously update their RE curricula [2]. Benchmarking is a strategic
approach to describing and comparing such practices and discussing the evolution of these
practices. A scheme for benchmarking is an instrument for structured and repeatable
documentation.
In RE education, there is a scarcity of schemes for benchmarking. There are syllabi, such as
[13], that could be used for such benchmarking. While the intent of teaching such practices is
easy to determine in course descriptions, it is unclear how evidence about the students’ learned
ability to apply these practices would be documented so that these abilities can be identified
In: D. Mendez, A. Moreira, J. Horkoff, T. Weyer, M. Daneva, M. Unterkalmsteiner, S. Bühne, J. Hehn, B.
Penzenstadler, N. Condori-Fernández, O. Dieste, R. Guizzardi, K. M. Habibullah, A. Perini, A. Susi, S. Abualhaija, C.
Arora, D. Dell’Anna, A. Ferrari, S. Ghanavati, F. Dalpiaz, J. Steghöfer, A. Rachmann, J. Gulden, A. Müller, M. Beck, D.
Birkmeier, A. Herrmann, P. Mennig, K. Schneider. Joint Proceedings of REFSQ-2024 Workshops, Doctoral
Symposium, Posters & Tools Track, and Education and Training Track. Co-located with REFSQ 2024. Winterthur,
Switzerland, April 8, 2024
anthea.moravanszky@fhgr.ch (A. Moravánszky), samuel.fricker@fhnw.ch (S. A. Fricker)
Copyright © 2024 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
and compared between education providers. Today, there is only a limited understanding of the
RE practices being learned, and discussions between education providers regarding best
teaching practices and their improvement hardly occur.
This paper aims to narrow this gap by proposing an evidence-based scheme for mapping
students' demonstrated specification skills. While specification skills are only a subset of the
requirement engineering practices being taught, they can be evaluated based on concrete results
that are created by the students.
2. Background
In software engineering, requirements specification documents are created to document the
results of RE activities. These documents contain software requirements and supplementary
details such as a vision, solution concept, reference documents, context, acceptance criteria, and
glossaries [3, 15]. Several works have addressed requirements specification and benchmarking
specification skills in the educational context. Fricker et al. [4] surveyed industry projects,
focusing on specification techniques. Their findings revealed diversity in the techniques and
methods employed, with natural language emerging as the predominant notation, often
complemented by integrating UML diagrams. Virodula and Fortino [1] assessed students'
requirement specification skills based on an Industry Body of Knowledge within a STEM
graduate program, aligning the curriculum with the International Institute of Business Analysts
(IIBA) standards. While successfully assessing theoretical knowledge, the study did not evaluate
students' practical application of requirements engineering (RE) specification skills. Anil and
Moiz [5] introduced a comprehensive rubric for evaluating students' software requirements
specifications. While providing a foundational tool for educators to assess Software
Requirements Specifications (SRS), this rubric may require substantial modifications for
universal application across different universities, diverse SRS templates, or a combination of
single specification artefacts. The Requirements Engineering Ontology (REO) by Saito, Iimura,
and Aoyama [6] maps requirements engineering techniques based on three RE Bodies of
Knowledge (SWEBOK, BABOK, and REBoK). However, it lacks sufficient depth in outlining
recommended specification techniques for benchmarking students' specification skills.
The primary goal of this work is to promote comparability between higher education
institutions. We do so here by assessing the specification skills of RE students. To achieve this,
we propose a mapping scheme for application across higher education RE courses. The scheme
focuses on mapping specification skills of students because it showcases their proficiency in
generating distinct requirements-related artefacts and applying techniques suitable for various
aspects of requirement specifications. We believe these specifications demonstrate that students
possess a robust understanding of RE, comprehend the interconnections between different RE
artefacts, and are capable of performing specific specification techniques, rendering them
'industry-ready.'
3. Methodological Approach
Addressing this gap involves a comprehensive examination encompassing Bodies of
Knowledge, model curricula, syllabi, and standards for identifying relevant RE specification
practices and requirements specification document templates. Our methodological approach is
based on the systematic mapping process outlined by Petersen et al. [7].
We selected our knowledge sources based on REE literature recommendations [8]. We
included the Software Engineering Body of Knowledge SWEBOK [9] (version 3.0, 2014, since
SWEBOK 4.0 is still under review) and the guide to the Business Analysis Body of Knowledge
(BABOK) [10]. Furthermore, we also included the RE Body of Knowledge (REBoK) [11].
However, as the REBoK itself is only available in Japanese, we utilised a descriptive presentation
of the REBoK authored in English instead. We also included standards such as IEEE 29148:2018
[12], which replaced earlier specifications. Additionally, the International Requirements
Engineering Board (IREB) syllabus [13] is a valuable resource, particularly the syllabus of the
CPRE FL certification. To address agile specification types, we incorporate Requirements
Artifacts of the Rational Unified Process [14], enhancing our mapping to encompass a
comprehensive range of RE specification practices. The screening process began with keywords
acquired from requirement types and specification techniques and structures in the
'specification' section of Fricker et al.'s [4] work on 'common requirements engineering
practices. From there, we mapped all our findings in the scheme and enhanced it with additional
findings from the screened sources. In each of the mentioned works, the respective chapters
were first read by one author, and then the entire work was subsequently searched again for
keywords by the same author. A second author reviewed the results and conducted random
checks via keyword searching.
4. Mapping Scheme for RE Specification Practices
The foundation of our mapping scheme was derived from Fricker et al.'s specification section
[4], which included "Requirement Types" and "Notations" as subparts. While we retained the
subsection on requirement types, our mapping results led us to separate notations to distinguish
specification techniques. Additionally, we introduced a new subsection, 'Requirements-related
Artefacts and Document Structures,' to provide a more nuanced and detailed section. The
decision not to include Storage was based on the understanding that it does not lend itself to
direct mapping from a student's specification.
Fig. 1 shows the developed mapping scheme. It is designed to comprehensively address
various aspects of Requirements Engineering (RE) specifications, offering versatility in its
applications. A key feature is its ability to capture specification variations, encompassing not
only the count of specific elements but also their percentage representation to illustrate
different class sizes of students.
The intended use of our mapping scheme is versatile and can serve multiple purposes: 1) For
Characterization of a Single Specification: Educators can employ the scheme to
comprehensively characterise and understand the nuances of individual students' specification
artifacts or entire specification documents. 2) For Characterization of Demonstrated Learning
Outcomes: The mapping scheme can be applied to assess and articulate the demonstrated
learning outcomes of a course in RE. 3) For Comparison Between Courses or Institutes:
Educators should be able to compare the demonstrated learning outcomes between different
courses or institutes of higher education, providing valuable insights for curriculum
development and improvement.
Figure 1: Basic mapping scheme
5. Early Validation with two Universities of Applied Sciences
Our mapping scheme underwent validation in two RE modules at different universities of
applied sciences offering bachelor's degree courses. Course A focuses on fundamental RE
practices such as requirements elicitation, analysis, validation, and management in software
development projects. It covers terminology, vision development, business analysis, prototype
workshops, and reviews of requirements specifications. Students explore requirements
specification languages and templates like Shall-, User Story-, and Use Case templates. Learning
objectives include understanding RE concepts, creating visions and requirements for software
systems, and quantifying quality requirements. The course emphasises developing release plans
within staged and agile software development lifecycles and includes group projects for
practical application. Course B is aligned with the syllabus of IREB CPRE FL and should prepare
students to pass the certification. Agile methods are excluded as they are part of a subsequent
course.
Evaluating students' assignments reveals differences and commonalities. Most students use
Use Case Diagrams and Specifications, with familiarity with activity and class diagrams.
Functional and quality requirements are present and expressed through various methods. While
elements like vision and goals are recognised, other document structures, except for the product
backlog, were not recognised clearly.
Lecturer A agreed with the mapping scheme's assessment, recognising its reflection of
taught requirements specification techniques and student choices. However, Lecturer A
suggested enhancing the scheme's sensitivity to detect more techniques in student documents
by establishing more explicit links between scheme categories and providing precise detection
guidelines for stakeholder descriptions.
Regarding the assessment of Course B, Lecturer B noted that students were assigned three
specific tasks: 1.) writing functional and non-functional requirements; 2.) modeling a use case
diagram and writing a use case specification for one of the use cases of the diagram; 3.) modeling
an activity diagram and class diagram based on a description) instead of creating a coherent
requirement specification document, as for Course A. This resulted in a limited view and
detected coverage of specification skills and the demonstrated variety.
Figure 2: Summarized Results of the Mappings of Student Specification Assignments
6. Discussion
Our work builds upon established knowledge frameworks to develop a mapping scheme to
assess students' specification skills in REE. Leveraging these frameworks, we aimed to create a
universal tool for mapping individual specification artefacts, thereby enabling the
benchmarking of courses or institutes by comparing students' specification skills demonstrated
in course assignments. The practical application of our mapping scheme uncovered its
effectiveness in characterising specification documents. However, it also revealed limitations,
particularly in supporting additional artefacts like implemented functional prototypes.
Challenges persisted in identifying specific templates, especially phrase-based and document-
template-based approaches. The flexibility and adaptability of students' expression of
requirements were evident, but the nuanced task of differentiating between templates
necessitated further consideration. One potential threat to the validity of our evaluation was
the reliance on our works for testing the mapping scheme, introducing the risk of bias. To
mitigate this, further validations on a diverse set of works are imperative to ensure the
independence and generalizability of the mapping scheme.
Our mapping scheme primarily reflects students' performance based on given assignments,
potentially limiting its ability to entirely capture acquired specification skills from the course.
The nature of the task assignment constrained the scope of skills showcased by the students,
underscoring the importance of considering the broader curriculum context when interpreting
the results. By providing insights into the nuances of students' performance and assignment
tasks, the paper contributes not only to the benchmarking of RE education but also offers a
pathway for continuous improvement in curriculum development. It encourages educators to
consider the broader implications of the mapping scheme, fostering further exploration and
adaptation to meet the industry's evolving demands and the diverse landscape of Requirements
Engineering education.
References
[1] T. Virodula and A. Fortino, "Assessment of Systems Requirements Specification Skills
Based on an Industry Body of Knowledge," IEEE Integrated STEM Education Conference
(ISEC), Princeton, US, 2021.
[2] M. Daun, A. M. Grubb, V. Stenkova, and B. Tenbergen, "A systematic literature review of
requirements engineering education," Requirements Engineering, vol. 28.2 (2023) 145-175.
[3] C. Rupp, Requirements-Engineering und-Management: Das Handbuch für Anforderungen
in jeder Situation, Carl Hanser Verlag, 2020.
[4] S. A. Fricker, R. Grau, and A. Zwingli, "Requirements engineering: best practice,"
in Requirements Engineering for Digital Health, Springer, 2014.
[5] G. R. Anil and S. A. Moiz, "A holistic rubric for assessment of software requirements
specification," 5th National Conference on E-Learning & E-Learning Technologies
(ELELTECH), Hyderabad, India, 2017.
[6] S. Saito, Y. Iimura, and M. Aoyama, "REO: Requirements Engineering Ontology Spectrum
Analysis of Requirements Engineering Knowledge and Its Practical Application," IEEE 39th
Annual Computer Software and Applications Conference, Taichung, Taiwan, 2015.
[7] K. Petersen, S. Vakkalanka, and L. Kuzniarz, "Guidelines for conducting systematic
mapping studies in software engineering: An update," Information and Software
Technology, 64 (2015) 1-18.
[8] S. Ouhbi et al., "Requirements engineering education: a systematic mapping
study," Requirements Engineering, 20.2 (2015) 119-138.
[9] The Software Engineering Body of Knowledge, 2024-02-06, URL:
https://www.computer.org/education/bodies-of-knowledge/software-engine.
[10] IIBA BABOK, Leitfaden zur Business Analyse Version 3.0, Götz Schmidt, 2017
[11] M. Aoyama, "Requirements Engineering Based on REBoK (Requirements Engineering
Body Of Knowledge)," 2011, URL: http://www.nise.org/eng/REBoK/REBoK-APSEC2011-
Tutorial-2011-12-05.pdf.
[12] ISO/IEC/IEEE 29148:2018 Systems and software engineering. Life cycle processes.
Requirements Engineering, 2018, URL: https://ieeexplore.ieee.org/document/8559686.
[13] IREB CPRE FL Syllabus, 2021, URL: https://www.ireb.org/content/downloads/21-cpre-
advanced-level-re-agile-syllabus/syllabus_cpre_al_re%40agile_en_v2.0.pdf.
[14] P. Kruchten, The Rational Unified Process: An Introduction, Addison-Wesley Professional,
2004.
[15] H. Kittlaus, S. A. Fricker, Software Product Management, Springer, 2017.