For the last few years, we have been living, working, teaching, and learning in a very dynamic reality. This situation affected and continues to influence all areas of our lives. One of them is the teaching of computer science at the university. Most of us had to do all the accompanying activities online. This fact caused several problems such as: the need to keep students engaged in the learning process; the continuing trend of finding the intersection between theory and practice (in one environment where new technologies are constantly emerging and trends in preferred programming languages are changing); increase students' motivation to be innovative in the projects they develop. This paper proposes a methodology that the authors use to solve these challenges during their teaching in various software disciplines (compulsory or elective) for IT specialties: informatics, software engineering, software technology and design, and business information technologies. It is based on the application of a combination of knowledge and practices set by the industry de-facto standards such as Capability Maturity Model Integration (CМMI), CMMI with Scrum, Test Maturity Model integration (TMMi), CERT Resilience Management Model (CERT-RMM) in combination with DevOps. An experimental mapping to the competences (knowledge and skills) of the European e-Competence Framework is also presented to ensure the coverage of essential competences expected by the most popular IT jobs profiles. The authors describe the achieved results of the applied methodology, as well as their analysis and ideas for its development and improvement in the future.
We live in a highly dynamic reality that requires flexibility in all aspects of our lives. This fact is exceptionally reasonable for people working and studying in Information Technology (IT) field. These circumstances require the education and development of particular ICT professional competences based on the newest European standard e-CF (European e-Competence Framework) [1] that IT professionals must-have, even during their training. In recent years, we have dedicated our eforts in this direction: to discover a practical multidisciplinary methodology for that purpose. In our previous work [2], we have described the results and conclusions from conducting the core course, “Software Quality Assurance (Q.A.)” by introducing software quality maturity models in software engineering education and small organizations. We concluded that the main benefits of the course consist of: (1) Introduction to industrial de-facto standard models like CMMI, CMMI with Scrum, and TMMI; and (2) Applying these models to small business organizations with the survey showing how one company department started with two workers four years ago and increased in size to 15 developers presently. Based on that, we demonstrated how graduate students could use this course as a reference framework to find an intersection point between theory and practice in diferent projects. The second direction was to apply the acquired knowledge and skills to further expand this methodology and use it multidisciplinary in various IT higher education programs or even in associated IT business training. The current paper presents some early and promising results of such a generalization.
Our representative methodology includes the following key sections [3]: 1. Type of research: its purpose is to be applied in courses in IT education and related to software businesses; 2. Data collection process: a survey of students from diferent computer science specialties and small business organizations; 3. Data analysis processes: based on the e-CF framework with applying the corresponding industry standards [4], [5], [6], [7], [8], [15], [16]; 4. Resources, materials, and tools [11]¸ [12], [13], [14], [17]; 5. The rationale behind the research: is explained in the Introduction section.
We have based our approach on the European e-CF framework [1], as it provides a widespread and cross-cutting tool to support the evaluation and analysis of the competency needs of trainee programmers, their career paths, and their progress. It contains a four-dimensional approach. The first dimension comprises competence areas (Dimension 1) and a set of competences (Dimension 2) instead of job profiles. Dimension 3 provides us with level assignments appropriate to the corresponding competence, and how many levels assigned to each competence are defined on the nature and type of related activities. Dimension 4 provides sample specifications of knowledge and skills. Another advantage of referring to the e-CF is that the knowledge and skills are well aligned with the organizational processes. This approach also helps define policies related to e-Skills develop ment in education and the workplace and ensures further multidisciplinary.
The e-CF framework’s context development allows us to easily use diferent general and ICT specific process models like CMMI, CMMI with Scrum, TMMi, and CERT-RMM, which we have already used in the software engineering program. The logic and structure of processes used are aligned with the software and IT development lifecycle, namely the five main competence areas: Plan, Build, Run, Enable and Manage. We use these industry standards because improvement eforts require a model of how our organization works, which functions it needs, and how those functions interact. They give software engineering students and workers a better understanding of organizational elements and assist in discussions/meetings of what can and should be improved [4], [5].
We have adapted the presentation of the process-oriented models to our specific methodological needs with the idea of cross-referencing and demonstrating their complementarity. First, we chose the CMMI model (or its variation CMMI with Scrum) because of its well-structured nature and provided mechanism containing frameworks for the organization and use practices for industry maturity for IT and software companies. Referring to the public version 1.3 of CMMI for Development [4], we selected process areas and specific and generic practices with many practical examples and exercises. Referring to the stage representation of the CMMI model (ver 1.3), we have limited the scope to Maturity Levels 2 and 3. Maturity Level 1, also known as “performed”, we use as an excellent illustration of companies in a “survival” chaotic mode (which is frequently characterized as “poor processes – bad product”, referring to Deming TQM, for example). Naturally, we develop in detail all process areas related to project management, as grouped mainly in Maturity Level 2 (Managed, previously known as Repeatable). The essential six process areas are presented and then exercised by the group projects, namely: Requirements Management (REQM), Project Planning (PP), Process and Product Quality Assurance (PPQA), Project Monitoring and Control (PMC), Measurement and Analysis (MA), and Configuration Manage ment (CM). The Supplier Agreement Management (SAM) goals and practices are covered only in a nutshell and are further elaborated as “supply chain management” or external dependencies management (covered within CERT-RMM, but also considered as a possible separate new course). As frequently applied in practice, we also present REQM in combination with the Requirements Development (RD) process area from Maturity Level 3 (Defined). To bridge with the newer version of the CMMI model (2.0), we merge the Requirements development and management, emphasizing the complementarity of the two processes and their critical importance for the entire software development lifecycle. As a reference model and templates for software requirements, a quick overview of another standard ISO 25065:2019 is also given, with Common Industry Format (CIF) for a user requirement specification, the content elements, and the format for stating those requirements. The other two process areas from ML3 presented in detail are Verification (VER) and Validation (VAL), with extensive practices in peer-reviews (not only of the code but also the other working documents such as requirements, project plans, testing plans, etc.). The need for technical solutions and technology monitoring (also from ML3) to ensure innovations and higher competitiveness are usually topics developed by the students in their teamwork. From higher maturity levels 4 and 5 (Quantitatively Managed and Optimizing), we outline the goals and business benefits and discuss how more mature organi zations gain from intelligent investments in process improvement.
We included the TMMI model to cover the testing phase in our projects [7], [8]. TMMi Foundation developed it as an independent guide and reference framework to help us identify the current level of test process maturity and, as a result, prioritize, plan and make improvements. TMMi is positioned as complementary to the CMMI model and uses staged representation and the concept of maturity levels for processes. Its structure allows process improvement identification at each required level. TMMi provides five levels (similar to the CMMI) with 16 Process Areas in summary. Similar to CMMI structure, they all include difer ent Specific and Generic Goals, which have corresponding Specific and Generic Practices. The enlisted 843 Sub-Practices provide a repeatable, detailed view of a company’s test process maturity and a pathway for test process improvement. In our approach, we use only the areas which correspond to those chosen by us from CMMI.
With new cyber threats emerging constantly, now it is not a question of “if” but “when” an organization will be attacked or compromised [18]. While companies cannot anticipate or prevent every cyber-attack, we need to predict and respond to changes in one risk environment and be prepared to continue our services and operations to meet our business mission when disruption occurs. These circumstances determined our choice when we chose the third pilar reference model used in our methodology: CERT Resilience Management Model, CERT-RMM [16]. It focuses on enterprise risk and resilience and helps organizations plan, predict problems, and pivot to address issues, together with building the capability. Accomplishing a continuity of operations during a disruption requires a resilience approach to cybersecurity. It integrates and can give us a holistic way of managing security risks, business continuity, disaster recovery, or IT operations according to the business missions and strategies. We use CERT-RMM to present to students how they can handle and mitigate diefrent disruptions and manage risks to critical assets by op timizing protection and continuity strategies. The model contains 26 process areas, organized into four categories according to their context: Engineering, Operations, Enterprise Management, and Process Management.
Similar to the CMMI approach, we describe the purpose of all process areas shortly, the need for a holistic approach to cyber resilience, and the critical role of the software developers and engineers to ensure “security by design” and “resilience by default” principles. A selection of process areas is presented in detail, mainly focusing on the foundations of information security and cybersecurity, critical assets management and risk management, services and business continuity. The selection includes the management of four groups of assets – Knowledge and Information Management (KIM), Technology Management (TM), People Management (PM), and Environmental Control (EC). Students develop their risk assessment examples and also work on mitigation plans. As a practical exercise and benefit to the most-popular practice, mapping to the ISO 2700x information security standards is illustrated. It includes the extrapolation of the fundamental CIA triad principle (confidentiality-integrity-availability) to other than informa tion assets based on unified methodology from Asset Definition and Management (ADM) process area. The controls to manage incidents, threats and vulnerabilities are referred to the processes from the relevant CERT-RMM process areas: Vulnerability Analysis and Resolution (VAR), Incident Management and Control (IMC). As a basis of operations security and interoperability, Identity Management (ID) and Access Management (AM) are described with various examples of multi-factor authentication, identity theft and social engineering, and typical developers’ mistakes. The higher-level organizational commitment and the holistic approach to resilience are illustrated by the Enterprise Focus (EF) and Compliance (COMP) process areas. The most critical area for software engineers is Resilient Technical Solution Engineering (RTSE), combined with Resilience Requirements Development (RRD). Here we extensively focus on Secure Coding principles with examples on diferent technical platforms and languages. We have realized the sad fact that most of the technical programming courses do not cover the secure coding requirements, as well as common weaknesses and vulnerabilities left by the developers. That was an additional motivation to force the multidisciplinary approach and link those security requirements to the other courses in the program, as listed below (web design, UX/UI, etc.).
If the use of CMMI and TMMi helps us show students how to develop projects that meet the quality new quality ISO / IEC 25000 standards, then based on CERT-RMM we also include ISO 2700x family of standards, which helps companies and organizations to keep information and other critical assets secure. Respectively in our approach, we teach the students to develop secure, safe, and standardized applications by adding and respectively testing the security-by-design requirements.
Another point of intersection between the models described so far is that all three ofer us a solution of “what” to do, not “how” to do it. They give us valu able practice and flexibility in choosing the tools we use in diferent courses for diferent specialties. In addition, it allows the realization of the multidisciplinary of our methodology.
DevOps presents a set of practices, tools, and a cultural philosophy that automates and integrates the processes between software development and teams [19]. It emphasizes team growth, communication and collaboration between groups, and technology automation. We decided to include these principles in our approach because they help diefrent stakeholders like developers who wrote code and the operations who deployed and supported the software to work together. DevOps provides a mechanism to show our “how” while executing a project. Finally, the essential aspects of the DevSecOps and their use for continuously delivering “secure” features to the software applications are also explained and demonstrated.
By introducing our students to these three de-facto industry standards (CMMI), TMMi, and CERT-RMM, and extending their knowledge with DevOps/DevSecOps philosophy, we bring an “industrial” atmosphere to diferent software engineering university programs.
The second part of our methodology includes exercising students through cooperative real team projects using the appropriate tools, according to the project and IT course topic [14], [17].
Our leading roles in all courses are to be mentors who define the function of aligning diferent project profiles with the respective process areas’ activities and responsibilities. All team members often have more than one position (needed for smaller teams) and act appropriately. That necessitated additional methods for their establishment as the RACI (Responsible, Accountable, Consulted, and Informed) chart matrix [20], which helps us, in addition, to develop the ICT professional competences [15] systematically.
If we have to summarize to ensure multidisciplinary in diferent university software programs of our methodology, we may formalize it as follows: • Introduce students to proven industry quality maturity models, such as CMMI, CMMI with Scrum, TMMi, corresponding to ISO/IEC 25000 standards, and CERT-RMM to fullfil the secure and resilience aspect of ISO 2700x family. This process provides all denfiitions and vocabulary according to pro cesses and organization level, institutionalization, goals and practices, software testing and quality software standards, a generic approach and practices, and management. We turn students’ focus on process improvement as critical for the software products and project quality, independent from the scope. • We overview selected process areas of the models related to the software project management and further demonstrate the logic and coherency of project phases with the respective process areas and specific practices. • We demonstrate diferent kinds of real projects with all aspects of the software lifecycle from low- and high-fidelity wireframing and prototyping and formalizing software requirements to the software testing stage. • We show the cross-point between theory and practice, like applying and exercising the models in an actual collaborative environment by assigning students the development of similar projects in teams consisting of 2-5 people, according to the specialty and the course we teach. Students can distribute between themselves, perform diferent project roles, and take on cor responding responsibilities (role-playing) [21], like keeping and preparing documentation compliant with the approved standards we mentioned earlier in this paper. We provide students with all the necessary templates, tools, and documentation to perform their tasks. • We encourage teamwork, creativity, and role-playing by involving students in studio projects that focus on real contemporary business problems. We give them the freedom to decide what will be the project’s scope, foster self-organization, leadership skills, and team management, but assist them all the time, as needed. That way of work provokes and cultivates the sense of a startup inside students. • We provide the opportunity for the teams to present their projects and thus allow the accumulation of experience for the colleagues of the whole IT course. • We introduce students to many modern tools and techniques to achieve their tasks. • We motivate students by allowing them to decide on the awards and provide additional “bonus” scores and unique certificates.
In conclusion, this methodology ofers a well-described step-by-step ap proach that helps students from diferent specialties, on the one hand, to get ac quainted with the described models and, on the other hand, to emphasize the practical side of their education. In this way, they become better professionals and develop specific competences, regardless of the field in which they work.
We tested our multidisciplinary methodology for the first year, which extends the described one [3] by observing the students from diferent computer science specialties (bachelor program) and courses in the Faculty of Mathematics and Informatics from the University of Plovdiv “Paisii Hilendarski”, listed below: 1. Informatics 2. Software engineering 3. Business Information Technologies 4. Software Technology and design 5. Mixed (in elective discipline)
The next section of this article is devoted to the results of the pilot implementation of this approach and our study.
We tested our methodology via a survey between five diferent specialties in ifve courses. We apply our approach generally by presenting the same models, but using various tools, defined from the software area and projects’ topics. We describe in Table 1 the outcome of our survey for each specialty.
For seven years of delivery, our bachelor course, “Software Quality Assurance” [10], outlined and demonstrated to more than 700 students from Informatics how process improvement is an essential factor in the quality of software programs. This year aimed to study the CMMI, and TMMi model, add CERTRMM for cybersecurity and resilience aspects and help us build quality software products and successful business. All introduced models complete each other and are used to develop competences, referred to in the e-CF framework. We describe the main processes in the lifecycle of a typical software project and cultivate software development skills by exercising theoretical knowledge through real projects (developed by students, separated into teams). This year, we increased the number of groups and ready projects with students: from 11 to 14 (with typical 3–5 people per team). Compared with the last year, it was not a peak, but a better result.
In the second discipline, “Cyber Security & Business Resilience” [11], we emphasize to CERT-RMM model to face the deficit in information and business security and risk management. It naturally maps the cybersecurity and resilience aspects to software projects and systems requirements (Development and Management) and Resilience Technical Solutions Engineering. We add to the typical “requirements” (customer and product requirements) also the “security and resilience by design” principles and secure coding, which most of the stakeholders usually underestimate, especially customers in the typical product requirements. The process-oriented description, goals, and practices made it easier for the student to understand these new areas. This year we gave students more practical work based on projects, which until now are 23 (with two-three students per team).
Teams
12 (2–5 average students per team)
23 (2 students per team)
36 (2 students per team)
49 (2 students per team)
28 (2 students per team)
For the Business Information Technologies specialty, in which students study “Web design” [10], we pay more attention to the following process areas: Requirements Management, Requirement Development, Project Planning, Process and Product Quality Assurance (process areas from CMMI), testing tools, and from CERT-RMM we shortly described Resilient Technical Solution Engineering, Incident Management, and Control, Knowledge and Information Management. This vast knowledge allows students to create better web design projects not only from a visual design perspective. We motivated them more than ever this year and received 36 projects in summary (including website specification and coded visual design).
The “Creation and processing of vector images” course [10] is a part of the Software Technology and Design specialty. Similar to the previous discipline, “Web design”, its topics are faced more in the development of visual design projects. Here we introduce mainly project areas, corresponding to extracting project specifications and project management. The response was positive, and we re ceived more than 40 projects in the current academic year. The conclusion is that when we have quality processes, precise project requirements, and well-defined roles, students are more motivated and eager to participate in teams.
The last course in our observation is “Build web interfaces with Bootstrap 5.0 by applying main UX/UI principles and design systems” [13]. Its nature, as an elective discipline, defines the participants, and they are students of all mentioned specialties until now. The projects here cover all processes in one web project: UX research and supporting documentation, through UI and design system building, to the developing an existing code with a Bootstrap 5.0 front-end framework. These features led to the need for a brief presentation of the main process areas of all three models. This additional knowledge helped students to do their projects, which were 28.
This paper introduced our author’s multidisciplinary approach to using
industry standards in the IT higher education programs, which help students acquire
e-CF competences. We described our diferent courses of study at the Faculty
of Mathematics at the University of Plovdiv, where we test this methodology:
“Software Quality Assurance”, “Cyber Security & Business Resilience”, “Web
design”, “Creation and processing of vector images” and “Build web interfaces
with Bootstrap 5.0 by applying main UX/UI principles and design systems”. We
demonstrate how the software quality maturity models like CMMI, TMMi, and
CERT-RMM adapt to software engineering education and how they can be
transferred and applied by students in small business projects and organizations,
especially working within the IT industry. In all presented software engineering
specialties, we have described how such industry standards and corresponding tools
help develop graduates’ skills and competences and how they could use them as
a cross-point between theory and practice in their current and future professional
work projects.
4. References
e-CF (European e-Competence Framework) version 3.0, adopted in 2016
as European standard and published by CEN as the European Norm EN
16234-1, 2021, URL:
https://itprofessionalism.org/about-it-professionalism/competences/the-e-competence-fr
URL: https://docs.microsoft.com/en-us/azure/devops/boards/work-items/
guidance/cmmi/guidance-background-to-cmmi?view=