We are currently living in an extraordinary situation. The circumstances, imposed by the ongoing pandemic situation, have led to a variety of unprecedented conditions with unpredictable outcomes. Most of us are working, teaching, and learning online, partially or completely. Therefore, we should be and we are more flexible than ever. What is the best strategy against the chaos, then? We figured out that in terms of software engineering education and small organizations we need at least two things. First, it is always important to search for the intersection between theory and practice, on a daily basis. Secondly, it is of utmost importance to have good described and defined inhouse work processes, no matter whether in consideration of education or of business companies. That is possible through a rigorous establishment of good practices and following the guidelines for models of software quality maturity. In this contribution, we will introduce some of the more prominent models. These are CMMI (Capability Maturity Model Integration) [1], CMMI with SCRUM [2], or TMMI (Test Maturity Model integration) [3]. Along with this, we will overview a roadmap to transferring and applying this knowledge in practice within the context of small business organizations.
The ongoing pandemic provides a plethora of challenges for planning, thus
imposing an ever growing need for flexibility and adaptability. The more prepared
you are in a given field, the easier the process of decision-making is against the
backdrop of unprecedented and constantly emerging diversity of challenges. Thus,
to be a good specialist means being able to find the intersection between theory and
practice especially if you are working in the field of Information Technologies (IT).
Therefore, it is crucial that university programs in software engineering and
development can face the challenge of achieving a balance between the theory related
to current and future industry demands, and the sufficient practices to develop the
required skills, knowledge, and competences. For several decades already,
processoriented models have been used to achieve software and IT services industry
maturity [
Understanding the software development lifecycle, its related processes, and
roles, specific aspects of teamwork, as well as the variety of organizational
models with their related advantages and implementation burdens, are those expected
competencies that differentiate developers and software engineers from what
students themselves address as “coders”. Those gaps and insufficiencies of the IT
university education content are frequently underlined by the major IT
stakeholders in Bulgaria and the region [
Fig. 1 shows the aggregated results from a survey with ICT/software
industry (8 countries in Eastern Europe until now) on skills and competences
expectations (bipolar scaling method, measuring negative to positive attitudes to a
statement, with values from 0 to 6, where 6 is “high” [
To address that gap and align Bulgarian educational practices against the
landscape of the global trends, the SEMP (“Software Engineering Management
Program”) nationwide project was launched in 2010. It is coordinated by the
European Software Institute – Center Eastern Europe (ESI CEE) and in
collaboration with the Carnegie Mellon University (Institute for Software Research and
Software Engineering Institute), involving 7 leading Bulgarian universities [
More recently, along with the CMMI models tailored for software
development, IT services, and acquisition, a new process-oriented model specialized
for software quality testing, the Test Maturity Model integration (TMMI), was
introduced by the TMMI Foundation [
Software and hardware industries are evolving faster than ever. Only by knowing and choosing the right models for project management, project planning, process optimizations and correct structuring, software quality, and software testing and prevention we can survive in the daily business. Therefore, it is very important how we can provide the right and latest information like teachers to our current and future students. Only in that way they can see and understand how to cross the theory and the practice and become are good professionals in the business companies after their graduation. 2
During the pilot phases of SEMP (the years 2010 – 2021), more than 22 academic courses were introduced or built upon already existing courses to align with the standards and methodologies obtained through the know-how transfer from the Carnegie Mellon University (Institute for Software Research – ISR, Software Engineering Institute – SEI, others). Seven courses were implemented during the first phase (2012-2013), while others were gradually implemented after 2014 (as elective or regular/core courses). Interuniversity and industry-recognized certificates from the SEMP program were issued to all students who completed the courses, supplementary to their scores and credits received from the university.
In total since the beginning of the project, more than 1500 students have completed SEMP courses. Several courses were also offered in an intensive (executive) format option and were attended by both students and industry professionals, thus giving additional benefit from mixed teaming.
The approach chosen by the Faculty of Mathematics and Informatics at the
University of Plovdiv (FMI-PU), also a partner in SEMP was one of the gradual
implementations. In addition to the continuous improvement of existing courses
to bridge the theory-practice gap and meet the new trends, like [
The course content was based on the materials and experience of similar
courses under the SEMP program, delivered by ESI CEE, or implemented at
the Carnegie Mellon University (CMU), Sofia University (FMI-SU), and other
SEMP partners [
Additionally, being an elective discipline, the course was unable to provide the targeted common understanding and unified project management “language” that links and relates to the other disciplines in the university program. Thus, the beneficial opportunity to unite practical tasks from other disciplines was not utilized at all and sometimes they were partly overlapping each other. They remain formal as “homework”, exercising some individual skills but yet too far from the real industrial projects. Nevertheless, the course was considered a significant asset for students when seeking employment in the IT sector (see feedback results in the next section, Fig. 3) or just using the acquired knowledge in their current job or projects.
To better align the course with the development strategy of the Faculty and to
address the above-mentioned shortcomings, a new “Software Quality Assurance”
course was introduced as a core course of the 4th year of study in the BSc program
of Informatics. Despite the usual association of “QA” with “testing”, the course
focuses on the quality of processes as a prerequisite for the quality of the product and
the roadmap from process improvement to excellence. This course is also based on
the CMMI for Development (CMMI-DEV) model, benefitting from its
completeness and recognition by the industry as a de-facto standard, but further adapting
the relatively “heavy” model to the knowledge and skills level of undergraduate
students. The model was applied as a generic sample model, to which other modern
methods and techniques such as Scrum, “Disciplined Agile” with CMMI, Six
Sigma, Kanban are consequently addressed in the course. The course also introduces
business aspects such as “cost of quality” along with the need to balance process
improvement and product quality with a brief introduction of Kaplan and Norton’s
Balanced Scorecards [
We also include in the current academic year (2020/2021) CMMI with Scrum and TMMI model, together with the software-testing tool to be aligned with the last development tendencies. The CMMI with Scrum just expands the CMMI model with the agile paradigm. We further included TMMI, because as it well complements the previous too. It can be used in the testing phase of the software. It contains the five maturity levels, known from CMMI (see Fig. 2). The initial phases of both models are the same – business processes are unpredictable, poorly controlled, and reactive. All companies start at level 1 of maturity. After integrating and applying the corresponding specific and general practices to each level, we go to the next upper one and levels cannot be skipped. Following the “generic” nature of the processes and practices in CMMI, in addition to Scrum and Agile, we illustrate to students how they map to another popular work and team organization, namely DevOps.
To face the deficit in “Information security” area training, especially in
software engineering profile, we have also introduced the engineering aspects
(process areas) from another capability and maturity model, developed by CERT at
Software Engineering Institute (Carnegie Mellon University) – CERT RMM
(Resilience Management Model) [
For six years of delivery, the core undergraduate course “Software Quality Assurance” outlined and demonstrated to more than 550 students how process improvement factors into the quality of software programs. The goal, however, is not to study the CMMI or TMMI only. They complete each other and they are used as a reference framework to describe the main processes along the lifecycle of one typical software project and cultivate software development discipline by exercising in life-related realistic projects. In addition, help us to build good software products and successful business.
Within the next sections, we will outline the main milestones of combining university education and small business companies. 3
With the global shift towards remote work and education, we well realize that through targeted efforts, we can work, study and teach online, not worse than in an offline setting. Zoom, Google Meet, Microsoft Teams, Skype, and Slack became a part of our daily routine and way of life. Software development technologies, frameworks, and platforms have rapidly evolved since the onset of the pandemic, revealing a need for drastic revisions in the academic content to remain “contemporary” and “outstanding”.
On the other hand, the logic behind most software projects’ lifecycle is
quite generic and the challenge is how to translate the abstract processes into
practical implementation rules in an academic environment, which can be
used in further real projects and small organizations. Undergraduate students
in informatics are typically interested in acquiring a wider range of practical
competences and obtaining good grades, so they can jump quickly into higher
positions in the industry [
After collecting and analyzing our experiences from the elective course
“Process Quality Management”, similar courses of the SEMP program, and
examples from the software industry (see Fig. 1), we have proposed for the
core course „Software Quality Assurance (Q.A.)” the following approach and
statements:
1. Introduce students to the more prominent models for software
quality maturity, such as CMMI, CMMI with Scrum, TMMI, corresponding
to ISO/IEC 25000 standards, to focus students on process improvement
as a key factor for software product quality. This process provides all
necessary definitions and vocabulary regarding processes and company
maturity, institutionalization, goals and practices, software testing and
quality software standards, as well as a generic approach and practices
for organizational process definition and management.
2. Overview selected process areas related to software project
management and further demonstrate the logic and interconnectivity of
project phases with the respective process areas and specific practices, with
good practical examples of their implementation, “what if not”
discussions, and appropriate tools.
3. Demonstrate real projects with all aspects of the software lifecycle
from low- and high-fidelity prototyping, extracting software
requirements to the black/white and grey software testing.
4. Exercise theory in a realistic collaborative setting, by assigning the
development of similar projects in teams, where students can assign
and perform different project roles, with their respective
responsibilities (role-playing), and think about developing innovative applications
based on well-prepared documentation, compliant with other
internationally approved standards, such as ISO/IEC 25000 [
deciding what to do and with which technologies, plans, and
deliverables. Provide mentorship (organizational and technical, involve
industry and professionals from other areas – from Q.A. to software and
design companies) [
An important factor that supported the successful incorporation of such
practical team projects was the approval of the course as a core course for BSc
students in Informatics. This allowed stable team composition and result-oriented
collective impetus (turned to be impossible with elective format). Such an
approach is widely used in master’s programs [
Thus, it was beneficial for students to introduce it earlier in the undergraduate program (see feedback in Fig. 3). Another motivation was that most of the students from the last year in the university already have working experience in IT companies and have observations on “bad practices” as well and learning online they were able to take part in all lectures and exercise we had. They are prepared to extend the understanding of quality assurance beyond quality code producing and testing, and the importance of the established processes, quality of requirements, planning, monitoring, and control for successful project delivery in time.
The main goal of this approach is via introducing models for software quality maturity in software engineering education to support students in finding this intersection between modern theory and practice and lead this valuable knowledge to the small organizations and IT industry in summary.
To achieve that goal, we focus on bringing the real in-company working atmosphere to the university classroom by applying modern teaching methods, training techniques, and style of organization to break the patterns of the ordinary teachexercise-grade education system. Through a proper role-playing and teamwork, we address the involvement of the different stakeholders – students, professors, future employers, government, end-users, IT companies, and clients working together based on the established “common language” – the language of quality.
Through the course of the development and implementation of these sets of courses, we concluded that to achieve a certain level of flexibility and success in the area of software engineering education and the small organizations we need at least two things. First, to find the intersection between theory and practice, and secondly, to have well-described and defined inhouse work processes, no matter whether in an educational or business setting. For both, we can rely on the already introduced CMMI (or in the specific agile situation CMMI with Scrum), TMMI models, and the famous quality software group of standard ISO/IEC 25000. The question is what exactly we can use from them in the education area and how we can map that in the companies. In other words, how this knowledge can be easily transferred from our students and applied to small businesses. We will describe in detail and we will approach an example with a real company, in which one of the authors of the current contribution has been building a team for the last four years – with an initial number of teammembers of 2 to currently 15 people.
We started with CMMI, which provides a variety of well-structured and complete reference frameworks for the organization and pathway to industry maturity for IT/software companies. Of course, this process requires careful selection and gradual introduction, with sufficient practical examples and exercises. The maturity of the processes as a differentiator for mature companies brings a natural basis for a deeper introduction to CMMI.
For the sake of better clarity, we use staged representation with Maturity Levels. We are limiting the scope to Levels 2 and 3. Maturity Level 1, also named “Performed”, is a good illustration of companies in a “survival” mode. Here we make a discussion session with students that work on their observations over such a typical picture – processes and deliverables are unpredictable, we have no control, and the mode is reactive, most of the people do not know what and when or even how to do, so project delays and unsatisfied customers are common.
For education purposes, an essential part of the material is dedicated to Maturity Level 2 Managed (also referred to as a Repeatable). All process areas are reviewed in detail as this maturity level is dedicated to successful project management. Goals and practices are to implement managed, institutionalized processes and with continuous improvement. We have six important process areas from Maturity Level 2: Requirements Management (REQM), Project Planning (PP), Project Monitoring and Control (PMC), Process and Product Quality Assurance (PPQA), Configuration Management (CM), and Measurement and Analysis (MA).
A prerequisite for the start of real projects and group work is the knowledge of two process areas, namely Requirements Management (REQM) and Project Planning (PP). This is the necessary minimum to start transforming the business ideas into project specifications, work breakdown structure with respective estimates, project plans (with Gantt charts). We provide templates for all documents, which result from these areas, and demonstrate the use of appropriate tools. We further include an introduction to the ISO/IEC 25000 standards at this stage as well.
Consequently, with the progress of work, the topic of what progress monitoring methods could be applied and why corrective measures are needed is discussed, thus introducing naturally the Project Monitoring and Control (PMC) process area with its respective specific practices. Afterward, students understand through experience what a “significant deviation” from the project plan means, how to follow corrective measures, and why Configuration Management (CM) is more than source control. Particular attention is paid to the specific goals and practices (that define the scope of the process areas), the generic goal, and the ten generic practices are repeatedly illustrated, finally giving the logical justification and link specific practices to be implemented for managing the quality of processes under Process and Product Quality Assurance (PPQA) process area. Some basic indicators needed for process improvement are also underlined by the mentors, so students can recognize the value of the practices under the Measurement and Analysis (MA) process area. The seventh process area from Maturity Level 2, Supplier Agreement Management (SAM) is only briefly introduced, as in several cases; a need to delegate or “outsource” some studio project tasks to other teams is identified.
From Maturity Level 3 (Defined) we cover only a selected subset (mainly around the Requirements development and validation, peer reviews under Verification) as the students need them for their team projects. Since the very first initiation stage of discussing the project ideas, the teams already need some guidance on formalizing “customer” requirements. Some hints come logically with specific practices from Requirements Development (RD) process area.
The Validation (VAL) process area becomes critical in few cases of real industry or business-related project ideas. Some of them are about “patching” or improving real systems in use (such as ERP software, for example) – a challenging task even for mature companies. From the higher Maturity Levels 4 and 5 (Quantitatively Managed and Optimizing), only the goals and business benefits are discussed (again referring to the Cost of Quality structure).
After having introduced the basics of CMMI and the ISO ISO/IEC 25000
standards, we further include TMMI [
The overview of existing international software development and IT services standards and models, their applicability in different industries and for different purposes, as well as cross-mappings, is discussed. A good example that supports our approach is one of the latest feedbacks received from a small software outsourcing company in which the second author is working in addition. After passing the „Software Quality Assurance” course for the first time a couple of years ago, she received an opportunity to build a new team consists of 2 people in the beginning. In the first year, the company was at Level 1 and we had no structured workflow or defined processes. The author was starting methodically to use REQM, PP, PMC, PPQA from Level 2, combined with the RD process area from Level 3. With the consistent implementation of these processes, as well as with the help of the general and specific practices, at the end of the first year, the team grew to 6 persons. In the next 3 years, they have increased respectively to 11 and then 13 until now when they have reached 15 (web designers, 3D designers, and front-end developers. Most of the team members are students from FMIPU and have passed that “Software Quality Assurance” course, which greatly supports the processes’ institutionalization as the teammates are already familiar with the model and its practices.
Currently, the business workflow is well defined and everybody knows what, when, and how to do it. That leads to successful results, respected deadlines, despite the remote work setting for more than a year, and the constantly changing requirements. All achieved through well-defined processes and practices in different process areas and consistency.
One of the other company departments (there are five in summary) is starting to use CMMI repeatable maturity model, after being introduced with the good experience of the described team. 5
Familiarizing with the de-facto industry standards (CMMI) and TMMI,
exercising them through collaborative realistic projects with the use of the right
tools, manages to convey an “industrial” atmosphere in the university classroom.
Each of the lectures challenges students with practical industrial challenges
through exercises structured around the development of a real project. A list of
contemporary tools (most of them fully open-source) and techniques with proper
samples are being used and introduced not only for their university projects but
also in real work business environments [
Our main function as mentors is to align those profiles with the respective
process areas’ activities and responsibilities and to make sure those members with
more than one “hat” (needed for smaller teams) act appropriately. Another
engagement is to make sure that we systematically develop the ICT professional
competences based on the newest European standard e-CF (European e-Competence
Framework) [
For the six academic years of running the course “Quality Code Assurance” as described, with 36 student projects completed. Although the circumstances, imposed by the Coronavirus pandemic, required entirely remote collaboration, we had more teams and successful projects than ever.
Furthermore, a noticeable evolution in the project ideas can be observed as well. In the beginning, projects were more trivial, concerning online shopping services or e-commerce systems, however, since the latest course, we have had business-to-business applications, game portals, mobile applications, ticketing systems, teachers’ diary, and a Python-based AI project. 6
Due to the pandemic allowing remote participation in the classroom activities, a much bigger interest, presence, and motivation of students to engage proactively in the different software projects within the course “Software Quality Assurance” was observed, further showing the relevance of our conclusions and approach.
Our students succeeded to put theoretical knowledge on process quality improvement into practice and develop valuable skills: 1) creating, developing, and managing software project specifications and plans, through different industrial de-facto standards like CMMI-DEV, TMMI, and CMMI with Scrum, 2) understanding and applying mature software development processes, analyze them for improvements, focus on defects prevention, work in teams (even manage them online) with roles and responsibilities, and 3) immerse in a real industrial atmosphere and style of work (including a “bonus” system for performance and creativity). Fig. 3. Final results from software quality management courses at FMI, Plovdiv University (core course with project teamwork, online for the last “pandemic” year) and FMI, Sofia University (elective, with group study and presentations as teamwork, again online this academic year).
We have done two different types of summary surveys. First, was made from FMI-PU students, participating in the last 6 years of the course delivery as a core course with studio projects is shown in Fig. 1, in comparison to the aggregated feedback from a similar CMMI-based course at FMI-SU. The essential difference between the two courses’ formats is that the latter is still elective (also for BSc, but mixed diverse years of study and specialties), and the practical teamwork is limited to group study and presentations (e.g., on process area or implementation mixed techniques with Agile, DevOps, Kanban).
In both cases, the feedback on the course benefits and value for real work and employment are comparable (high). However, the significant difference in students’ opinion on teamwork format clearly shows the advantage of the studio project.
We can also add from this group of students some open comments related to the benefits of the course, such as: • “The teamwork and the practical knowledge gained on different processoriented models”, • “The knowledge I received for my future projects”, • “The knowledge I gained, which I can apply in my practice”, • “The teamwork and the simulation of an actual work environment”, • “The very motivating teachers who want to teach us in a way we can easily understand the material, not only take the exam after the course”.
The second type of study (shown in Fig. 4) was made based on how many teams were involved in projects this year (made only for FMI-PU), comparing the last 5 years, when we studied in the auditoriums, not online.
It is easily observable that in the current academic year we have our peak in the number of projects – 11 teams with students. That is double the year before. We may conclude, supported by the student’s feedback that this higher interest is mainly due to the forced by the pandemic online teaching/studying process. Through the remote setting of education, our working colleges can also participate in lectures and exercises, even from their offices or at home, without the traveling downtime, or being absent from their job.
Moreover, the use of different social platforms for communication and video conferencing provided both teachers and students, with the ability to keep in touch constantly and to be much more flexible through various collaboration platforms. In addition, this is additional experience for their future teamwork, especially within project management tools and environments, which was previously limited to the in-class exercises. 7
This paper introduced authors approach to the models for software quality
maturity in software engineering education. It represented how they can be
transferred and applied from students in small business organizations, especially
functioning within the IT industry. We described our core course „Software Quality
Assurance (Q.A.)” for the last (4th) year of study at the Faculty of Mathematics
at the University of Plovdiv [
We have demonstrated how such a course could be used by graduated
students as a framework to find an intersection between theory and practice in their
current and future projects. Trough the studio-projects we have encouraged the
teamwork and practical exercising of roles and functions in software
development teams, thus cultivating at university the so much desired by the industry
behavioral competencies, such as leadership, conflict management,
communications, and teaming [