<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>ACM, New York, NY, USA, Article</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Software Engineering Expectations</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>Jussi Kasurinen LUT University Lappeenranta</institution>
          ,
          <country country="FI">Finland</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2018</year>
      </pub-date>
      <volume>4</volume>
      <issue>6</issue>
      <abstract>
        <p>Software engineering as a discipline covers a large spectrum of concepts, from the very detailed technical aspects of system programming, to the high-end abstract concepts of the organizational software processes. Fundamentally, this also indicates that the software engineering researchers can focus their work on any of the several sub-disciplines, but also imposes an identity problem; how do the Master's and Bachelor's level students learn the fundamentals of software engineering, and how should they be taught these aspects? In this study, we observe three implementations of courses focusing on software engineering methods, and identify diferent approaches and recommendations on what works, and how software engineering courses should be constructed to promote student motivation and interest towards the discipline. Based on our observations, the fundamentals should focus more on the small-scale practical applications of the software engineering tools and practices, and promote real-life-applicability, since the students expect applicable experience, and software-industry relevance.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>CCS CONCEPTS</title>
      <p>• Social and professional topics → Software engineering
education; Model curricula;</p>
    </sec>
    <sec id="sec-2">
      <title>INTRODUCTION</title>
      <p>
        Software engineering as a discipline covers a large spectrum of
diferent concepts, from the very detailed technicalities of system
programming, to the highly abstract concepts of software
organizations and their process models [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. In this sense, this enables
the software engineering researchers a freedom to focus on many
diferent types of aspects within their discipline, but on the other
Copyright © 2018 for the individual papers by the papers' authors. Copying permitted
for private and academic purposes. This volume is published and copyrighted by its
editors.
      </p>
      <p>
        SWEPHD2018, September 17th, 2018, St. Petersburg, Russia
hand it also promotes an identity problem since the discipline might
not be very clearly defined for the university students [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        The ACM curricula for software engineering [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] recommends
that the discipline is approached by first introducing the diferent
software process models, methods and tools which are commonly
applied in the software engineering world. Even if there are studies
which question the value of these tools and process models in the
actual software industry [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], they provide the fundamental
models and base foundation, from which other concepts and modern
software engineering approaches are derived or developed.
      </p>
      <p>
        In this paper, we conduct a qualitative case study on three courses
following the general guidelines and recommendations of the ACM
2004 curricula for software engineering, and based on their
feedback, assess the student expectations and general needs of the
software engineering courses in the university-level education. First
one of the case study courses was aimed towards Master’s level
students and heavily applied distance learning approaches,
focusing on the software engineering methods and their application via
software engineering design tools such as the UML2 (for
example [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]). The second course focused on the fundamental process
models and concepts of the software engineering, and was aimed
towards Bachelor’s level students, organized with more traditional
lectures and weekly exercise events. The third course was fully
long-distance-enabled course in software testing and quality
assurance, intended for Bachelor’s level students, where the student was
required to only take the final exam as an local event.
      </p>
      <p>The results indicate that there are concerns on how the students
perceive software engineering as a discipline, and recommendations
on what actions should be taken to enable better learning outcomes,
and better motivation towards the software engineering as an area
of academic interest. Overall, our research questions are How should
the fundamental software engineering courses be taught to enable
more interest towards the subject? and What are the current areas of
interest for the student body in software engineering?.</p>
      <p>
        This paper is also a continuance study which follows our
software engineering research group’s prior case studies on software
engineering and university-level education, for example in the
topics of application of video-based course lectures [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], how
organizations adopt new practices of methods [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ], how international
standards can be applied in the self-assessment processes [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], or
how computer science course curricula applies in the games
industry context [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ].
      </p>
      <p>Rest of the paper is constructed as follows: In the Section 2
we discuss the recent research studies with the similar topics and
concepts, while in Section 3 the applied research approach is
described. Section 4 presents the case study results, while Section 5
provides discussion, observations and implications of these results.
The Section 6 closes the paper with the conclusions.
2</p>
    </sec>
    <sec id="sec-3">
      <title>RELATED RESEARCH</title>
      <p>The software engineering discipline covers several diferent areas
of interest, which makes the development of the course curricula
dificult. To address this issue of expectations vs. the course contents,
there are some studies which address the preferred methods of
approaching the software engineering course contents, and two
model curricula which include recommendations for the taught
topics.</p>
      <p>
        The most comprehensive model for software engineering
curriculum is the ACM SE curriculum, first published in 2004 and
supplemented with additions on the 2014 edition [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. This model
defines ten major categories for the software engineering from
computing essentials and computer science, to the software design,
human-computer interaction to the software quality, security and
professional practices. Overall, the curriculum has definite
emphasis on the design aspects of the software development, but shares
several features with other computing-related disciplines. in fact,
the ACM SE 2014 identifies that the capstone projects can be
focused also into the computer science aspects, to emphasize the
programming expertise if programming proficiency is expected in
the latter doctoral or master’s level programs.
      </p>
      <p>
        The other more common model for software engineering is the
IEEE Software Engineering Book of Knowledge (SWEBOK) with the
version 3 being the most recent model available [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. The SWEBOK
model divides the software engineering discipline into 15
knowledge areas, such as Software Testing, Management or Professional
practices. The model includes topics and definitions of "commonly
accepted best practices and methods" by peer review process, and
is constantly maintained with updates. However, the model does
not include a separate curriculum, nor does it recommend a degree
structure to cover the presented content.
      </p>
      <p>
        In the assessment of these models, Alarifi et al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] have defined a
method for assessing the actual development needs for the software
engineering courses, which follow the general guidelines such as
SWEBOK v3 or the ACM SE2004. Their study observes that the
courses which follow the principles of the more recent model
curriculas tend to perform better based on the course feedback, and
identify five key areas which are critical success factors: 1)
application of the latest SE tools and techniques, 2) Coverage of the
key concepts identified in the SWEBOK model, 3) involving the
actual software industry and practical assignments in the course, 4)
application of software engineering standards and finally, 5)
following accredited best practices such as NCAAA-accreditation in the
course contents. Interestingly, the intrinsic student motivation was
not found to be one of the critical factors, similarly as for example
observed by [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] with the fundamentals of programming and LEGO
robots; the students who are interested in the subject might even
consider some forms of ’motivational activities’ to have a negative
impact.
      </p>
      <p>
        On the flip side of the coin, Kuang and Han [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] discuss the
common reasons why software engineering courses fail. Their study
indicates, that the more common reasons are problems such as that
the taught methods and course content in general is outdated, the
given exercises do not prepare the students to work in the
software industry, and that the faculty members simply lack relevant
industry experience to provide real-world-equivalent learning
experiences. Similar to Alarifi et al., Kaung and Han promote the need
to constantly maintain and revise the course contents to match the
industry expectations and requirements to provide a realistic view
and realistic expectations for the students who are considering
software engineering as their main profession. This expectations and
capability gap has been also identified by Almi et al. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] in software
engineering, and for example also by McGill [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] in the software
engineering -related domain, entertainment software engineering,
games industry. In this sense, it can be argued that the expectation
gap is more universal problem, but in any case it is present in the
software engineering discipline.
3
      </p>
    </sec>
    <sec id="sec-4">
      <title>RESEARCH METHOD</title>
      <p>
        Assessing the feasibility and student motivation towards university
courses is not straightforward, since there might be several
underlying phenomena which might afect the outcomes. To compensate
for these issues, we applied an explanatory case study approach
following the general principles presented by Eisenhardt [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], applying
open coding as defined by Corbin and Strauss [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] in the available
qualitative data. As the objective was to assess the feasibility of the
case course implementations and to identify possibly problematic
areas, we found that this exploratory approach was suficient to
provide the data needed, and to allow us to compare our
observations against the prior research results discussed in the previous
section.
      </p>
      <p>To study the student expectations and the course contents ofered
in Software Engineering courses, three implementations of the
fundamental software engineering courses delivered by one faculty
in one university were selected for in-depth case study. The first
course which was selected for the analysis was Bachelor’s level
course Fundamentals of software engineering (Course A), arranged
in the academic year of 2017-2018, the second course Master’s level
course Software Engineering Methods (Course B), from the academic
year of 2013-2014 and the third one Fundamentals of Software Testing
(Course C), arranged in 2014-2015. The course contents included in
all courses included topics such as the common software process
models, commonly applied software standards, software design
tools -especially UML2-based-, software development environments
and an overview of the common software product architectures,
with boundary-control-entity-based (BCE) approach on the system
module modeling. In all courses, the students were not expected to
write algorithms or solve programming-based problems, but were
expected to understand source code, and be able to create software
engineering-related documents and designs based on the given
materials and project repositories.</p>
      <p>Combined, these three courses had over two hundred students,
representing the diferent study majors from computer science, to
industrial engineering and management, and electrical engineering
from one university. The Course A was arranged as a traditional
university course, with local weekly lectures and exercise events,
whereas the Courses B and C were arranged as a
long-distanceenabled courses applying course recordings and a separate tutorial
video archives. All courses had exercise events which afected the
course grade, plus at least one larger project involving teamwork
and a mandatory separate exam. These courses and their
components are defined in more detail in the Table 1.</p>
      <p>
        From these three case courses, the data analysis on the
expectations and the student considerations towards software engineering
was collected via course feedback, course activity, submitted works,
student-teacher communication and from the completion statistics.
The open feedback and other eligible components were analyzed
following the principles of open coding from the Straussian grounded
theory [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] to identify and classify the common themes, concepts
and problematic areas of these courses. The amount of material
did not warrant a full GT analysis, used to pinpoint the topics of
interest and to allow us to compare the results against the statistical
data, and known issues identified from the related research. In this
paper, we report only the issues which were identified both from
the qualitative feedback, and from the quantitative sources.
4
      </p>
    </sec>
    <sec id="sec-5">
      <title>RESULTS</title>
      <p>The case courses were organized three years apart, and applied two
diferent approaches on the course organization. The Course A was
more or less traditional university course with the focus on the
face-to-face teaching events with the course contents following a
textbook, whereas the Courses B and C were organized as an
longdistance-enabled course promoting self-study availability to select
your own working hours. In terms of traditional course performance
metrics, all of these courses performed very similarly; most of the
students who actually started the course -submitted at least one
graded exercise or project - completed all of the mandatory tasks,
and the course average in all three cases was similarly around the
midpoint of the grading scale. Similarly, the feedback indicated
that the students felt that the course time consumption and overall
dificulty was at an acceptable level. Interestingly, even though the
courses were similarly laborious and dificult, the "traditionally
organized" Course A had over 1 full grade worse student-given
grade (3.0 vs. 4.3/4.1), and from the observed cases, it also was the
only one where the amount of negative feedback superseded the
positive.</p>
      <p>Besides student workload and dificulty, the students also
performed on these courses at an acceptable level. The courses B and C
had over 75 percent pass rates when considering the students who
actually made an efort to participate and pass the course. Similarly,
the Course A had over 80 percent of the students eligible for the
ifnal grade, provided that they pass the final exam at some point in
the future. The general course statistics and the amount of collected
feedback is summarized in the Table 2.</p>
      <p>In the feedback for the courses, the students were requested to
list the topics they were expecting to learn from the course, and
how the course contents matched with the student expectations.
For example, in the Course C, the students were expecting to learn
real-life applicable experiences (59% of the feedback), Management
and design of software quality practices (50%), actual testing work
techniques (45%) and testing exercises with a real software project
and source code (36%). Similarly, the students in Course B ranked
the actual exercise events more important than the lectures (grade
3.95 vs. 3.86 on scale 1-5, 5 best), the demo cases more important
than the lectures (4.45 vs. 3.86) and both of the project works more
important than the lectures (4.45 and 4.23 vs. 3.86). The most
important diference between the separately graded course components
was between the video lectures and tutorial archive versus the
traditional lectures; the video-based approach was considerably
more favored than the face-to-face lecturing (4.68 vs. 3.86). Similar
observations were also made in both courses A and C; for example
in the Course A, the summary video was watched by 35 percent of
the students before the first exam, whereas in the last two lectures
only less than ten percent of the student population attended.</p>
      <p>In all three feedback surveys the students were also asked
comments on how they felt concerning the course arrangements and
the taught topics. To generally understand how students perceive
software engineering, the feedback were analyzed and coded for
observations as one dataset, where the diferences between the course
topics, and feedback concerning some course components such as
the exercise submission systems, were abstracted or ignored.</p>
      <p>In general level, there were more students which were interested
in the software engineering as a topic, than those who found the
topic to be boring or uninteresting (19% vs. 7% of given feedback).
The most common feedback theme, which was expressed by 25
percent of the respondents was that they take software engineering
courses because they want to learn practical skills, which can be
applied in a real-life scenario.</p>
      <p>[When asked on why did the student enroll to the course] "The
course really seems like it is training us to work in real-life, not just
showing things on the theoretical, abstract level." - Student from
Course C</p>
      <p>"It was nice to find out that the UML tools are not some mystery
box, but a set of practical and simple illustrations." -Student from
Course B</p>
      <p>This sentiment was somewhat similar to the fourth and fifth
most common items, "I want to learn about managing software
processes" (15% of feedback) which meant that the student was
interested in the management aspects of the software projects, and
"I want to learn about &lt;tool&gt;" (15%), which means that the student
was expecting to learn about some tools, technique or programming
language on that course.</p>
      <p>[What part would you keep unchanged if you could only select
one‘] "The actual software testing work in the exercises." - Student
from Course C</p>
      <p>"[The UML2 diagrams] are the most useful part of this course."
-Student from Course A</p>
      <p>Interestingly, the third most commented feature of the case study
courses was that the students actually would have preferred
projectbased grading over the course exams, and because of that were
complaining that the projects do not have enough impact on the course.
Some of the comments declared that the project work was very
eficient way to learn, and 10% of the feedback actually mentioned
this separately.</p>
      <p>"I personally found that demo assignments and projects were really
good way to learn. -Student from Course B</p>
      <p>"The project assignments really explained everything to me. I think
we could drop the separate exercises. Or the exam." -Student from
Course A</p>
      <p>On the negative feedback, it was interesting to observe, that the
most commented aspect was not the lack of motivation or lack of
interest towards the subject, but that the students considered the
course content or applied tools obsolete, or at least old-fashioned.
Other common complaint was that the applied textbooks and
exercises derived from those course materials were obsolete, or at best
borderline cases. This concern was raised by 13 % of the feedback.</p>
      <p>"I expected more advanced topics in software engineering like
mobile apps engineering or web applications engineering or even
consideration about future of software engineering." - Student from Course
B</p>
      <p>"Usage of better tools to create the diagrams. [Tool 1 name] is not
the best idea. Personally, I have found a tool called [Tool 2 name]
which is easier, with more functionalities and more stable. -Student
from Course B</p>
      <p>"Questions straight from the 70’s way of working with software
were unnecessary and in no way relevant to anything." -Student from
Course A</p>
      <p>"Besides being a student, I also work as a developer in [company].
None of the stuf shown here is no longer used, at least in my domain."
- Student from Course A</p>
      <p>Besides these items, the feedback also frequently mentioned
aspects such as that the student expected to be working with a
real-life software project (Open source or such) (13%), the course
theory-practice-balance was somehow of (7%), or that they lost
motivation to some other aspects of the course implementation,
such as unclear instructions (9%), dislike of the lectures or lecturing
style (6%) or that they simply did not find the topic interesting (7%),
usually because they were forced to take the course as a mandatory
part to their study program.</p>
      <p>"I simply do not find these exercises interesting" -Student from
Course C.</p>
    </sec>
    <sec id="sec-6">
      <title>5 DISCUSSION AND IMPLICATIONS</title>
      <p>Observing the student expectations and the success factors of the
software engineering courses based on three implementations of
university courses discussing the fundamental topics of software
engineering is obviously not enough to declare any new theories
or very deep insight, but it does provide insight into the student
perspective and provides implications on the possible areas of
interest. First of all, the given feedback promotes a number of clear
implications for the areas of interest:
(1) The students expect practically applicable skills, whether it
means new programming languages, development tools or
simply experience transferable to real-life scenarios.
(2) The preferred way of learning software engineering
discipline aims towards project and case exercises, instead of
simple programming tasks or theory and examinations.
(3) The most problematic area was not the intrinsic motivation
or general interest towards the software engineering as a
discipline, but the impression that the taught materials were
outdated and obsolete, making the learning efort
meaningless.</p>
      <p>
        These items more or less are in line with the observations made
by [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] and [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]; the software engineering students expect to learn
applicable information, and the course contents need constant
revisions and cooperation with the software industry partners to stay
relevant. Besides the open feedback, in our case courses this
observation was also observable from the course statistics: the Course A
which was also materials-wise oldest, was by students considered
the worst even if the overall dificulty and required efort was the
same. Even if there was a concern over the applied teaching
methods of local events versus online events, as discussed for example
in [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], the open feedback really did not reflect this. The students
did not voice concerns over the possibility for long-distance
studies, but were more critical towards grading methods, the relatively
low importance of project assignments and the fact that they want
to learn relevant, modern and practical skills. Few of the
projectrelated comments also included concerns over the workload of the
project, indicating that at least some of the participants were very
critical towards the possibility of social loafing [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], and wanted
more challenge and responsibility over their coursework.
      </p>
      <p>
        Considering the research questions, How should the fundamental
software engineering courses be taught to enable more interest towards
the subject? and What are the current areas of interest for the student
body in software engineering?, the implications of this case study are
clear. The students who are interested towards software engineering
in general, seem to be more interested towards practical work
and applicable experience, with the critical factor being that the
students perceive the course contents to be useful and relevant. The
approach into the teaching methods, long-distance enabled learning
environment nor the course arrangements, are the most relevant
concern if the course contents are seen something less than useful.
Based on our observed cases, the students in general do not require
a cutting edge digital infrastructure, if the materials and presented
course content is sensible and relevant. Similar observations and
considerations have been also identified and presented earlier, for
example by [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        However, it should be remembered that the course feedback
was given by 68 (29.8 %) students from the total of 228 course
enrollments, a minority of the entire pool of course participants.
The first impression might be that the feedback was only given
by the most vocal or active participants, but since the feedback
surveys were voluntary and to a varying degree anonymous, a
response rate of 29.8% can be considered acceptable for this type of
usage, similarly as described by Fink [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. In general, the qualitative
analysis of the feedback data surfaced a number of aspects which
should be taken into account in the future in the revisions of the
software engineering courses. As with all qualitative studies, these
observations are not strong, confirmatory results, but more like
guidelines or suggestions on what aspects to take into account
in the future work [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. Similarly, there are inherent risks in the
qualitative studies, which in any case mean that the qualitative
data, even when triangulated against the quantitative data sources,
cannot completely and universally explain the observations, only
describe the phenomena in that particular organization or
sociotechnical ecosystem [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ][
        <xref ref-type="bibr" rid="ref22">22</xref>
        ].
6
      </p>
    </sec>
    <sec id="sec-7">
      <title>CONCLUSIONS</title>
      <p>Software engineering as a discipline covers a vast area of diferent
concepts, and from the viewpoint of student, this might cause
confusion or identity problems for the software engineering students.
Furthermore, the modern student expect certain level of services,
which promotes motivation and interest towards the taught topics
and concepts. In this paper, we observed three university courses,
which all taught diferent fundamental software engineering
concepts.</p>
      <p>Based on our results, the software engineering students first and
foremost expect practically applicable skills and experience, and
require that the course materials are up-to-date and modern. A bit
surprisingly, the student feedback when comparing online-enabled
courses and a traditional lecture-based course, was focused on the
relevance of the data, not on the ability to participate long-distance,
or work at their own pace and schedules. Besides relevance, the
students expect to learn how to manage real-life software projects,
and prefer project-based approach over the theory-based lectures.</p>
      <p>These results obviously do not form a strong theory or validate
any prior works, but like qualitative studies in general, identify
and pinpoint potential problems, and provide suggestions or
recommendations which should be taken into account in the future
works in similar conditions. In our case, the future work would be
to modernize certain parts of the Course A analyzed in this study,
and observe the changes in the student feedback and considerations.
Other approach would be to include more advanced courses into
the analysis, and develop a framework or recommendations on how
the software engineering course design should be approached, or
how the current ACM SE-curricula matches the modern student
expectations.</p>
    </sec>
    <sec id="sec-8">
      <title>ACKNOWLEDGMENTS</title>
      <p>I would like to thank the course participants for their feedback and
efort, and acknowledge the support from both Lappeenranta
University of Technology (LUT) and South-Eastern Finland University
of Applied Sciences (XAMK).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>Arlene</given-names>
            <surname>Fink</surname>
          </string-name>
          (Ed.). [n. d.].
          <source>The survey kit (2nd ed ed.)</source>
          (
          <year>2003</year>
          ). Sage Publications.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2] [n. d.].
          <source>SWEBOK V3 âĂć IEEE Computer Society</source>
          . https://www.computer.org/ web/swebok/v3
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Frank</given-names>
            <surname>Achtenhagen</surname>
          </string-name>
          . [n. d.].
          <article-title>Criteria for the development of complex teachinglearning environments</article-title>
          .
          <volume>29</volume>
          ,
          <issue>4</issue>
          ([n. d.]),
          <fpage>361</fpage>
          -
          <lpage>380</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Abdulrahman</given-names>
            <surname>Alarifi</surname>
          </string-name>
          , Mohammad Zarour, Noura Alomar, Ziyad Alshaikh, and Mansour Alsaleh. [n. d.]. SECDEP:
          <article-title>Software engineering curricula development and evaluation process using SWEBOK</article-title>
          .
          <volume>74</volume>
          ([n. d.]),
          <fpage>114</fpage>
          -
          <lpage>126</lpage>
          . https://doi.org/10. 1016/j.infsof.
          <year>2016</year>
          .
          <volume>01</volume>
          .013
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Nurul</given-names>
            <surname>Ezza Asyikin Mohamed Almi</surname>
          </string-name>
          , Najwa Abdul Rahman, Durkadavi Purusothaman, and Shahida Sulaiman. [n. d.].
          <article-title>Software engineering education: The gap between industry's requirements and graduates' readiness</article-title>
          . IEEE,
          <fpage>542</fpage>
          -
          <lpage>547</lpage>
          . https://doi.org/10.1109/ISCI.
          <year>2011</year>
          .5958974
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>Mark</given-names>
            <surname>Ardis</surname>
          </string-name>
          , David Budgen, Greg Hislop, Mark Sebern, Jef Ofutt, and Willem Visser. [n. d.].
          <article-title>Software Engineering 2014 Curriculum Guidelines for Undergraduate Degree Programs in Software Engineering A Volume of the Computing Curricula Series</article-title>
          . https://www.acm.org/binaries/content/assets/education/se2014.pdf
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Juliet</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Corbin</surname>
          </string-name>
          and Anselm L. Strauss. [n. d.].
          <article-title>Basics of qualitative research: techniques and procedures for developing grounded theory (fourth edition ed</article-title>
          .). SAGE.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Kathleen</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Eisenhardt</surname>
          </string-name>
          . [n. d.].
          <source>Building Theories from Case Study Research</source>
          .
          <volume>14</volume>
          ,
          <issue>4</issue>
          ([n. d.]),
          <fpage>532</fpage>
          -
          <lpage>550</lpage>
          . https://doi.org/10.5465/amr.
          <year>1989</year>
          .4308385
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Ana</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>FernÃąndez-SÃąez</surname>
          </string-name>
          , Marcela Genero, and
          <string-name>
            <surname>Michel</surname>
            <given-names>R.V.</given-names>
          </string-name>
          <string-name>
            <surname>Chaudron</surname>
          </string-name>
          . [n. d.].
          <article-title>Empirical studies concerning the maintenance of UML diagrams and their use in the maintenance of code: A systematic mapping study</article-title>
          .
          <volume>55</volume>
          ,
          <issue>7</issue>
          ([n. d.]),
          <fpage>1119</fpage>
          -
          <lpage>1142</lpage>
          . https://doi.org/10.1016/j.infsof.
          <year>2012</year>
          .
          <volume>12</volume>
          .006
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>Nahid</given-names>
            <surname>Golafshani</surname>
          </string-name>
          . [n. d.].
          <source>Understanding Reliability and Validity in Qualitative Research</source>
          .
          <volume>8</volume>
          ,
          <issue>4</issue>
          ([n. d.]),
          <fpage>597</fpage>
          -
          <lpage>606</lpage>
          . https://nsuworks.nova.edu/tqr/vol8/iss4/6
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Antti</surname>
            <given-names>Herala</given-names>
          </string-name>
          , Antti Knutas, Erno Vanhala, and Jussi Kasurinen. [n. d.].
          <source>Experiences from Video Lectures in Software Engineering Education. 9</source>
          ,
          <issue>5</issue>
          ([n. d.]),
          <fpage>17</fpage>
          -
          <lpage>26</lpage>
          . https://doi.org/10.5815/ijmecs.
          <year>2017</year>
          .
          <volume>05</volume>
          .03
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Rune</surname>
            <given-names>HÃÿigaard</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Reidar</surname>
            <given-names>SÃďfvenbom</given-names>
          </string-name>
          , and Finn Egil TÃÿnnessen. [n. d.]. The Relationship Between Group Cohesion, Group Norms, and Perceived Social Loafing in Soccer Teams.
          <volume>37</volume>
          ,
          <issue>3</issue>
          ([n. d.]),
          <fpage>217</fpage>
          -
          <lpage>232</lpage>
          . https://doi.org/10.1177/ 1046496406287311
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Jussi</surname>
            <given-names>Kasurinen</given-names>
          </string-name>
          , Saeed Mirzaeifar, and Uolevi Nikula. [n. d.]. Computer Science Students Making Games:
          <article-title>A Study on Skill Gaps and Requirement</article-title>
          .
          <source>In Proceedings of the 13th Koli Calling International Conference on Computing Education Research</source>
          (
          <year>2013</year>
          )
          <article-title>(Koli Calling '13)</article-title>
          . ACM,
          <volume>33</volume>
          -
          <fpage>41</fpage>
          . https://doi.org/10.1145/2526968.2526972
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Jussi</surname>
            <given-names>Kasurinen</given-names>
          </string-name>
          , Per Runeson, Leah Riungu, and Kari Smolander. [n. d.].
          <article-title>A Self-assessment Framework for Finding Improvement Objectives with ISO/IEC 29119 Test Standard</article-title>
          .
          <source>In Systems, Software and Service Process Improvement (2011- 06-27) (Communications in Computer and Information Science)</source>
          . Springer, Berlin, Heidelberg,
          <fpage>25</fpage>
          -
          <lpage>36</lpage>
          . https://doi.org/10.1007/978-3-
          <fpage>642</fpage>
          -22206-
          <issue>1</issue>
          _
          <fpage>3</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>J.</given-names>
            <surname>Kasurinen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Taipale</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Smolander</surname>
          </string-name>
          . [n. d.].
          <source>How Test Organizations Adopt New Testing Practices and Methods?</source>
          .
          <source>In 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)</source>
          (
          <year>2011</year>
          -
          <fpage>03</fpage>
          ).
          <fpage>553</fpage>
          -
          <lpage>558</lpage>
          . https://doi.org/10.1109/ICSTW.
          <year>2011</year>
          .63
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Li-Qun Kuang</surname>
          </string-name>
          and Xie Han. [n. d.].
          <source>The Research of Software Engineering Curriculum Reform</source>
          .
          <volume>33</volume>
          ([n. d.]),
          <fpage>1762</fpage>
          -
          <lpage>1767</lpage>
          . https://doi.org/10.1016/j.phpro.
          <year>2012</year>
          .
          <volume>05</volume>
          .282
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Monica</surname>
            <given-names>M.</given-names>
          </string-name>
          <year>McGill</year>
          . [n. d.].
          <article-title>Learning to Program with Personal Robots: Influences on Student Motivation</article-title>
          .
          <volume>12</volume>
          ,
          <issue>1</issue>
          ([n. d.]),
          <volume>4</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>4</lpage>
          :
          <fpage>32</fpage>
          . https://doi.org/10.1145/2133797. 2133801
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>William</given-names>
            <surname>Isaac McWhorter and Brian C. O'Connor</surname>
          </string-name>
          . [n. d.].
          <article-title>Do LEGOÂő MindstormsÂő motivate students in CS1?</article-title>
          .
          <source>In ACM SIGCSE Bulletin</source>
          (
          <year>2009</year>
          ), Vol.
          <volume>41</volume>
          . ACM,
          <volume>438</volume>
          -
          <fpage>442</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>Dan</given-names>
            <surname>Pilone</surname>
          </string-name>
          and Neil Pitman. [n. d.
          <source>]. UML 2</source>
          .
          <article-title>0 in a nutshell (1st ed ed</article-title>
          .).
          <source>O'Reilly Media. OCLC: ocm58999170.</source>
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Roger</surname>
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Pressman</surname>
          </string-name>
          . [n. d.].
          <article-title>Software engineering: a practitioner's approach</article-title>
          (6th ed ed.).
          <source>McGraw-Hill. OCLC: ocm53848343.</source>
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>Colin</given-names>
            <surname>Robson and Kieran McCartan</surname>
          </string-name>
          . [n. d.].
          <article-title>Real world research: a resource for users of social research methods in applied settings (fourth edition ed</article-title>
          .). Wiley.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Robin</surname>
            <given-names>Whittemore</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Susan K. Chase</surname>
          </string-name>
          , and Carol Lynn Mandle. [n. d.].
          <source>Validity in Qualitative Research</source>
          .
          <volume>11</volume>
          ,
          <issue>4</issue>
          ([n. d.]),
          <fpage>522</fpage>
          -
          <lpage>537</lpage>
          . https://doi.org/10.1177/ 104973201129119299
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>