<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Group Programming in COVID-19 Time: The Experience of the Students of the ESI-CR of the UCLM</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Ana Isabel Molina Díaz</string-name>
          <email>anaisabel.molina@uclm.es</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Carmen Lacave Rodero</string-name>
          <email>carmen.lacave@uclm.es</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Dpto. Tecnologías y Sistemas de Información, Universidad de Castilla-La Mancha</institution>
          ,
          <addr-line>Ciudad Real</addr-line>
          ,
          <country>España</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Dpto. Tecnologías y Sistemas de Información, Universidad de Castilla-La Mancha</institution>
          ,
          <addr-line>Ciudad Real</addr-line>
          ,
          <country>España</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>-Collaborative learning activities have become a common practice in current university studies due to the implantation of the EHEA. But the COVID-19 pandemic has led to a radical and abrupt change in the teaching-learning model used in most universities, based on a face-to-face model. A rapid and unexpected adaptation to a new model of nonpresential teaching has been required, which has been able to be implemented, even in an improvised manner, thanks to the effort of teachers and students. Given this new scenario, our interest is mainly focused on discovering to what extent our students of the Computer Engineering Degree have approached the group programming tasks, which they must perform in a large number of subjects. For this purpose, we have carried out an experience aimed at finding the strategies and software tools they have used to address these tasks. The results of the study indicate that students have adopted a programming model based on work division or distributed peer programming, and very few have chosen to make use of synchronous distributed collaboration tools.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Keywords— Group programming, Peer Programming,
groupware, COVID-19</p>
    </sec>
    <sec id="sec-2">
      <title>I. INTRODUCTION</title>
      <p>
        The implantation of the European Higher Education Area
(EHEA) has led to group activities becoming a common
practice for university students [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ]. Specifically, the Degree
in Computer Engineering (GII) at the University of
CastillaLa Mancha (UCLM) promotes the development of group
programming projects of small and medium size in most
subjects [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. Working groups are usually formed by two
students who must cooperate to develop some program or
practical project. The most frequent ways to approach such
joint activities are the distribution of programming tasks in
different parts of the program/project (different files, modules
or functions), the use of shared repositories (Git, GitHub,
Google Drive, OneDrive, ...) or the application of Peer
Programming (PP) techniques [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], mainly in the context of
face-to-face laboratory classes.
      </p>
      <p>
        Pair Programming is the term used to describe the process
followed by two programmers working in the same computer,
performing a particular programming task or the design of an
algorithm. In this scenario two roles are defined: the driver,
who controls the programming activity and is responsible for
writing the source code; and the observer, who gives
indications to his/her partner about the development being
carried out, the existence of possible syntax errors, etc. Both
roles can be exchanged, alternating the control each team
member during the programming activity. Several studies
have proved that the use of the PP technique improves the
process of solving programming problems, the productivity of
the team, and the quality of the programs generated [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ].
      </p>
      <p>
        Pair Programming also requires working face-to-face in
the same location. When it is carried out in a distributed
environment, it is called Distributed Pair Programming
(DPP) [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. In this case, both team members collaborate
synchronously on the same programming task, but they are
geographically distant, so they must use specific collaboration
support tools (groupware) to develop their work [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. When the
number of programmers is not limited to two, the technique is
known as Collaborative Programming (CP). To ensure the
efficiency of this process, the tools used must incorporate
support mechanisms to the group activity (coordination,
access to shared information, awareness in the case of
working synchronously, ...) [
        <xref ref-type="bibr" rid="ref10 ref11 ref9">9, 10, 11</xref>
        ].
      </p>
      <p>
        In the second half of March 2020, the confinement due to
the COVID-19 pandemic forced a shift from a face-to-face to
an online education model from just one day to the next [
        <xref ref-type="bibr" rid="ref12 ref13">12,
13</xref>
        ]. In the case of the universities, each one provided different
tools for teachers and students to address this non-face-to-face
modality, facilitating the adaptation of methodologies,
planning and evaluation [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
      </p>
      <p>Within this process of improvised and rapid adaptation,
the UCLM decided to maintain the usual platform of online
communication with students (Moodle1) and the institutional
shared information repository (MS OneDrive2), as well as to
provide the university community with video conferencing
applications (MS Teams3), and video creation and playback
(MS Stream4). The combination of these tools allowed to solve
the problems of teacher-student communication in a more or
less satisfactory way. However, as teachers of the GII, and in
view that online teaching has come to stay (to a greater or
lesser extent), our interest is focused on knowing how the
student-student communication was approached in the context
of group programming tasks, which our students have had to
perform in most of the subjects. It is clear that, in the context
of the imposed confinement, students at all universities had to
make use of new strategies and work tools to move from a
traditional PP-based model to a DPP or CP approach.</p>
      <p>Therefore, this article describes the research experience
carried out at the Escuela Superior de Informática de Ciudad
Real (ESI-CR) of the UCLM, which aims to know the
mechanisms, tools and difficulties of the students of the GII,
1 https://moodle.org/?lang=es
2
https://www.microsoft.com/es-es/microsoft-365/onedrive/onlinecloud-storage
3
https://www.microsoft.com/es-es/microsoft-365/microsoftteams/group-chat-software
4 https://www.microsoft.com/es-es/microsoft-365/microsoft-stream
as well as their subjective perception, to address the group
programming tasks they performed during the state of alarm,
in the period between March and May 2020. Section 2
describes the details of the experience carried out
(questionnaire designed, results and discussion); and section 3
comments on the conclusions drawn from this work and the
work to be undertaken in the future.</p>
    </sec>
    <sec id="sec-3">
      <title>II. EXPERIENCE</title>
      <p>In an effort to find out how the students of the GII of the
ESI-CR carried out their group programming tasks during the
decreed state of alarm between March and May in Spain, an
experience was carried out with the voluntary and anonymous
participation of a total of 112 students (14 in the first year, 49
in the second year, 35 in the third year and 14 in the fourth
year). The experience took place during the first half of June
and consisted of collecting information of interest by means
of a questionnaire, displayed through the MS Forms5 tool, and
described below.</p>
      <sec id="sec-3-1">
        <title>A. Questionnaire</title>
        <p>Given the objective of this research, a questionnaire was
designed to find out the following aspects:
 Need for group programming activities in a distributed
way. This item asked whether they had to perform
group programming activities during the confinement
period.
 Size of the programming groups. Although in most
cases the groups consisted of two members, it was
asked whether the groups were made up of two, three
or more members.
</p>
        <p>How they have approached group programming tasks.
This item inquired about the solution adopted to
perform group programming tasks when the members
of the work team could not meet face-to-face. Several
answer options were provided (Table I), and several of
them could be selected. In the case of choosing one of
the last four options, an additional question was
enabled to indicate which tool/s they had used.
Note that option (d) is the one that best aligns with the
PP approach, but in a distributed (online) format: the
videoconferencing tool allows sharing the
development environment (IDE 6 ) (for instance,
Eclipse 7 ) so that the members of the couple can
alternatively take turns to adopt the roles of driver and
observer. On the other hand, option (e) refers to the use
of a groupware programming environment, which
would allow the application of a CP approach.
 Subjective perception about different strategies for
group programming. This aspect was integrated by
three items in which three possible strategies for group
programming were presented (Table II). Each of the
possible response options was ranked by means of a
5level Likert scale, which allowed to indicate the degree
of agreement (value closest to 5) or disagreement
(value closest to 1) with each one.
I have chosen not to do group practice. I have chosen
or changed (if the subject allowed it) to the modality
of individual work
We have distributed the practical work of different
subjects so that each member of the group can work
individually on each one of them, and not have the
need to work together on the same project/program
We have used version control systems (e.g., Git,
GitHub, ...) to work on the same programming
project, but asynchronously (not both at the same
time on the same project/file)
We have made use of video conferencing tools (e.g.
MS Teams or similar), sharing the screen or the IDE
We have made use of a synchronous collaborative
software development environment
We have made combined use of some of the above
options</p>
        <p>Statement
I believe that it is better to divide the work
when it is necessary to program in group in
the same practical project, i.e., that each group
member works in an independent way in a
certain component (ex. package, file, class, ...)
and, then, the contribution of each one is
integrated to the final result
I think that group programming is best done
in an asynchronous way (each member of the
group working on the same code/project, but
at different times), taking turns not to "step
on" the work
I think that group programming is best done
in a synchronous way (both partners working
at the same time on the same code), using
additional channels of chat, video or audio to
get organized and make decisions together

</p>
        <p>The first scenario proposes the division of work and
the progress in programming autonomously, in
different sections of the final program or project. In
other words, a cooperation strategy was presented, in
which it is possible to work in a synchronous or
asynchronous way, but in different parts of the code.
The second one suggests working on the same code but
in different moments of time, that is, it describes a
collaboration scenario, based on the assignment of
shifts and, therefore, asynchronous. The last scenario
corresponds to a purely collaborative and synchronous
group work.</p>
        <p>Need for tools to support distributed and synchronous
group programming. This section asked about the need
for tools to support collaborative synchronous work,
which matches the last scenario described in the
previous question.</p>
        <p>Features and functionalities needed to support
synchronous distributed programming activities.
Based on a hypothetical scenario of synchronous
distributed programming, a series of features and
functionalities that could be considered desirable, and</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>7 https://www.eclipse.org/</title>
      <p>even necessary, for effective and efficient group
programming are presented.
total) answered positively. Most of these students belonged to
2nd and 3rd year (Figure 1).</p>
      <p>
        Participants were asked to rate the usefulness/need for
each of these features or functions on a scale of 1 to 5,
with the lower end of the scale (1) representing that it
would not be necessary or useful at all, and the upper
end (5) indicating that it would be very necessary or
very useful. Table III shows the list of features, which
includes the main communication tool (text-chat,
audio and video), coordination and access control to
the shared workspace (blocking of code sections,
version control) mechanisms, as well as aspects related
to awareness (connected users and visual highlighting
of access to the shared area) [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. Awareness [
        <xref ref-type="bibr" rid="ref15 ref16">15, 16</xref>
        ]
is the set of visualization techniques that are
incorporated into the user interface of collaborative
applications to provide information about group
activity, that is, visual information about the people the
user is working with, the activities they are carrying out
and about which part of the shared artefact they are
working with.
      </p>
      <p>Finally, an item in which students could indicate any
feature or functionality not listed that they considered
necessary or useful was included.</p>
      <p>The preprocessing of the data provided by the participants
in the questionnaire reduced the sample size to N=111. The
subsequent analysis of the responses provided yielded very
interesting results, which are described below.</p>
      <p>Regarding the need to perform group programming tasks
during the confinement, 98 of the 111 students (88% of the</p>
    </sec>
    <sec id="sec-5">
      <title>8 https://discord.com/</title>
    </sec>
    <sec id="sec-6">
      <title>9 https://github.com/</title>
      <p>As for the size of the groups in which they participated, 52
of the 98 students indicated that they programmed in pairs, 16
in groups of three and 13 in groups of more than three.
Another 17 indicated that they were part of several groups of
different sizes.</p>
      <p>The solutions adopted to deal with group programming
(Table I) consisted, mostly, in the combination of various
strategies, highlighting the use of some videoconferencing
system (being MS Teams and Discord 8 the most cited)
together with an asynchronous version control system
(GitHub9 was the most outstanding). Figure 2 also shows the
percentage of students who chose to use only one tool.</p>
      <p>Among those who combined several strategies, five
participants from the third and fourth year indicated that they
used a synchronous collaborative development environment.</p>
      <p>Table IV shows the main descriptive statistics (mean,
median and mode) of the answers related to the participants'
subjective perception about the convenience of applying the
three group programming scenarios described in the previous
section (cooperation, asynchronous collaboration and
synchronous collaboration) (Table II), the need to have tools
to support the last of these three scenarios, in a distributed
context (synchronous distributed collaborative
programming), and, finally, the functionalities and features
they considered most necessary or useful in a tool to support
this programming strategy (Table III).</p>
      <p>As far as the different strategies proposed (Figure 3), the
best evaluated was synchronous collaborative programming
(µ=4.00), while asynchronous collaboration modality was the
worst valued (µ=2.74). Most of the students considered that
having tools to support synchronous distributed programming
scenarios should be necessary (µ=4.00). The features they
considered most useful for the software supporting this
programming strategy (Figure 4) were that the collaborative
functionalities should be integrated in a known IDE (e.g.
Eclipse) (µ=4.05), and that both version control support
(µ=4.29) and the recording of individual contributions made
by each team member to the final result should be included
(µ=4.23). With respect the communication mechanisms that
should be incorporated, the best rated was audio (µ=4.21),
followed by chat (µ=4.07), with the video signal being the one
they considered the least useful (µ=3.26).
adopted, in which only 30 out of 98 students have selected the
use of version control tools (GitHub, for the majority), among
which there are no first-year students and 68% are third year
students.</p>
      <sec id="sec-6-1">
        <title>C. Discussion</title>
        <p>
          The results obtained show that, despite the fact that
students positively valued synchronous distributed
programming (CP), they have opted, as a first option, for a PP
(driver-observer roles) programming model, but in an online
mode (DPP) during the confinement period. Most of them
have made use of MS Teams or Discord, sharing the
development IDE (Eclipse for Java; MS Visual Studio10 for
Visual Basic, RStudio11 for R and Visual Studio Code12 for C
and ADA) and changing turns alternatively to code. In the
same way that teachers have opted to transfer the face-to-face
magisterial class model to online support, in what has been
referred as remote emergency teaching [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], students have
opted for a similar approach. In most cases, they extrapolated
the way they work in the practice laboratories to a distributed
model. Very few students made use of a distributed and
synchronous programming environment, possibly due to a
lack of knowledge of the one that would suit their needs. In
the very few cases that they did so, the tools used were Google
Colab13 for Python programming and MS Visual Studio Live
Share14 or Atom15 for C and ADA programming.
        </p>
        <p>Among the desirable features included in a software for
group programming, at the same time and in a distributed way,
they considered that having an audio channel can be very
useful and, possibly, the most agile method to communicate.
The video signal is not considered as very necessary, being in
many cases rather a source of distraction, while textual
communication through a chat, that they are very used to, is
also well valued.</p>
        <p>
          Version control and the possibility of recovering previous
states of the practical projects were highly appreciated by
students, for their obvious usefulness [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ], although the fact
that no first-year students and very few second-year students
used them suggests that using this type of tool requires a
certain "maturity" not only in the use of technology, but also
in how to address group work. Therefore, and considering the
advantages that the use of these kinds of tools could offer to
GII students [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ], it would be necessary to consider for the
future introducing students to the use of version control
systems in the first or second year.
        </p>
        <p>On the other hand, those who best value version control
tools also consider it necessary to record the individual
contributions of each team member to the final result. This
feature, in order to evaluate and justify the personal
involvement in the deliveries, is very useful for both teachers
and students.</p>
        <p>The incorporation of awareness mechanisms was
considered very necessary (µ=4.36), although the possibility
of having visible information about the users connected or
their availability at any given time (µ=3.88) was considered
not very useful.</p>
        <p>Finally, we analysed whether the assessment of each of
these features depends on some factors, such as the course in
which the student was enrolled or the size of the groups in
which they had worked. The ANOVA of the data reflected
significant differences at 95% (p-value=0.003) only in the
version control tool when considering the student's course as
a factor. The subsequent post-hoc revealed that these
differences occur between students of the 1st and 2nd courses
with respect to those of the 3rd and 4th courses. These results
are in line with the data reflected in the section on the solutions
10 https://visualstudio.microsoft.com/es/vs/
11 https://rstudio.com/
12 https://code.visualstudio.com/
13 https://colab.research.google.com/notebooks/intro.ipynb
14 https://visualstudio.microsoft.com/es/services/live-share/
15 https://atom.io/</p>
        <p>
          In our research group (CHICO 16) the support of group
programming has been, for years, a topic of interest [
          <xref ref-type="bibr" rid="ref10 ref19 ref9">9, 10, 19</xref>
          ].
Among the latest developments, the COLLECE 2.017 system
stands out. This is a synchronous collaborative programming
environment that incorporates many of the features that have
been most appreciated by the students in this study (it is a
plugin integrated in Eclipse, which incorporates awareness
mechanisms, version control, blocking of code regions, ...).
Although several pilot experiences of the use of this system
have been carried out with students of the ESI-CR [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ], this
software has not yet been implanted as a tool of habitual use
in class, so we are considering to use it in several programming
subjects during the course 2020-2021.
        </p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>III. CONCLUSSIONS</title>
      <p>In this article we have described an experience conducted
with more than a hundred students of the Computer
Engineering Degree of the ESI-CR of the UCLM, whose
objective was to know how they had approached the practical
tasks of group programming during the state of alarm decreed
in the second quarter of the 2019-2020 academic year.</p>
      <p>The results obtained show that, although the students
considered interesting the use of synchronous collaborative
programming tools, that is, to apply a CP approach, they
mostly opted for a DPP model (in which they share the IDE
with their colleagues, making use of videoconferencing
applications). The division of programming tasks in different
parts of the program or project was the second most used
option. Possibly the lack of knowledge of support tools for
distributed synchronous programming influenced the choice
of these strategies.</p>
      <p>Our research group has developed several environments
that implement the three programming approaches (PP, DPP
and CP). Outstanding among these systems is COLLECE 2.0,
which we plan to introduce in several programming subjects
during the next academic year. This system incorporates many
of the features considered most useful by the participants in
this study (it is integrated a widely used IDE, such as Eclipse,
incorporates a version control system, blocking of code
regions, a very rich set of awareness mechanisms, ...). Even
so, some of the features that have also been positively valued
by the students could be added to this system, such as the
incorporation of an audio channel between the team members.
Similarly, we plan to continue studying the tools that support
CP through a systematic literature review, which allows us to
know the state of current research in this field.</p>
    </sec>
    <sec id="sec-8">
      <title>ACKNOWLEDGEMENTS</title>
      <p>This work has been funded by the Ministry of Economy,
Industry and Competitiveness and by the European Regional
Development Fund, with reference TIN2015-66731-C2-2-R.</p>
      <p>The authors would also like to express their gratitude to the
students who volunteered to participate in this experience.
16 https://blog.uclm.es/grupochico/
17 http://blog.uclm.es/grupochico/proyecto-iapro/collece-2-0/</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1] MECD. “
          <article-title>La Integración del Sistema Universitario Español en el Espacio Europeo de Educación Superior”</article-title>
          , Ministerio de Educación, Cultura y Deporte,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>R.</given-names>
            <surname>Olanda</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Sebastian</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. I. Panach.</surname>
          </string-name>
          <article-title>Aprendizaje colaborativo basado en tecnologías multimedia</article-title>
          .
          <source>Proceedings of the XX Jornadas de Enseñanza Universitaria de la Informática (JENUI</source>
          <year>2014</year>
          ),
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3] UCLM. “
          <article-title>Memoria para la solicitud de verificación de títulos oficiales</article-title>
          . Propuesta de título de Graduado en Ingeniería Informática”,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>L.</given-names>
            <surname>Williams</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. R.</given-names>
            <surname>Kessler</surname>
          </string-name>
          .
          <article-title>Pair programming illuminated</article-title>
          .
          <source>AddisonWesley Professional</source>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S.</given-names>
            <surname>Faja</surname>
          </string-name>
          .
          <article-title>Evaluating effectiveness of pair programming as a teaching tool in programming courses</article-title>
          .
          <source>Information Systems Education Journal</source>
          ,
          <volume>12</volume>
          (
          <issue>6</issue>
          ),
          <fpage>36</fpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>L. A.</given-names>
            <surname>Williams</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. R.</given-names>
            <surname>Kessler</surname>
          </string-name>
          .
          <article-title>All I really need to know about pair programming I learned in kindergarten</article-title>
          .
          <source>Communications of the ACM</source>
          ,
          <volume>43</volume>
          (
          <issue>5</issue>
          ),
          <fpage>108</fpage>
          -
          <lpage>114</lpage>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>P.</given-names>
            <surname>Baheti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Gehringer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Stotts</surname>
          </string-name>
          .
          <article-title>Exploring the efficacy of distributed pair programming</article-title>
          .
          <source>In Proceedings of the Conference on Extreme Programming and Agile Methods</source>
          (pp.
          <fpage>208</fpage>
          -
          <lpage>220</lpage>
          ). Springer, Berlin, Heidelberg.
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>B. J. da Silva</given-names>
            <surname>Estácio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Prikladnicki</surname>
          </string-name>
          .
          <article-title>Distributed pair programming: A systematic literature review</article-title>
          .
          <source>Information and Software Technology</source>
          ,
          <volume>63</volume>
          ,
          <fpage>1</fpage>
          -
          <lpage>10</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>C.</given-names>
            <surname>Bravo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Duque</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Gallardo</surname>
          </string-name>
          .
          <article-title>A groupware system to support collaborative programming: Design and experiences</article-title>
          .
          <source>Journal of Systems and Software</source>
          ,
          <volume>86</volume>
          (
          <issue>7</issue>
          ),
          <fpage>1759</fpage>
          -
          <lpage>1771</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>A. I.</given-names>
            <surname>Molina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Gallardo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Redondo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bravo</surname>
          </string-name>
          .
          <article-title>Assessing the awareness mechanisms of a collaborative programming support system</article-title>
          .
          <source>Dyna</source>
          ,
          <volume>82</volume>
          (
          <issue>193</issue>
          ),
          <fpage>212</fpage>
          -
          <lpage>222</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>F.</given-names>
            <surname>Jurado</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. I.</given-names>
            <surname>Molina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Redondo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ortega</surname>
          </string-name>
          . Coleprogramming:
          <article-title>Shaping collaborative learning support in eclipse</article-title>
          .
          <source>IEEE Revista Iberoamericana de Tecnologías del Aprendizaje</source>
          ,
          <volume>8</volume>
          (
          <issue>4</issue>
          ),
          <fpage>153</fpage>
          -
          <lpage>162</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>H.</given-names>
            <surname>Fardoun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Yousef</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>González-González</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.A.</given-names>
            <surname>Collazos</surname>
          </string-name>
          .
          <article-title>Estudio exploratorio en iberoamérica sobre procesos de enseñanza-aprendizaje y propuesta de evaluación en tiempos de pandemia</article-title>
          .
          <source>Education in the Knowledge Society (EKS)</source>
          ,
          <volume>21</volume>
          ,
          <issue>9</issue>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>A.</given-names>
            <surname>Skulmowski</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. D.</given-names>
            <surname>Rey</surname>
          </string-name>
          .
          <article-title>COVID‐19 as an accelerator for digitalization at a German university: Establishing hybrid campuses in times of crisis</article-title>
          .
          <source>Human Behavior and Emerging Technologies</source>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14] AENUI. “
          <article-title>Declaración de AENUI sobre retos educativos para el curso 2020-2021”</article-title>
          . http://www.aenui.net/ (Junio,
          <year>2020</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>P.</given-names>
            <surname>Dourish</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Bellotti</surname>
          </string-name>
          .
          <article-title>Awareness and coordination in shared workspaces</article-title>
          .
          <source>In Proceedings of the 1992 ACM conference on Computer-supported cooperative work</source>
          (pp.
          <fpage>107</fpage>
          -
          <lpage>114</lpage>
          ),
          <year>1992</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>C. A.</given-names>
            <surname>Collazos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. L.</given-names>
            <surname>Gutiérrez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Gallardo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ortega</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. M.</given-names>
            <surname>Fardoun</surname>
          </string-name>
          ,
          <string-name>
            <surname>A. I. Molina.</surname>
          </string-name>
          <article-title>Descriptive theory of awareness for groupware development</article-title>
          .
          <source>Journal of Ambient Intelligence and Humanized Computing</source>
          ,
          <volume>10</volume>
          (
          <issue>12</issue>
          ),
          <fpage>4789</fpage>
          -
          <lpage>4818</lpage>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>K. M. Ying</surname>
            ,
            <given-names>K. E.</given-names>
          </string-name>
          <string-name>
            <surname>Boyer</surname>
          </string-name>
          . Understanding Students'
          <article-title>Needs for Better Collaborative Coding Tools</article-title>
          .
          <source>In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems</source>
          (pp.
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          ), April,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Lu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Mao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Yin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Li</surname>
          </string-name>
          .
          <article-title>Improving students' programming quality with the continuous inspection process: a social coding perspective</article-title>
          .
          <source>Frontiers of Computer Science</source>
          ,
          <volume>14</volume>
          (
          <issue>5</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>18</lpage>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>M.</given-names>
            <surname>Ortega</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Redondo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bravo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. I.</given-names>
            <surname>Molina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Lacave</surname>
          </string-name>
          , Y. Arroyo,... and
          <string-name>
            <given-names>D.</given-names>
            <surname>Fuentes</surname>
          </string-name>
          .
          <article-title>CHICO 2019 (Computer-Human Interaction and Collaboration)</article-title>
          ,
          <source>UCLM. IE Comunicaciones</source>
          ,
          <volume>30</volume>
          (
          <issue>30</issue>
          ),
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>C.</given-names>
            <surname>Lacave</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>García</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. I.</given-names>
            <surname>Molina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Sánchez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Redondo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ortega</surname>
          </string-name>
          .
          <article-title>COLLECE-2.0: A real-time collaborative programming system on Eclipse</article-title>
          .
          <source>In Proceedings of the XXI International Symposium on Computers in Education (SIIE</source>
          <year>2019</year>
          )
          <article-title>(pp</article-title>
          .
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          ). IEEE,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>