<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>A pedagogical agent with embedded data mining functions to support collaborative writing</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Daniel Epstein</string-name>
          <email>daepstein@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Eliseo Reategui</string-name>
          <email>eliseoreategui@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Graduate Program of Computers in Education, Federal University of Rio Grande do Sul, (UFRGS)</institution>
          ,
          <addr-line>Porto Alegre -</addr-line>
          <country country="BR">Brazil</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Internet growth has induced the development of a large number of collaborative tools for online writing and information sharing. Educators quickly realized the bene ts of such tools for learners, allowing them to work online, to share their knowledge and help each other. Distance learning is a key concept in today's educational research; collaborative learning environments are becoming widespread, being more dynamic and resourceful. However, distance learning also introduced a series of problems, such as high evasion rates resulting from lack of support and personalized feedback. It has also introduced di culties for educators to follow and reviews students' assignments. Based on this scenario, the work presented here proposes the development of a pedagogical agent supported by an intelligent tutoring system to provide students and teachers with assistance in order to minimize some of these problems. The use of a pedagogical agent allows students to have a constant feedback and guidance based on the identi cation of problems that may emerge from an online collaborative writing activity. The presence of this agent is intended to help students coordinate their e orts in writing a text collaboratively, and improve their work in terms of coherence. Furthermore, the pedagogical agent is also be able to assist teachers, reporting problems and simplifying their tasks related to the analysis of the work produced by each student. To support our pedagogical agent we propose the use of data mining tools to extract information related to the students' writings, and a recommender system to suggest additional resources.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Pedagogical Agent</kwd>
        <kwd>Intelligent Tutoring System</kwd>
        <kwd>Data Mining</kwd>
        <kwd>Collaborative</kwd>
        <kwd>Distance Learning</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Alongside with the internet growth and online collaborative
learning environments, data mining has become more
important and popular in the education eld. From the student's
perspective, the possibility to easily search for learning
material using indexes and other reference tools increases their
resources at the same time that reduces the e ort needed to
nd relevant information. From the teachers' perspective, it
can help them to summarize students' writings in a distant
learning context [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], to assess the quality of posts in
discussion forums [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] and to evaluate the students participation
in discussion forums [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Furthermore, it can provide
useful feedback to teachers so they can easily identify the main
concepts in students' writings and the connection between
these concepts [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>
        The possibility to work collaboratively is attractive both
for students and teachers. It allows students to exchange
knowledge, to help each other and complement their work
with di erent ideas. However, it also creates several di
culties for teachers when evaluating the work done, once it's
hard and demanding to monitor each student production
[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. Besides, in a distance learning context, certain barriers
may hinder the establishment and maintenance of distance
learning programs, such as technical problems,
infrastructure, motivational di culties, necessary skills, social
problems and time/interruptions [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. This project's goal is to
contribute with the development of a social learning
environment in which collaborative writing takes place in a cohesive
way, minimizing technical and interactive/social di culties
that are inherent to online collaborative work.
      </p>
      <p>This work has also been motivated by the fast increase in
the number of collaborative writing tools available online,
and the problems that often originate from their use. For
instance, the lack of interaction and actual collaboration in
collaborative writing tasks, and the lack of supervision and
feedback to students' work.</p>
      <p>This project proposes a pedagogical agent to be used in
a collaborative writing environment, so that it may assist
teachers and students in their tasks. The agent uses data
mining to identify problems in the students' writings, and
based on this information it tries to guide them in improving
their collaborative text production. Besides, the agent gives
teachers more accurate information about the students'
participation, their di culties and interactive writing process.</p>
    </sec>
    <sec id="sec-2">
      <title>2. RELATED WORK</title>
      <p>
        Intelligent tutoring systems (ITS ) are computer tools
capable of providing customized instruction or feedback to
learners [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. They usually operate without the need of human
intervention. They di er from traditional content-delivery
computerized learning systems for their ability to improve
the e ectiveness of a learner's experience through the use of
an arti cial intelligence [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>
        ITS often uses a variety of computational resources for
analyzing the users interaction with the system. These are
adaptive mechanisms, capable of personalizing learning
according to individual student characteristics, such as
knowledge on the subject, mood and emotion [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] and learning style
[
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. They may be programmed to identify user's
information as they interact with the system and choose from many
actions the one that most likely would be bene cial to each
particular user.
      </p>
      <p>
        However, the ability to properly help the user often depends
on the interface between the system and the user. Not rarely,
ITS require a virtual character to interact with users. These
characters are called pedagogical agents. The use of
pedagogical agent (PA) in educational applications has
demonstrated that these animated characters may improve
student's engagement and learning experience [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
A PA is a human-like virtual character that has the
advantage to operate continuously and autonomously. It is
capable of searching and interpreting information received or
perceived through the system and provides a more natural
interaction with the user. PAs are capable of adapting their
actions and interventions, providing feedback and guiding
problem solving, re ection, understanding and collaborative
learning [
        <xref ref-type="bibr" rid="ref12 ref13 ref14">12, 13, 14</xref>
        ].
      </p>
      <p>
        Among the many bene ts of using PAs are the increase of
motivation, perception of ease and comfort in the
learning environment, the promotion of fundamental behaviors of
learning, the realization of a need for personal relationships
in learning and gains in terms of memory, understanding
and problem solving [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. Not only PAs can present
contents to the users, as they may suggest additional resources,
highlighting important issues and recommending new
exercises and reference materials according to user's progress
[
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. Studies have shown that the use of a PA with text
mining features could help students bring relevant contributions
to a reading discussion [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]
According to sociocultural theory, learning can be
considered a regulatory process that is mediated by social
interaction among individuals, cultural artifacts (computer,
pedagogical agent) and speech [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. Users' interactions help them
in knowledge constructing and sharing, being internalized by
them during this process. The experiences and knowledge
acquired may be reused in their future experiences [
        <xref ref-type="bibr" rid="ref19 ref20 ref21 ref22">19, 20,
21, 22</xref>
        ]. The PA may also be seen is a tool capable of
mediating learning, once it may interact with learners, individualize
feedbacks and foster autonomy and collaborative skills.
Social interaction, according to the sociocultural perspective,
is essential for the promotion of learning and development
[
        <xref ref-type="bibr" rid="ref19 ref20 ref21">19, 20, 21</xref>
        ].
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. RESEARCH PROPOSAL</title>
      <p>Based on the highlighted di culties of the development of
collaborative work online, we propose a pedagogical agent
to be inserted in an online collaborative writing
environment. This agent will be capable of helping students through
immediate feedback about their collaborative text
production, and it will assist teachers through the presentation of
information/indicator regarding students' participation and
progress in the assigned tasks. Our goal is to provide a full
time assistance to the users of the collaborative environment
and to reduce the amount of work needed to analyze their
interactions and work.</p>
      <p>In order to do that, we designed a pedagogical agent to
be integrated in the intelligent tutoring system. The ITS
will be responsible for collecting all information regarding
the students activities in the environment. The student
could be adding text, images, audio, video or any other
resource to the project or simply reviewing and modifying
some previous work. In any case, the ITS must keep a log
of those interactions in order to determine which action to
perform (when needed). Among the di erent information
collected by the system, we may list: the student's
contribution to the project (either by the addition of new contents
or revision/edition of previous work); frequency (how many
times each student accessed the collaborative environment,
for how long and when) and a concept map summarizing
what has been written by each student. Once we are
considering a collaborative environment, several users may be
at the system at the same time and it is important for the
ITS to identify which student is responsible for which
action. Whether the students access the environment
simultaneously or separated, it is important for all users to have an
identi er that will inform the system which user is current
online and modifying any given document.</p>
      <p>
        All the information collected by the ITS will be processed
using data mining techniques. This allows the system to
identify what type of contribution the user has made to the
project and infer how cohesive and coherent the text
produced collaboratively is. A data miner similar to Sobek[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]
will be used to perform these tasks. Sobek is a text miner
that uses statistical analysis to obtain the most relevant
concepts in a text and the relationships between them. A data
mining process will be used to convert multimedia resources
present in the project in concepts and relations.
The system will combine the data extracted from the users'
writings with the data provided by the teacher to evaluate
if the students project is related to the requirements. In
order to do it, the agent will compare the concepts and
relationships extracted from the students' writings with the
concepts and relationships extracted from the task
specication and resources provided by the teacher. Breno et.
al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] showed that this kind of comparison could provide
useful information regarding the quality/relevance of
students' contributions in discussion forums. The results of
this comparison will determine if the PA has to make any
intervention to help students improve their text. It is
particularly important for this intervention to result in positive
reinforcement.
      </p>
      <p>There are several types of interventions planned for both
students and teacher. The most common type of
interventions for students is a direct message sent by the PA
inquiring about some aspect of the project. Those inquiries are
intended to foster critical thinking and help students
correct what the ITS identi es as a problem. The messages
are sent when the students' work is incomplete or lack
cohesion. In both cases, it is possible for the agent to suggest
additional material that may help them correct the
problem. Another possible intervention is the use of e-mail
messages when students are not participating in the
collaborative work, or when their contributions are not coherent with
the remaining project. The last type of intervention is not
an automatic answer from the PA, but an explicit request for
help from a student. Speci c functions are being developed
to enable students to ask the PA for further information
about some aspect of the project, or about its structure and
coherence.</p>
      <p>
        A key feature in those interactions between PA and students
is the agent's ability to identify additional resources that
may help the students' text production. As the students'
project may include several types of media and di erent
format of resources, the agent is able to recommend students
with additional learning objects extracted from repositories
as well as from the web. Learning objects are usually
indexed using metadata, where keyword and object
descriptions are very common, information that is used to search
for learning objects that are most related to the topic at
hand. Using a similar technique to the one used by Breno
et. al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], the learning objects that present the highest
similarity values are presented to the students. Learning objects
selected will be separated by their format (video, audio,
image, text, etc.) so that it will be easier for the student to
select the most appropriate ones.
      </p>
      <p>Some of the PA's functions are speci c to help teachers. As
it is often di cult and time consuming for teachers to
analyze the individual production of each student in
collaborative writing tasks, the PA will provide accurate information
about each student's contributions and progress based on
the information collected by the ITS. This will allow
teachers to identify relevant contributions made by each student
or the absence of a student in a particular subject or part
of the project.</p>
      <p>
        The interaction between the PA and the teachers will be
set in two ways. The rst one is through direct messages
sent by the PA to the teacher's email or other
communication method. Those messages are sent to inform about lack
of participation from some student or if the system detects
that one or more students need further assistance with the
project (this may be identi ed through a constant request for
help from a student, when his/her work is always identi ed
to have coherence/cohesion problems, or when the student's
work is constantly edited by other students). The second
form of interaction will be a support interface, where the
teacher will be able to request speci c information
regarding student's activities in the environment. All those PA
features are meant to provide a more personalized contact
between the agent, the students and/or teachers.
Another aspect the agent is concern about is the coherence
of the project. The students may not be together when
writing the project and it may result in disjoint texts,
unrelated or redundant information. Therefore, it is important
for the agent to identify coherence problems and contact the
students who produced the incoherent parts of the project.
The agent may also use the information gathered from those
incoherent parts to search for additional material and
learning objects that could help students ll the possible gaps in
the project. Although we considered using Latent
Semantic Analysis to perform this task [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ], it is still problematic
to decide on how to interpret multimedia resources that are
neither text nor learning objects (in which case it is possible
to use its keywords and descriptions).
      </p>
    </sec>
    <sec id="sec-4">
      <title>4. STATE OF THE PROJECT</title>
      <p>In the current state of the project, we are developing a script
program that will collect the data from students' writings
and send it to the pedagogical agent. This is part of the
development of our intelligent tutoring system. This is also
one of the most challenging parts of this project, as it is
important to correctly interpret and evaluate multimedia
resources. Through the use of scripts without a particular
user interface, we intend to create a more reusable system,
allowing it to be used in many tools and environments
without the need for changing the program's core.</p>
      <p>Our data mining system based on Sobek is already being
modi ed to provide useful information regarding resource
similarity, concepts and relationships. The input for our
data miner is very restrictive, but we are working on
making it more general. This is most useful to investigate how to
successfully mine multimedia resources. Using the learning
object repository, we may conduct experiments that will
assess the quality of the results and the reliability of our data
mining tool.</p>
      <p>The experiments will be carried out in projects using Google
Drive 1. The choice for this environment has been the
extensive database and number of projects that are developed
with this tecnology daily. It is one of the most known and
complete environment for collaborative writing, and it also
allows the input of scripts that could facilitate the
integration of our pedagogical agent with the system. Google
support and APIs make it a natural choice for our project to
be integrated in Google Drive Documents, to ensure the
dissemination of our ideas and software to a larger number of
people across the world.
1https://drive.google.com
Daniel Epstein has Master Degree in Computer Science
from Universidade Federal do Rio Grande do Sul working
with robotics and arti cial intelligence. He is a Ph.D.
student at the Post Graduation Program in Computer Science
in Education, currently researching Data Mining and
Pedagogical Agent use on collaborative learning. He started the
program in March 2013 and is currently writing his thesis
proposal. He will complete his PhD in the middle of 2016.
Eliseo Berni Reategui has a PhD degree in Computer
Science from the University of London, England, and MS in
Computer Science from Universidade Federal do Rio Grande
do Sul, Brazil. After nishing his PhD and working in the
industry for 5 years, Dr. Reategui held a lecturer position at
the University of Caxias do Sul, Brazil, for a few more years.
Nowadays, he works as a lecturer and researcher at the
Federal University of Rio Grande do Sul. His research interests
are related to the use of computers in education, involving
areas such as arti cial intelligence and human-computer
interaction.
The main topics on which the student would like to receive
advices are:</p>
      <sec id="sec-4-1">
        <title>Data Mining</title>
      </sec>
      <sec id="sec-4-2">
        <title>Recommender system</title>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>S. G.</given-names>
            <surname>Tammaro</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. N.</given-names>
            <surname>Mosier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. C.</given-names>
            <surname>Goodwin</surname>
          </string-name>
          , and G. Spitz, \
          <article-title>Collaborative writing is hard to support: A eld study of collaborative writing." Computer Supported Cooperative Work</article-title>
          , vol.
          <volume>6</volume>
          , no.
          <issue>1</issue>
          , pp.
          <volume>19</volume>
          {
          <issue>51</issue>
          ,
          <year>1997</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>E.</given-names>
            <surname>Reategui</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Epstein</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Lorenzatti</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Klemann</surname>
          </string-name>
          , \
          <article-title>Sobek: a text mining tool for educational applications</article-title>
          ,"
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Chen</surname>
          </string-name>
          , X. Cheng, and
          <string-name>
            <given-names>Y.</given-names>
            <surname>Huang</surname>
          </string-name>
          , \
          <article-title>A wavelet-based model to recognize high-quality topics on web forum."</article-title>
          <source>in IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology. IEEE</source>
          ,
          <year>2008</year>
          , pp.
          <volume>343</volume>
          {
          <fpage>351</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>B. F.</given-names>
            <surname>Azevedo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. A.</given-names>
            <surname>Behar</surname>
          </string-name>
          , and
          <string-name>
            <given-names>E. B.</given-names>
            <surname>Reategui</surname>
          </string-name>
          , \
          <article-title>Qualitative analysis of discussion forums,"</article-title>
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>A. A.</given-names>
            <surname>Juan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Daradoumis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Faulin</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F.</given-names>
            <surname>Xhafa</surname>
          </string-name>
          , \
          <article-title>Developing an information system for monitoring student's activity in online collaborative learning."</article-title>
          <source>in Proceedings of the International Conference on Complex, Intelligent and Software intensive Systems</source>
          , CISIS,
          <year>2008</year>
          , pp.
          <volume>270</volume>
          {
          <fpage>275</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>F.</given-names>
            <surname>Dabaj</surname>
          </string-name>
          , \
          <article-title>Analysis of communication barriers to distance learning: A review study,"</article-title>
          <source>Online Journal of Communication and Media Technologies</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>J.</given-names>
            <surname>Psotka</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Massey</surname>
          </string-name>
          , and
          <string-name>
            <given-names>S.</given-names>
            <surname>Mutter</surname>
          </string-name>
          , Intelligent Tutoring Systems: Lessons Learned. L. Erlbaum Associates,
          <year>1988</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>P.</given-names>
            <surname>Brusilovsky</surname>
          </string-name>
          and
          <string-name>
            <given-names>C.</given-names>
            <surname>Peylo</surname>
          </string-name>
          , \
          <article-title>Adaptive and intelligent web-based educational systems,"</article-title>
          <source>International Journal of Arti cial Intelligence in Education</source>
          , vol.
          <volume>13</volume>
          , pp.
          <volume>159</volume>
          {
          <issue>172</issue>
          ,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>S. D</given-names>
            <surname>'Mello</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. W.</given-names>
            <surname>Picard</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Graesser</surname>
          </string-name>
          , \
          <article-title>Toward an a ect-sensitive autotutor," IEEE Intelligent Systems</article-title>
          , vol.
          <volume>22</volume>
          , no.
          <issue>4</issue>
          , pp.
          <volume>53</volume>
          {
          <issue>61</issue>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>V.</given-names>
            <surname>Yannibelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Godoy</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Amandi</surname>
          </string-name>
          , \
          <article-title>A genetic algorithm approach to recognise students' learning styles." Interactive Learning Environments</article-title>
          , vol.
          <volume>14</volume>
          , no.
          <issue>1</issue>
          , pp.
          <volume>55</volume>
          {
          <issue>78</issue>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>S. D.</given-names>
            <surname>Craig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Gholson</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D. M.</given-names>
            <surname>Driscoll</surname>
          </string-name>
          , \
          <article-title>Animated pedagogical agents in multimedia educational environments</article-title>
          ,"
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>R.</given-names>
            <surname>Moreno</surname>
          </string-name>
          , \
          <article-title>Multimedia learning with animated pedagogical agents</article-title>
          ,"
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>R.</given-names>
            <surname>Moreno</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. E.</given-names>
            <surname>Mayer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. A.</given-names>
            <surname>Spires</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J. C.</given-names>
            <surname>Lester</surname>
          </string-name>
          , \
          <article-title>The case for social agency in computer-based teaching. do students learn more deeply when they interact with animated pedagogical agents?" 2001.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>D. M.</given-names>
            <surname>Dehn</surname>
          </string-name>
          and
          <string-name>
            <surname>S. van Mulken</surname>
          </string-name>
          , \
          <article-title>The impact of animated interface agents: A review of empirical research</article-title>
          ,"
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>A.</given-names>
            <surname>Gulz</surname>
          </string-name>
          , \
          <article-title>Bene ts of virtual characters in computer-based learning environments: Claims and evidence</article-title>
          ,"
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>E. B.</given-names>
            <surname>Reategui</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. M. A.</given-names>
            <surname>Santos</surname>
          </string-name>
          , and
          <string-name>
            <surname>T. L. M. R</surname>
          </string-name>
          , \
          <article-title>Pedagogical agents and the e ciency of instructional conditions in educational applications</article-title>
          ,"
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>I. da Costa</given-names>
            <surname>Pinho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Epstein</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E. B.</given-names>
            <surname>Reategui</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Correa</surname>
          </string-name>
          , and E. Polonia, \
          <article-title>The use of text mining to build a pedagogical agent capable of mediating synchronous online discussions in the context of foreign language learning,"</article-title>
          <source>2013 IEEE Frontiers in Education Conference (FIE)</source>
          , pp.
          <volume>393</volume>
          {
          <issue>399</issue>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>D. H.</given-names>
            <surname>Jonassen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. B.</given-names>
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Yang</surname>
          </string-name>
          , and L. La ey, \
          <article-title>The collaboration principle in multimedia learning. the cambridge handbook of multimedia learning,"</article-title>
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>L. . S.</given-names>
            <surname>Vygotsky</surname>
          </string-name>
          , \
          <article-title>Mind in society: the development of higher psychological processes</article-title>
          ." Cambridge, MA,
          <year>1978</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>L. S.</given-names>
            <surname>Vygotsky</surname>
          </string-name>
          , \Thought and
          <string-name>
            <surname>language.</surname>
          </string-name>
          " Cambridge, MA,
          <year>1986</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21] ||, \
          <article-title>The collected works of l. s. vygotsky</article-title>
          .
          <source>v.1. thinking and speaking."</source>
          New York, N.Y.: Plenum Press,
          <year>1987</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>J.</given-names>
            <surname>Lantolf</surname>
          </string-name>
          and
          <string-name>
            <given-names>S.</given-names>
            <surname>Thorne</surname>
          </string-name>
          , \
          <article-title>Sociocultural theory and the genesis of second language development</article-title>
          . oxford university press,
          <source>2006." Applied Linguistics</source>
          , vol.
          <volume>28</volume>
          , no.
          <issue>3</issue>
          , p.
          <fpage>477</fpage>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Hearst</surname>
          </string-name>
          , \
          <source>The debate on automated essay grading," IEEE Intelligent Systems Vol.15 N.5</source>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>