=Paper=
{{Paper
|id=Vol-495/paper-7
|storemode=property
|title=Adapting an Educational Tool to Be Used by Non-severe Cognitive Disabled Students
|pdfUrl=https://ceur-ws.org/Vol-495/paper7.pdf
|volume=Vol-495
|dblpUrl=https://dblp.org/rec/conf/aied/Perez-MarinPM09
}}
==Adapting an Educational Tool to Be Used by Non-severe Cognitive Disabled Students==
https://ceur-ws.org/Vol-495/paper7.pdf
45
Adapting an educational tool to be used by
non-severe cognitive disabled students
Diana Pérez-Marín, Ismael Pascual-Nieto and Pilar Rodríguez
Computer Science Department, Universidad Autonoma de Madrid
Abstract. Since 2007, we have been using the Will Tools, a set of Blended
Learning applications able to automatically assess students’ free-text answers and
provide immediate personalized feedback to each student. In this paper, our
hypothesis is that these tools can be easily adapted to be used by students with
some type of non-severe cognitive impairment. In order to test this hypothesis, we
present a procedure to transform the Will Tools into the Will Tools ALADE (the
version of the Will Tools designed for students with cognitive disabilities).
Moreover, an experiment in which 13 students, some of them with Down
syndrome and others type of non-severe mental disabilities, have successfully used
the Will Tools ALADE is described.
Keywords. Blended learning, free-text assessment, Natural Language Processing,
cognitive disability, Down syndrome.
Introduction
In the last decades, there has been a revolution in the field of the New Technologies for
Information and Communication. The lower prices of the computer equipments and the
greater number of computer applications have favored a change in the use of computers.
In fact, computers are not longer regarded as complex tools just for Computer Science
people, researchers or engineers, but useful tools for a great majority of the population.
10% of the world population has some kind of disability [1]. They should not be
excluded from the new technologies. On the other hand, they can benefit, even in some
cases more than people without any disability from the use of the new technologies to
get more integrated in the society.
People having a disability are not longer regarded just as ill people to marginalize
into hospitals, but as people to be integrated in the society. Therefore, computer
applications should be designed according to the Universal Design to be used not only
by impaired people but by everyone [2,3,4].
The new technologies help people with some kind of impairment to improve their
quality of life. The role of computers in achieving this goal is helping as a facilitating
tool. Currently, there is a wide number of available computer applications for several
types of impairment. Many of them are free and they can be downloaded from Internet.
Some of them even offer their code so that programmers can offer an improved version
of them to the community.
A great percentage of these computer applications for the attention to diversity are
based on the idea of multimodal information redundancy. That is, they present the
information not only graphically, but also in other media such as sound or haptic
46
information. That way, people with visual limitations can still access the information of
a document.
Graphical interfaces and visual stimuli has also benefited other types of disabilities
such as autism. In fact, it has been claimed that autistic children interact better with
those new environments [5].
People with hearing difficulties could be considered one of the most benefited
collectives because the available information is usually visual. This is the reason why
there are less computer applications developed to improve their accessibility to
information. On the other hand, they can also take advantage of tools to communicate
via text such as electronic mails, chats and specific programs designed to call by text.
Physically disabled people can also get benefited from the new technologies to
access information designed to overcome their limitations. For instance, tetraplegic
people can work from home interacting with the computer with eye screeners or special
buttons to use virtual keyboards instead of physical ones.
In this paper, the focus is placed on people with cognitive disabilities such as
students with Down syndrome, or with some type of mental disability. In particular,
our hypothesis is that Artificial Intelligence in Education (AIED) tools such as the Will
Tools can also be adapted to be used by people with non-severe cognitive disabilities
(i.e. people that can still talk, read and write but at a lower pace or with assistance).
The Will Tools are a set of Blended Learning on-line tools [6]. Blended Learning
or Hybrid Learning tools combine traditional teaching methods with the application of
computer applications for education. That way, it is possible to take advantage of the
benefits of e-learning without bearing its disadvantages [7]. For instance, students can
review their lessons after class, and teachers can monitor their students’ progress.
These tools have been used with positive results in our home university since 2007,
both in technical and non-technical domains, and by students with and without
technical training (i.e. knowledge in how to use computers and computer applications)
[6,8,9]. Therefore, we would also like to develop a version of the tools that could be
used by people with cognitive disabilities, so that they can take advantage of the
benefits of these web-based learning systems.
We asked a group of our home university, who teach to students with Down
syndrome and other types of mental disabilities, for their opinion about which changes
should be made to adapt the Will Tools to a version that could be used by their students
too.
They provided us with a list of changes, and offered us the possibility of testing the
new version of the Will Tools, that we called Will Tools ALADE, with their students.
That way, we came up with a procedure to adapt Blended Learning tools to people with
non-severe cognitive disabilities, and we implemented it in Will Tools ALADE
(Atención a LA DivErsidad, attention to the diversity).
In December 2008, we tested the Will Tools ALADE with a group of 13 students
with Down syndrome and mental retardation, and they could successfully use them,
supporting our hypothesis.
The paper is organized as follows: Section 1 outlines the main features of the Will
Tools; Section 2 focuses on the Will Tools ALADE version; Section 3 describes the
experiment performed; and, finally Section 4 ends with the main conclusions and lines
of future work.
47
1. The Will Tools
The Will Tools are accessible on-line at http://orestes.ii.uam.es/willtools, and consist of
four subsystems: Willow (the student tool), Willed (the authoring tool), Willov (the
teacher tool), and Willoc (the administration tool).
Willow is an automatic and adaptive free-text scorer system. It is able to provide
feedback adapted to each student according to his or her answer written in natural
language (in Spanish or in English). Figures 1 and 2 show a sample of question-answer
interaction in Willow.
Figure 1. Sample snapshot of Willow asking a question to the student.
Figure 2. Sample snapshot of Willow providing feedback to the student.
48
As can be seen in both figures, the metaphor of a dialogue between the system and
the student is followed. The system is represented by an owl as its avatar, and the
student can choose its own avatar from the gallery of available avatars.
In this paper, it is not explained the process of automatically evaluating the
students’ free-text answers, or how to generate a conceptual model indicating which
concepts should be reviewed (My Model and Class Model options in the menu) as both
topics are out of the scope of this paper, and already published elsewhere [10].
However, it is important to mention that the core idea for the automatic assessment
of students’ free-text answers is the comparison between the student answer and one or
more correct answers provided by the teachers. In this way, the more similar the
student answer is to the correct answers provided by the teachers, the higher the score
that is provided to the student.
Willed is an authoring tool in which teachers can create courses to be delivered in
Willow. The courses can be created by interacting with Willed, or just uploading a
plain text template with the content of the course.
The template should contain the name of the course, a brief description, the
language of the course, and per each lesson of the course a set of questions. For each
question, it should be provided its statement, maximum score, level of difficulty and
one or more correct answers written in plain text.
Willov is a monitoring tool in which teachers can keep track of the performance of
their students with Willow. Moreover, they can look at automatically generated
graphics showing when the students have accessed Willow, how many questions they
have answered and how long they have reviewed.
Finally, Willoc is an administration tool in which it is possible to enroll or remove
students in courses and manage the students’ data.
2. The Will Tools ALADE
As can be seen from Figures 1 and 2, although the interface has been designed using
Human-Computer Interaction principles so that it is simple, user-friendly, consistent
and no student without cognitive disabilities has difficulties using it, it still has some
options that could be complex to use by students with some type of cognitive
disabilities. For instance, the menu, which has many different options.
In the meeting with the group of experts in teaching people with some type of
cognitive impairment, they advised us:
To simplify the interface so that the options of the menu that are not completely
necessary are removed.
To add more focus on the dialogue between the system and the student.
To make answering the questions easier, especially as writing text into the
computer may be really difficult for people with cognitive disabilities.
To be constant in all interfaces so that once students know how to use some
element in the interface, they do not have to keep learning new elements.
To avoid taking into account the orthography of the sentence as the goal was to
assess the content, and these students tend to have many problems writing the
words with a correct orthography.
We applied these guidelines to transform the Will Tools, especially Willow which
is the student tool, into the Will Tools ALADE in less than a month-time of
programming. The tools are also available on-line at http://orestes.ii.uam.es/alade/
49
Figure 2 and 3 show the new interface adapted to be used by any student in Willow
ALADE for a question-answer interaction. The interface is in Spanish as there are no
English courses in the system.
As can be seen the menu has been removed as the options included in it have been
moved to other systems of the Will Tools ALADE, or removed. For instance, the
model options have been removed as in the current version of the system are
considered too complex, and the modification of data, selection of topics and course
have been moved to the monitoring tool so that teachers are now in control of these
choices.
The dialogue between the system and the student occupies now nearly all the
screen, and the student is given the possibility of looking at different possible answers
to the question by clicking on the help icon (i.e. the red question mark next to the
evaluation button). Only one of the possible answers provided is correct. The student
has to choose which one is, and to type it into the text area to answer.
It is also important to highlight here that this version of the tools meet the
aforementioned Universal Design principle. As it has been designed thinking that can
be used by all students. That is, given the focus on the dialogue and by providing hints,
it is expected that can be used by students with Down syndrome, or some type of
mental disorder, but also by children or adult students without any mental disability.
Figure 3. Sample snapshot of Willow ALADE asking a question to the student (the question translated from
Spanish into English is “what should you do if you arrive late for a job interview in a company?” and the
answers shown are: “To ask for the interviewer when we arrive.” or “To wait until someone talk to us”) .
50
Figure 4. Sample snapshot of Willow ALADE providing feedback to the student (the answer translated from
Spanish into English is “To ask for the interviewer when we arrive” ).
Whenever a student click on the “Evaluate answer” button, the feedback page is
generated as in Willow. However, the difference is now that the self-assessment feature
is removed, as it is no longer necessary. Given that the student writes one of the
possible answers from the list provided, the assessment of the answer does not have
any difficulty, as it is just a comparison between two sentences that should be equal to
pass the question (taking into account that no penalization should be applied because of
bad orthography as the teachers advised us).
It can also be noticed that the feedback page has been simplified, so that the
correct answers are no longer shown separated from the qualification, but they appear
in the same note. Moreover, a graphical help has been provided so that the students
only need to look at a face: happy in case they have passed the question, sad otherwise.
Willow ALADE keeps asking questions that have been failed until the student is
able to pass them (a feature that the teachers want to keep from Willow).
3. The experiment
In December 2008, we asked the teachers, experts in cognitive disabilities, to test
Willow ALADE with a group of their students. They allowed us to go to one of their
classes, as it was the first experiment in which Willow ALADE was going to be used.
They also told us that although the idea was that students try to use Willow ALADE on
their own, they will be in the class in case that we need their help.
Moreover, the teachers helped us into choosing which topic should be the more
interesting for the course of the experiment. They told us that a course about social
skills would be helpful as these students have problems with situations such as job
interviews or how to react in new situations. It was estimated that students could pass
this course in 2 hours as it has 15 questions, and to pass the course it was necessary that
the students pass at least half of the questions (about 15 minutes per question).
51
They gave the content of the course to us as previously explained in Willed (it does
not change for the ALADE version of the Will Tools), and the data of 13 students,
some of them with Down syndrome and the rest with mental retardation. These
students were chosen by the teachers to be the first students in using Willow ALADE.
They were chosen because they did not have a severe cognitive impairment, that is,
they were able to turn on a computer, use a keyboard to type and a mouse to point into
the screen, and interact with simple options in text processors (e.g. to write a letter).
Given that Willow ALADE is an on-line application the students could use the
system from their computer lab. First of all, we gave a 5-minute talk explaining the
goal of the system, without explaining the interface in detail.
After that, the students started using the system during two hours with our
supervision and the supervision of their teachers. Most of them did not find any
difficulty into using the system, and they only asked questions about the content of the
questions. In fact, 6 out of the 13 students (46%) were able to pass the whole course in
less than the 2 hours that we have estimated, and 11 out of the 13 students (85%)
completed the course in time.
One of the students who was unable to complete the course was because he had to
leave the class; and, the other student was because he has the more severe cognitive
impairment. He needed our help to understand the mechanism of the system, and once
he was able to understand what he had to do, he needed help to type the answers as it
was too complicated for him to remember the sentence and type it again in the text area.
Figure 5 shows a snapshot of Willov ALADE interface in which the teacher can
see the progress made by the students. As can be seen, the percentage of questions that
the student has tried and the percentage of questions that s/he has managed to pass
appears next to the name of each student (removed for privacy reasons). The same
information is also graphically displayed with a bar in which the green part indicates
the questions that have been passed, the red part indicates the questions that have been
tried but not passed, and the grey part indicates the questions that have not even tried.
The last column is not used in this case because all students belong to the same
group, but if the teachers had classified them into several groups, the label of the group
to which each student belongs would appear here.
We also asked the students to write their individual opinion about the system
(especially focusing on which feature they have enjoyed the most and what they would
like to change). Some of the sentences that the students wrote are:
- “I have enjoyed this class very much, the questions were very good and I have
enjoyed the program”.
- “What I have liked the most is the topic of the questions. That way, we can get a
job. Thank you for helping us to get a job”.
- “What I like about the program is some questions, but other questions are a little
difficult and I did not know to answer them. The help icon is good because if you do
not know how to answer, it helps you”.
- “I have enjoyed this class very much as it has been fun”.
We were surprised to notice that not only the students were able to correctly use
the system and pass the questions of the course, but they seemed to like it, and some of
them even claimed that they have found it a funny way to interact with the computer.
52
Figure 5. Sample snapshot of Willov ALADE with the progress made by the students
4. Conclusions and future work
In this paper, our hypothesis that the Will Tools, a set of Blended Learning tools able to
provide automatic and adaptive feedback to each student from their answers written in
natural language, could be adapted to be used by students with some type of non-severe
cognitive impairment, has been supported.
We asked the opinion of a group of teachers expert in students with cognitive
disabilities, and they provided us with a list of changes to transform the Will Tools into
the version Will Tools ALADE. The implementation of these changes was quite easy,
and in less than a month programming time the new version was built.
The Will Tools ALADE were used by 13 students, some of them with Down
syndrome and others with mental retardation, and except for two students (one who had
to leave the class, and other who considered the task too complex), the rest of them
were able to complete the course with Willow ALADE without any difficulty.
As future work, we would like to repeat the experiment with more students and a
control group to be able to do a more complete statistical study of the data gathered.
Acknowledgment
This work has been sponsored by Spanish Ministry of Science and Technology, project
TIN2007-64718.We would also like to thank Dr. Dolores Izuzquiza and her group for
her invaluable help in the design of the Will Tools ALADE, and for letting us test the
system with her students. Last but not least, our thanks for the students who have
volunteered to use the system.
53
References
[1] R. Lorenzo & L.C.P. Bueno, Tratado sobre discapacidad, Thomson, 2007.
[2] V. Iwazaki, Fujitsu’s Activities for Universal Design, FUJITSU Sci.Tech. 41 (2005), 3-9.
[3] Vodafone, Tecnologías de la Información y Comunicaciones y Discapacidad, Fundación Vodafone
España, 2005.
[4] D. Archambault, R. Ossmann, T. Gaudy & K. Miesenberger, Computer Games and Visually Impaired
People, Upgrade (2007).
[5] L. Pérez-de-la-Maza, Programa de estructuración ambiental por ordenador para personas con trastornos
del espectro autista: PEAPO, in 'Actas del XI Congreso Nacional de Autismo', 2005.
[6] D. Pérez-Marín, I. Pascual-Nieto, E. Alfonseca, E. Anguiano, & P. Rodríguez, P. A study on the impact
of the use of an automatic and adaptive free-text assessment system during a university course, in 'Blended
Learning', Prentice Hall, Pearson Education, 2007.
[7] C. Graham, The Handbook of Blended Learning: Global Perspectives, Local Designs, Pfeiffer, 2006.
[8] I. Pascual-Nieto, D. Pérez-Marín, M. O’Donnell & P. Rodríguez, Enhancing a free-text Adaptive
Computer Assisted Assessment system with self-assessment features, in 'Eigth IEEE International
Conference on Advanced Learning Technologies', 2008.
[9] I. Pascual-Nieto, D. Pérez-Marín, P. Rodríguez & M. O’Donnell, Using Automatically Generated
Students’ Clickable Conceptual Models for e-tutoring, CEUR-WS 354 (2008), 1-8.
[10] D. Pérez-Marín, Adaptive Computer Assisted Assessment of free-text students' answers: an approach to
automatically generate students' conceptual models, PhD thesis, Escuela Politecnica Superior,
Universidad Autonoma de Madrid, 2007. Available on-line at
http://orestes.ii.uam.es/dperez/tesisDianaPerezMarin.pdf (link checked on 04/09/2009).