People with cognitive disabilities have some difficulties with memory, literacy skills, attention and problem solving. Computers and specifically, adaptation mechanisms can be used to improve their learning. The adaptation allows fitting the learning process to each user. This paper presents a proposal to adapt learning activities while people are interacting using multitouch tabletops. The adaptation mechanism takes into account structural aspects, content adaptation and the interaction provided.
Some of the main categories of functional cognitive disabilities include deficits or
difficulties with memory, literacy skills, attention and problem solving [
Computers offer valuable assistance to people with special needs, including those
with physical and cognitive limitations [
Adaptation methods and techniques can contribute to adapt existing software to
better suit the user’s needs [
We are currently working on a project for adaptation of educational activities in
multitouch tabletops to Down syndrome people. An activity is composed of a set of
tokens distributed over common areas (shared by all the students) and individual areas
(particular to each student). Activities can be performed either individually or
collaboratively, affecting this to the token distribution. Two types of activities have
been defined to this point: simple or multiple selection, and pair matching. Using
FLING (Flash Library for Interpreting Natural Gestures) [
In order to provide adaptation, user’s information is stored in a user model
comprising both static information, such as the background information, previous
experience, motor functionality, visual impairments or index scores representing the
major components of intelligence (verbal comprehension, working memory,
perceptual organization and processing speed) [
Adaptation is then supported by means of rules mostly based on the
recommendation mechanism of CoMoLE [
For example, learning activities can be adapted according to the users’ previous background with multitouch tabletops, motor function ability (if they have mobility problems or not) and their results of previous activities. Additionally we may have a project (A) composed of six learning activities: a demo, to show how they can interact with the device (B), two simple selection activities (C, D), a multiple selection activity (E), a pair matching activity (F) and a review activity (G).
We have defined two different structural rules based on the users’ previous experience with tabletops (see table 1). The first time students interact with the tabletop (see structural rule ) they have to see a demonstration and to perform the selection activities (C, D and E), performing afterwards the review activity (G). However, if all the students have previous experience with the device, all activities but the demonstration will be performed (see structural rule ).
As an activity G is a review of all previous activities, we can specify that students will only perform it when their results are lower than 7 (see table 2).
Finally, we have defined two content adaptation rules (see table 3). The rule changes the default interaction of the pair matching activities (drag and drop) when the user has motor function problems. In this case, the user should sequentially click over the elements to be matched instead of drag and dropping. The second content adaptation rule establishes that the contents of our three types of activities must be resized when the user has visual impairments. Thus, the user model can affect to the list of activities to be performed and the order in which they are performed as well as the interaction and presentation modes. Now, we have to consider too that several users may be interacting simultaneously on the same surface. In this situation, each student may tackle the project individually, thus activities will have to use only individual areas since common areas will otherwise have to match the different activities that are concurrently running. Conversely, a collaborative project requires a group model combining the features of each student involved in the activity. In this way, activation condition will refer to group features rather than to individual ones. An aggregation policy determines how group features are obtained as the combination of the corresponding features of each student involved in the activity. We have defined different policies to be selected by the teacher: highly restrictive, less restrictive, by majority and by minority. Thus, in a highly restrictive policy, the group feature takes the value of the lower (i.e. more restrictive) individual value, as in the majority policy the group feature takes the most common value among the students.
All adaptation capabilities can be defined using an authoring tool. This authoring tool is based on the CoMoLE’s web-based authoring tool. Figure 1 shows a snapshot of this tool where the teacher can define the features to be considered in the adaptation mechanism.
In addition, feature values can be either numerical or stereotypes. If a characteristic is numerical, its value can range between a defined minimum and maximum thresholds. If stereotyped, it is necessary to specify two or more possible values from which the characteristic will be chosen. In order to ease the process to teachers, the most common useful features are already added to this tool. Adaptation capabilities are defined with this tool too.
Once the teacher has defined all the adaptation features to be considered, students will use the D2-Player to perform the activities. The adaptation mechanism presented in this section is implemented in an external module responsible of selecting the most suitable activities and contents for the users around the tabletop. Figure 2 shows an example of a student area for a simple selection activity using the D2-Player (the D activity of the previous example).
Aiming to help teachers, this area is automatically replicated to the number of students around the tabletop, from one to four. In this case, the teacher does not need to explicitly specify any type of adaptation. 3
This paper presents a proposal to adapt activities in inclusive learning environments using multitouch tabletops. The adaptation is based on a recommendation mechanism previously applied to mobile learning systems. When working with users with cognitive disabilities, it is really important to adapt activities to their main features both individually and collaboratively.
This project emerged from the collaboration between the Fundación Síndrome de Down de Madrid and the researchers of the ASIES project. Teachers from this foundation are really interested in the development of educational tools to design adaptive learning activities for multitouch tabletops as this type of devices are physical spaces strongly promoting collaboration.
Acknowledgments. This work has been funded by the Spanish Government (ASIES Project - Ministerio de Ciencia e Innovación de España, TIN2010-17344). The D2Player has been funded by Fundación Sindrome de Down Madrid.