Adaptive Activities for Inclusive Learning using Multitouch Tabletops: An approach David Roldán1, Estefanía Martín1, Pablo A. Haya2, Manuel García-Herranz2 Universidad Rey Juan Carlos, c/Tulipán s/n, Móstoles, Madrid, 28933, Spain {david.roldan, estefania.martin}@urjc.es Universidad Autónoma de Madrid, Campus Cantoblanco, Madrid, 28049, Spain {pablo.haya, manuel.garciaherranz}@uam.es Abstract. People with cognitive disabilities have some difficulties with memory, literacy skills, attention and problem solving. Computers and specifically, adaptation mechanisms can be used to improve their learning. The adaptation allows fitting the learning process to each user. This paper presents a proposal to adapt learning activities while people are interacting using multitouch tabletops. The adaptation mechanism takes into account structural aspects, content adaptation and the interaction provided. Keywords: Multitouch tabletops, adaptation, learning, Down syndrome. 1 Motivation Some of the main categories of functional cognitive disabilities include deficits or difficulties with memory, literacy skills, attention and problem solving [1], and often visual or motor impairments. People with cognitive limitations have troubles to comprehend and to perceive the environment. They have difficulties with the transfer and consolidation of learning. For this reason, in the context of learning, new concepts to learn and problems to solve should be contextualized in daily situations to help learners to transfer knowledge to the real world. Furthermore, people with cognitive disabilities need concepts to be presented in a repeated and flexible way in order to assimilate them. Finally, due to their attention difficulties they can feel frustrated and disoriented if activities are monotonous or their level of difficulty is higher than their capabilities to solve them. As an example, unnecessary items should be deleted as they may distract the user and increase the difficulty of the task. Computers offer valuable assistance to people with special needs, including those with physical and cognitive limitations [2]. Hardware (e.g. special Braille displays for users with visual disabilities) and software applications (e.g. speech output, word prediction, speech recognition software, etc.) contribute to this goal. They offer new opportunities to learn, share information and gain independence [3]. However, designing human-computer interfaces for users with disabilities is a challenging task [4]. Brajnik [5] grouped the WCAG 1.0 guidelines by their impact on specific user groups: blind, low-vision, deaf, color blind and physical handicapped users, as well as people with cognitive disabilities. Nevertheless, these guidelines are not enough in several cases being incomplete and not covering some of the user’s needs [6]. Adaptation methods and techniques can contribute to adapt existing software to better suit the user’s needs [7]. In the context of HCI, Universal Access introduces a new perspective recognizing values and attempts to accommodate a wide range of human abilities, skills, requirements and preferences in the design of computer-based products [8]. This implies an effort to design products and applications that can adapt themselves to suit the broadest possible end-user population. In this direction, there is a need to model user features for adaptation purposes [9]. Adaptive techniques are used in different application areas. Regarding systems focused on helping users with special needs, the first one to employ adaptive techniques in order to ensure accessibility and high-quality interaction for all potential users was AVANTI [10]. This system aimed to address interaction requirements of individuals with diverse abilities, skills, needs and preferences, using Web-based multimedia applications and services. Other example is the adaptive e-learning system presented in [11]. It provides adaptation to users with problems in mental programming (i.e., showing difficulties in organizing tasks or in figuring out problem solving strategies). Finally, regarding social abilities, Sc@ut [12] is used for improving social integration of people with temporary or permanent communication difficulties, specifically of autistic children. 2 The Approach We are currently working on a project for adaptation of educational activities in multitouch tabletops to Down syndrome people. An activity is composed of a set of tokens distributed over common areas (shared by all the students) and individual areas (particular to each student). Activities can be performed either individually or collaboratively, affecting this to the token distribution. Two types of activities have been defined to this point: simple or multiple selection, and pair matching. Using FLING (Flash Library for Interpreting Natural Gestures) [13] to interpret multiple- finger input into meaningful gestures, we allow people to interact mainly through natural gestures using their fingers. In the case of simple or multiple selection activities in which users are given a set of tokens from which to chose according to a global question or statement, selection is done by touching the tokens directly with the finger. In the case of pair matching activities, in which users have to correctly associate tokens according to a global criteria, the interaction will be performed by default by drag and dropping one token over its paring one. Additionally, activities are grouped into projects: a set of activities that will be performed either sequentially or randomly. In order to provide adaptation, user’s information is stored in a user model comprising both static information, such as the background information, previous experience, motor functionality, visual impairments or index scores representing the major components of intelligence (verbal comprehension, working memory, perceptual organization and processing speed) [14], as well as dynamic information, such as physical location of the users or their evolution over the learning process (e.g. activities performed, results obtained, etc.). Dynamic information is updated according to the users’ interactions with the tabletop. Adaptation is then supported by means of rules mostly based on the recommendation mechanism of CoMoLE [15] in which an activation condition could be associated to the rules. Rule activation conditions determine for which users the rule will be applied. If an activation condition is not defined, the rule will be always triggered. There are three different adaptation rules: 1. Structural rules allow defining the different activities associated to a project and their accomplishing order (sequential or random). 2. Individual requirements are specific restrictions related to the accomplishment of a specific activity. 3. Content adaptation rules define how to adapt contents and to change the default interaction characteristics according to an activation condition (if any) and the activity type. For example, learning activities can be adapted according to the users’ previous background with multitouch tabletops, motor function ability (if they have mobility problems or not) and their results of previous activities. Additionally we may have a project (A) composed of six learning activities: a demo, to show how they can interact with the device (B), two simple selection activities (C, D), a multiple selection activity (E), a pair matching activity (F) and a review activity (G). We have defined two different structural rules based on the users’ previous experience with tabletops (see table 1). The first time students interact with the tabletop (see structural rule ) they have to see a demonstration and to perform the selection activities (C, D and E), performing afterwards the review activity (G). However, if all the students have previous experience with the device, all activities but the demonstration will be performed (see structural rule ). Table 1. Example of two structural rules. Activation condition defines when the rule is applied and the guidance column specifies a sequential accomplishing order. Activation condition Guidance Activity Subactivities  Background=no Direct A B, C, D, E, G  Background=yes Direct A C, D, E, F, G As an activity G is a review of all previous activities, we can specify that students will only perform it when their results are lower than 7 (see table 2). Table 2. Example of a specific requirement affecting to the subactivity G from the list provided by the activated structural rule. Activation condition Activity  Results < 7 G Finally, we have defined two content adaptation rules (see table 3). The rule  changes the default interaction of the pair matching activities (drag and drop) when the user has motor function problems. In this case, the user should sequentially click over the elements to be matched instead of drag and dropping. The second content adaptation rule establishes that the contents of our three types of activities must be resized when the user has visual impairments. Table 3. Examples of two content adaptation rules. Activation condition Types of activities Content Adaptation  Motor_function = low Pair_matching Interaction = click  Visual_impairment > Simple_Selection, Content_size = big 25% Multiple_Selection, Pair_matching Thus, the user model can affect to the list of activities to be performed and the order in which they are performed as well as the interaction and presentation modes. Now, we have to consider too that several users may be interacting simultaneously on the same surface. In this situation, each student may tackle the project individually, thus activities will have to use only individual areas since common areas will otherwise have to match the different activities that are concurrently running. Conversely, a collaborative project requires a group model combining the features of each student involved in the activity. In this way, activation condition will refer to group features rather than to individual ones. An aggregation policy determines how group features are obtained as the combination of the corresponding features of each student involved in the activity. We have defined different policies to be selected by the teacher: highly restrictive, less restrictive, by majority and by minority. Thus, in a highly restrictive policy, the group feature takes the value of the lower (i.e. more restrictive) individual value, as in the majority policy the group feature takes the most common value among the students. All adaptation capabilities can be defined using an authoring tool. This authoring tool is based on the CoMoLE’s web-based authoring tool. Figure 1 shows a snapshot of this tool where the teacher can define the features to be considered in the adaptation mechanism. Fig. 1. Snapshot of the authoring tool. In addition, feature values can be either numerical or stereotypes. If a characteristic is numerical, its value can range between a defined minimum and maximum thresholds. If stereotyped, it is necessary to specify two or more possible values from which the characteristic will be chosen. In order to ease the process to teachers, the most common useful features are already added to this tool. Adaptation capabilities are defined with this tool too. Once the teacher has defined all the adaptation features to be considered, students will use the D2-Player to perform the activities. The adaptation mechanism presented in this section is implemented in an external module responsible of selecting the most suitable activities and contents for the users around the tabletop. Figure 2 shows an example of a student area for a simple selection activity using the D2-Player (the D activity of the previous example). Fig. 2. Example of a simple selection activity generated by the authoring tool and reproduced in the D2-player. The final presentation is adapted according to the user profile. Aiming to help teachers, this area is automatically replicated to the number of students around the tabletop, from one to four. In this case, the teacher does not need to explicitly specify any type of adaptation. 3 Conclusion and current work This paper presents a proposal to adapt activities in inclusive learning environments using multitouch tabletops. The adaptation is based on a recommendation mechanism previously applied to mobile learning systems. When working with users with cognitive disabilities, it is really important to adapt activities to their main features both individually and collaboratively. This project emerged from the collaboration between the Fundación Síndrome de Down de Madrid and the researchers of the ASIES project. Teachers from this foundation are really interested in the development of educational tools to design adaptive learning activities for multitouch tabletops as this type of devices are physical spaces strongly promoting collaboration. Acknowledgments. This work has been funded by the Spanish Government (ASIES Project - Ministerio de Ciencia e Innovación de España, TIN2010-17344). The D2- Player has been funded by Fundación Sindrome de Down Madrid. References 1. Solheim, I.: Personalisation of user interface for the cognitively disabled: how to test cognitive functions and ICT skills. In: First T4P Conference Technology for Participation and Accessible eGoverment Services, pp. 98--101 (2007) 2. Friedman, M.G., Nelson B.D.: Web Accessibility Design Recommendations for People with Cognitive Disabilities. Technology and Disability, vol. 19(4), pp. 205--212. IOS Press (2007) 3. Special Technological Possibilities for Students with Special Needs. Learning & Leading with Technology, vol. 29, number 3, pp. 32--39 (2001) 4. Biswas, P., Robinson, P.: Modelling User Interfaces for Special Needs. Accessible Design in the Digital World (ADDW) 2008, York, UK (2008) Available at: http://www.cl.cam.ac.uk/~pb400/Papers/pbiswas_ADDW08.pdf 5. Brajnik, G. Web Accessibility Testing: When the Method is the Culprit. In: K. Miesenberger et al. (Eds.). Computers Helping People with Special Needs, ICCHP 2006. LNCS 4061, pp. 156--163. Springer (2006) 6. Vigo, M., Kobsa, A., Arrue, M., Abascal, J.: User-tailored web accessibility evaluations. In: Proceedings of the 18th Conference on Hypertext and Hypermedia, pp. 95—104. Manchester, UK (2007) 7. Brusilovsky, P.: Adaptive hypermedia. In: Kobsa, A. (eds.) User Modeling and User Adapted Interaction, Ten Year Anniversary Issue, vol. 11 (1/2), pp. 87--110 (2001) 8. Stephanidis, C.: Adaptive Techniques for Universal Access. User Modeling and User- Adapted Interaction 11, pp. 159—179. Kluwer Academic Publishers (2011) 9. Brusilovsky, P., Millán, E.: User Models for Adaptive Hypermedia and Adaptive Educational Systems. The Adaptive Web: Methods and Strategies of Personalization, Lecture Notes in Computer Science vol. 4321, pp. 3--53. Springer-Verlag. (2007) 10.Fink, J., Kobsa, A., Nill, A.: Adaptable and adaptive information provision for all users, including disabled and elderly people. New Review of Hypermedia and Multimedia, vol. 4 (1), pp. 163--188 (1988) 11.Kurhila, J., Varjola, H.: Using Adaptive Hypermedia to Evaluate Basic Arithmetic Skills in Special Education. In: Proceedings of the 8th International Conference on Computers Helping People with Special Needs table of contents, LNCS, vol. 2398, pp. 98--106. Springer-Verlag London, UK (2002) 12.Gea-Megías, M., Medina-Medina, N., Rodríguez-Almendros, M.L., Rodríguez-Fórtiz, M.J.: Sc@ut: Platform for Communication in Ubiquitous and Adaptive Environments Applied for Children with Autism. User-Centered Interaction Paradigms for Universal Access in the Information Society, pp. 50--67, Springer Berlin / Heidelberg (2004) 13.Llinás, P., García-Herranz, M., Haya, P.A., Montoro, G.: Unifying Multi-Modal Events for Interpreting User Intentions through Natural Gestures. In: INTERACT 2011, 13th IFIP TC13 Conference on Human-Computer Interaction 2011. In press. (2011) 14.Wechsler, D.: The Wechsler Adult Intelligence Scale. San Antonio, TX: Psychological Corporation (1997) 15.Martín, E., Carro, R.M.: Supporting the Development of Mobile Adaptive Learning Environments: A case study. IEEE Transactions on Learning Technologies, vol. 2, no.1, pp. 23--36. (2009)