Beyond Math Manipulatives: Smart Tangible Objects for Algebra Learning

Abstract

This workshop position paper presents ongoing research on using smart tangible objects for algebra learning. While mathematical manipulatives have played an important role in children's mathematics development for decades, employing tangible objects in the classroom has been rarely explored yet. In our work, we investigate the potentials of using smart objects for algebra learning. Our smart tiles are based on traditional algebra tiles, passive mathematical manipulatives used in many schools in Northern America, and we currently extend these by 1.) multimodal input and output capabilities, 2.) dynamic constraints and 3.) adaptivity and feedback. In this paper, we give an overview on the overall system concept, the interaction with the tangible objects and their current design, as well as on the potentials of actuated smart objects for future interaction.

-

Copyright © 2018 for this paper held by its author(s). Copying permitted for private and academic purposes.

Author Keywords

Tangible user interface; smart objects; tabletop interaction; embodied interaction; multimodal feedback; collaborative learning; adaptive system.

Introduction

In math education, simple passive manipulatives provide valuable “hands-on” approaches to teach students abstract concepts, especially when the students start to learn a novel unit of math, e.g., arithmetic, geometry, or algebra. These approaches are in-line with models from didactics like Bruner’s concrete-representational-abstract approach [ 3 ] or the constructivistic objects-to-think-with approach [ 17 ] that suggest to use physical objects for abstract concepts, especially for beginners. While a considerable body of research on using tangible user interfaces (TUIs) for learning has been conducted, more research efforts are needed to address the question how tangible user interfaces can be made smarter in order to facilitate a better learning environment and better support for learners. In our research, we investigate the potentials of smart objects for learning. The objects are based on traditional algebra tiles, which are passive mathematical manipulatives (see Fig. 1) as used in many schools in Northern America to support algebra learning. We are extending these tiles to smart “tiles” by 1.) multimodal input and output capabilities, e.g., light and display, 2.) dynamic constraints, e.g., electromagnets attracting or repelling objects, and 3.) adaptivity and feedback, e.g., user support and hints.

In educational research tactile models are common, as for example the ones by Bruner [ 3 ] or Kieran [ 10 ]. They showed that by using physical objects it is possible to teach already small children mathematical, in particular algebraic concepts. Common digital learning platforms, such as Dragonbox1 lack the haptic and tactile components and therefore the benefits that come with tangibility. By transforming the algebra tiles to smart objects we want to combine the richer feedback that can be provided by digital platforms with the benefits of tactile interaction, where the smart tiles themselves create dynamic constraints and allow for multimodal input and output, also in combination with a touch screen.

Related Work

Tangible user interfaces emerged in the 1990s, as Ishii and Ullmer [ 9 ] describe their vision for “tangible bits”. The gap between the physical world and the digital world should be closed by allowing users to directly manipulate these bits, which can be everyday physical objects. Early examples are the metaDesk [ 20 ], the transBOARD [ 8 ] and the Urban Planning Workbench [ 21 ]. Since then, a popular application domain for TUIs has been learning. Examples for using tangibles for math learning have been provided by Falca¯ o et al. [ 5 ], Girouard et al. [ 6 ], Manches and O’Malley [ 12 ], and Marichal et al. [ 13 ], amongst many others. Others like Rick [ 19 ] incorporate touch to be able to directly manipulate math objects presented on a screen. Research about how tangibles can support learning or how learning theories can inform tangible development is for example presented by the “Tangible Interaction Framework” by Hornecker and Buur [ 7 ] or the “Tangible Learning Design Framework” by Antle and Wise [ 1 ]. They propose design principles for tangibles and a taxonomy about the relationship between TUIs, interactions and learning. Furthermore Marshall [ 14 ] and Marshall, Price and Rogers [ 15 ] critically discuss how tangibles can support learning.

Examples for technological approaches for smart objects are Sifteos (previously Siftables) [ 16 ], small objects which have all technology needed inside that react to each other and encourage interaction with the objects themselves. Sifteo cubes were launched as product in 2011 but are not available anymore. A newer approach are the Actibles [ 4 ], which are tangibles with a smartwatch core and light feedback. They allow a variety of interactions, including shaking, tilting, stacking and neighbouring.

Multimodal Algebra Learning with Tiles

Algebra tiles as shown in Fig. 1 consist of three types of tiles: single units used as “ones”, x-tiles and x2-tiles. Each tile has a positive and a negative side, whereby the negative side is colored red and the positive value is represented by a unique color of that object. The tiles are typically placed on a 2 2 area (compare Fig. 2), where the two squares on the left represent the left side of an equation and the two squares on the right represent the right side of the equation. The top square on both sides is the “addition zone”, i.e., all tiles there are connected by addition, while the lower areas are the “subtraction zones”, i.e., all tiles there are subtracted from the top ones. An equation, for example 3 + ( x) 5 = x, is put up in tiles (see top image in Fig. 3). There would be three positive ones and one negative x-tile in the addition section on the left side, and five positive ones in the subtraction zone. On the right side there is just one positive x-tile in the addition zone. The model comes with a set of possible actions that correspond to typical algebraic manipulations, when dealing with linear equations (addition, subtraction, multiplication and division). Generally, the goal is to apply a sequence of legal actions in order to transform the equation to another form or to isolate the x-tile on one side and the one-tiles on the other side. With the traditional tiles, a student can transform the equation, but does not get any feedback about the correctness of the actions. A teacher is still necessary to verify them. On the contrary, in our approach, the algebra tiles themselves are aware of the equation they are part of and give feedback or hints about the steps. They can verify the result of a student’s actions or support grouping of tiles with visual feedback (see subsection Multimodal Input and Output for more details). In combination with a touchscreen and the capacity to track the tiles, multimodal algebra learning is possible, as the benefits of the tangible tiles are combined with the rich feedback such a system can provide.

Interaction with Smart Tangible Objects

In our current setup, the smart tangible objects are placed on an interactive tabletop, where they are identified, located and tracked with regard to orientation. This way, they are integrated into a system that also uses the tabletop surface for input and output capabilities (e.g., visual feedback and multi-touch interaction). While we used passive tiles in early versions of our systems (see Fig. 2, left side), the smart tiles (see Fig. 4,5,6,7) are designed to provide rich interaction capabilities themselves as this enhances the learning environment and supports learners. Furthermore, this approach would also allow for a setting in which the smart tiles are used fully functional on their own without a multi-touch table. The intelligence situated in our tangible objects and their surrounding system currently improves the interaction by three approaches: multimodal input and output capabilities, dynamic constraints, and adaptivity and feedback.

Multimodal Input and Output

Multimodal input and output facilities of learning systems can enhance learning experiences, as more senses are involved in the learning process. Our tangibles follow this approach by allowing direct haptic interaction with the tiles, which can be moved around and placed in different areas of the system for input, i.e., to perform algebraic operations. Moreover, the objects directly give visual feedback, for which we developed a model with two kinds of visual feedback with central display and edge light feedback (see also [ 4, 2 ] for a similar approach with low-resolution edge displays). With this approach, the smart tiles can display their current state in the center, e.g., the current value a tile presents (see Fig. 6 and Fig. 7). At the same time they communicate feedback on the performed operations via edge light animations, e.g., if the moves were correct or about current relations to surrounding tiles such as grouping of tiles where tiles are combined to one unit. Other output modalities such as sound and vibration feedback are currently tested.

Dynamic Constraints

Our smart objects are designed to provide dynamic constraints that guide the interaction. In our system, the dynamic constraints can direct the grouping of tiles, which is an essential action for performing operations and thus for transforming and solving equations, they can also prevent wrong combination of tiles (e.g., placing a tile of unit one along the long side of an x-tile). The dynamic constraints change according to the current value of each tile and the possible combinations. We realized the dynamic constraints by adding neodymium magnets and electromagnets to the sides of the objects. When the electromagnets are switched off, the neodymium magnets repel two objects. In case the electromagnets are switched on, realized by closed circuits when two fitting objects have contact, the objects attract each other (see next section for further information). With this approach, we can also stack objects. Furthermore, this physical grouping also gives a great physical representation of grouped units and enhances the haptic interaction.

Adaptivity and Feedback

Among the advantages of our system is that it can be adapted and, to some degree, can automatically adapt to learners with different levels and needs with regard to feedback and hints provided. Partly, this is directly provided by the smart tiles themselves via display and light feedback as well as magnetic hints, partly this is currently communicated by the surrounding system, in our case the interactive tabletop system. In order to make the system smart and allow for good feedback and hints, we have integrated and further employ a number of approaches. One of these is using Wolfram Alpha as computation knowledge engine2 that computes algebraic transformations and allows for feedback if an operation is a useful move towards the result for example. Moreover, machine learning approaches can support

2https://www.wolfram.com/engine/

the identification of typical errors and the integration of useful feedback, which will also improve the feedback the tiles communicate directly. A central challenge lies in finding a good balance of learner level and adequate feedback and hints.

Current Design of the Smart Tangible Objects

For our tangible learning system with algebra tiles we started with passive tiles and optical tracking. Using the reacTIVision3 framework we had an early setup with tracking from below to avoid occlusion problems. Currently we are working on capacitive tracking in combination with motion sensors to enable working on touch screens like the Microsoft Surface Hub4. The underlying software on the touchscreen is programmed in Unity5 and for supporting the equation solving we are using the Wolfram Alpha API. the tiles have contact and close a circuit through conductive contacts. These contacts only close a circuit when the pairing is allowed, which can be changed dynamically. If combined, the two objects stick together and represent a unit. Through the neodymium magnets stacking tiles is generally also possible.

A concrete example for the use case of the magnets attracting would be two tiles of opposite value, e.g., +1 and 1, which can be paired as an “zero pair” and be removed from the working area, as they cancel out. Additionally we have light feedback to support the same pairing process and grouping. With the light feedback one smart tile can support more advanced learners, which already know that +1 on the left side on an equation can be moved to right side and then results in an 1. Smart tiles can automatically change color to show this sign change.

Envisioning Future Smart Tangible Objects

The current setup and underlying model have some limitations that could be addressed by actuated tangibles. During solving, especially children tend to compare the resulting x-tile in size with the one-tiles. For the future, smart x-tiles that could change their shape regarding size would be beneficial to this step, after the equation is solved. Additionally it may occur the case that the user is able to see the solution even though there is still more than one x-tile on the area. Being able to change the size for all xtiles synchronously would enhance the effect that all x are the same. Thus, exploring shape change (c.f. the design space of shape-changing interfaces by [ 18 ]) for tangible presentations of variables in algebraic expressions would be valuable. Another case is that especially after a division it happens that multiple objects need to be removed at once, which tends to be bothersome for the user. Here actuated smart tiles would do the trick and after performing a division, they could automatically remove themselves from the working area. Vice versa, for multiplication they would automatically enter the working area. Also in case of providing help they could rearrange to give another view on the equation. Approaches like used in small swarm robots as the Zooid Swarm Robots [ 11 ] could be applied to make the tiles and the interactions smarter.

Conclusion

In this paper, we presented our approach for smart tangible objects to support algebra learning. Starting from traditional passive math manipulatives, we developed a concept to make these learning objects more intelligent in order to provide better learning environments that allow for rich and multimodal interactions, dynamic constraints, as well as adaptivity and feedback. We presented our current prototype, including the overall math system and the design of the smart objects. Furthermore, we discussed strategies to design even smarter tangible objects, which would address some of the current limitations by realizing actuation, both with regard to shape-change as well as to self-moving tiles. Overall, while our ongoing work focuses on providing a contribution to designing smart tangible objects for learning scenarios, our approaches to make the interaction and the objects smarter could also be valuable for other application contexts.

Tangible Objects via Low-Resolution Edge Displays.

ACM Press, 481–488. DOI: http://dx.doi.org/10.1145/3024969.3025078 on Human factors in computing systems. ACM, 234–241.

References

1. Alissa N. Antle and Alyssa F. Wise . 2013 . Getting Down to Details: Using Theories of Cognition and Learning to Inform Tangible User Interface Design . Interacting with Computers 25 , 1 (Jan. 2013 ), 1 - 20 . DOI: http://dx.doi.org/10.1093/iwc/iws007

2. Ahmed Sabbir Arif , Brien East, Sean DeLong , Roozbeh Manshaei, Apurva Gupta, Manasvi Lalwani, and Ali Mazalek . 2017 . Extending the Design Space of

3. Jerome Seymour Bruner . 1966 . Toward a theory of instruction . Vol. 59 . Harvard University Press.

4. Brien East , Sean DeLong , Roozbeh Manshaei, Ahmed Arif, and Ali Mazalek . 2016 . Actibles: Open Source Active Tangibles . In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces (ISS '16) . ACM, New York, NY, USA, 469 - 472 . DOI:http://dx.doi.org/10.1145/2992154.2996874

5. Taciana Pontual Falcao , Luciano Meira, and Alex Sandro Gomes. 2007 . Designing tangible interfaces for mathematics learning in elementary school . In Proceedings of III Latin American conference on human-computer interaction.

6. Audrey Girouard , Erin Treacy Solovey, Leanne M. Hirshfield , Stacey Ecott, Orit Shaer, and Robert J. K. Jacob . 2007 . Smart Blocks: A Tangible Mathematical Manipulative . In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07) . ACM, New York, NY, USA, 183 - 186 . DOI: http://dx.doi.org/10.1145/1226969.1227007

7. Eva Hornecker and Jacob Buur . 2006 . Getting a Grip on Tangible Interaction: A Frameowrk on Physical Space and Social Interaction . In CHI 2006 . ACM Press, Montréal, Québec, Canada.

8. Hiroshi Ishii . 2008 . The Tangible User Interface and Its Evolution . Commun. ACM 51 , 6 ( June 2008 ), 32 - 36 . DOI:http://dx.doi.org/10.1145/1349026.1349034

9. Hiroshi Ishii and Brygg Ullmer . 1997 . Tangible bits: towards seamless interfaces between people, bits and atoms . In Proceedings of the ACM SIGCHI Conference

10. C Kieran . 2007 . Learning and teaching algebra at the middle school through college levels. I: Lester , FK Jr. Second Handbook of Research on Mathematics Teaching and Learning ( 2007 ).

11. Mathieu Le Goc , Lawrence H Kim, Ali Parsaei, Jean-Daniel Fekete , Pierre Dragicevic, and Sean Follmer . 2016 . Zooids: Building blocks for swarm user interfaces . In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM , 97 - 109 .

12. Andrew Manches and Claire O'Malley . 2012 . Tangibles for Learning: A Representational Analysis of Physical Manipulation . Personal Ubiquitous Comput . 16 , 4 (April 2012 ), 405 - 419 . DOI: http://dx.doi.org/10.1007/s00779-011-0406-0

13. Sebastián Marichal , Anadrea Rosales, Fernando Gonzalez Perilli, Ana Cristina Pires, Ewelina Bakala, Gustavo Sansone, and Josep Blat . 2017 . CETA: Designing Mixed-reality Tangible Interaction to Enhance Mathematical Learning . In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17) . ACM, New York, NY, USA, Article 29 , 13 pages. DOI: http://dx.doi.org/10.1145/3098279.3098536

14. Paul Marshall . 2007 . Do tangible interfaces enhave learning . In TEI '07: Proceedings of the 1st international conference on Tangible and embedded interaction , Brygg Ulmer and Association for Computing Machinery (Eds.) . Curran, Red Hook, NY. OCLC: 254584537 .

15. Paul Marshall, Sara Price, and Yvonne Rogers . 2004 . Conceptualising tangibles to support learning . In Proceedings of the 2003 Conference on Interaction Design and Children: 2003 , Preston, England, July 01-03 , 2003 , Conference on Interaction Design and Children, Stuart MacFarlane , ACM Digital Library, and Association for Computing Machinery (Eds.). Association for Computing Machinery , New York, N.Y. OCLC: 612478047 .

16. David Merrill, Jeevan Kalanithi , and Pattie Maes . 2007 . Siftables: Towards Sensor Network User Interfaces . In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07) . ACM, New York, NY, USA, 75 - 78 . DOI: http://dx.doi.org/10.1145/1226969.1226984

17. Seymour Papert and Idit Harel . 1991 . Constructionism. Norwood. ( 1991 ).

18. Majken K. Rasmussen , Esben W. Pedersen, Marianne G. Petersen, and Kasper Hornbaek . 2012 . Shape-changing Interfaces: A Review of the Design Space and Open Research Questions . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12) . ACM, New York, NY, USA, 735 - 744 . DOI: http://dx.doi.org/10.1145/2207676.2207781

19. Jochen Rick . 2010 . Quadratic: Manipulating algebraic expressions on an interactive tabletop . In Proceedings of the 9th International Conference on Interaction Design and Children . ACM, 304 - 307 .

20. Brygg Ullmer and Hiroshi Ishii . 1997 . The metaDESK: models and prototypes for tangible user interfaces . In Proceedings of the 10th annual ACM symposium on User interface software and technology. ACM , 223 - 232 .

21. John Underkoffler and Hiroshi Ishii . 1999 . Urp: a luminous-tangible workbench for urban planning and design . In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM , 386 - 393 .