The creation of tangible interfaces can have multiple means: using electronics, computer vision or with electromagnetic sensing. In this workshop we propose to create tangible interfaces and interactive experiences using simple color detection that can be easily integrated onto objects or for the creation of dedicated interactors. The goal of this workshop is to permit the emergence of new tangible interfaces, with a highlight on projection-based augmented reality. A first group created an emergency situation management mock-up, and the second a pedagogical tool to teach the impact of the Moon and Sun on tangible interaction, augmented reality, interactive projection, paper interfaces, education, maker, EmerEtis'22, Fifth European Tangible Interaction Studio - Nov. 7-10 2022, ENAC Toulouse, France CEUR Workshop Proceedings
Tangible interfaces can provide user interfaces through object manipulations. However, object
identification, tracking and instrumentation usually requires long design process, complex or
costly dedicated tracking mechanisms using cameras[
The origin and motivation of this workshop is twofold :
• We propose a simple low-cost vision-based tracking system using coloured dots. This
tracking system was initially inspired by DynamicLand [
In this paper and the workshop we present a few diferent examples of applications concepts and realisations using this kind of markers. The main context is projection-based AR. The system uses a projector and a camera calibrated together, which are located above a table creating an AR interactive surface on the table. https://github.com/poqudrof (J. Laviole)
We created diferent interactors, menus, physical items, selection tokens in various experiments and a commercial application. In this workshop we want to expose the tools to the community to enlarge the design space of TUI.
The workshop is proposed by CATIE, which is a technology resource center. The first goal
of CATIE is to enable companies to understand new technologies: between communications,
marketing and research article, little can be applied in real-world scenarios. We also focus on
research that has a short-term (2-5 years) potential, to provide new technologies for
companies. In this context, Augmented Reality and tangible interfaces already play a role in current
technologies as well as future uses. The Human Factor (HF) team works on vulgarization of
behaviour and cognitive user studies with a dedicated web platform (Peac²h[
The main contact person is Jeremy Laviole. He has a PhD in Augmented Reality, and created a toolkit for see-through AR and projection-based AR. This toolkit has been developed for over 10 years with a few years gap here and then. He created an AR company to push projection-based interfaces, and is now part of CATIE as a research engineer.
Quentin Gobert is a student at the Optic Institute of Aquitaine, and will join CATIE as engineer. His education is physic-based and he specialized in the informatics branch of optic (3D rendering, AR and VR technology). During his internship, he developed a demonstrator based on PapARt, a projection-based AR technology developed by Jeremy Laviole, and whose goal is to be shown in a showroom at the Estia Engineering school in Bidart, France.
The workshop lasted for a little less than the 4 hours planned at first. Here is the plan we followed : • 15mins: Workshop introduction, presentation of the subject, materials and previous results.1 • 15mins: Presentation of the participants, creation of groups and subject picking or creation. • 1hours: Group work start. • 30mins: Cofee break. • 30mins: Group work end. • 30mins: Test implementation of the group’s work • 30mins: Group presentation with demonstration.
During the workshop, each group was invited to discuss around demonstrations and ask
questions for their workshop activities and around the project. Here what was demonstrated:
1Slides are available at: https://doc.natar.fr/doc/presentations
• Circular detection, color reading inside the circle and discrimination of 5 colors in CIELAB
color space. HSV, RGB, XYZ are also available.
• Circular detection at scale: 8mm detector will get 8mm circular object, and 25mm will
get only bigger not seeing the smaller ones. It is impossible to mix sizes in groups and
lines for now.
• Tracking of dots, groups, and lines over time and filtering with 1€ filter[
The group work is focused on ideation and creation of interfaces. In order to kickstart the ideas on TUIs using this system we introduce here some of our explorations over the past years. Groups have then the possibility to build ideas on top of the concepts presented here of propose completely diferent ones.
The strengths and weaknesses of the system will be tested during the workshop, in order to check the real-world possibilities and constraints. Here is an example group work program, for an accessibility focused group: • Discussions on the subject that gathered the group together : tangible UIs that are recognizable by touch. • Proposition to recreate an existing work of physical UI shapes recognizable by touch. • Creation of low fidelity mockup on paper and cardboard. • Test of the detections and usage constraints using the provided detection system. • Preparation of the group presentation with the structure: Problem tackled, existing project, current exploration, strengths and limitations, conclusion.
We present here multiple projects or pieces of design that uses tangible interfaces which are
permitted by these small markers. Stickers or colored dots can be used to create UIs, moreover
it is notable that a colored sticker can be applied on a nail to achieve finger tracking on the table.
Likewise, coloured nail polish also works. Using the same idea, markers can be put on objects
or mock-ups augmenting the visualization of the model [
Keywords: Communication project, electronics, technology demonstration
The project aims to a presentation of electronic cards from 6TRON project[
Keywords: Scientific popularization, Neural Networks, Hackathon
Artificial Neural Network (ANN) and Deep Neural Network (DNN) are popular machine
learning tools. They can be quite obscure and hard to comprehend because of their large
structure that is partly bio inspired. This example is taken from a workshop on AI[
The goal of this workshop is to open new possibilities for researchers and students. The
open-source project[
The sticker tracking opens easy tangible design compared to electronic instrumentation, or using depth cameras. In our opinion, it can be a first step before using more complex tracking techniques, using commercial software for feature-based tracking, model-based tracking or retro-reflective infrared markers. The design of projection-based interface, and experiences is still a recent research field, and we hope it could lead to new explorations.
The materials available were: • Play dough with 8 diferent colors • Paper, cardboard, scissors, post-it notes.
• Colored stickers, felt-tip pens, pencils.
The organizers did bring a projector-camera system to prototype tracking and display for the groups if necessary. The possibility to code and create applications by workshop members was taken out for the sake of time and ease of use.
The workshop gathered 11 people who gathered in two groups of 5 and 5 persons each. The ifrst group worked on emergency situation management control system. The second group created an education tool to experiment with the Earth, Moon, Sun trio.
The first group created a container to store five tangible objects. Each object had its own use, unique shape and color. Two objects were used to position elements on a map : a bus and a helicopter to pinpoint the location where they should go to evacuate the population. The second type of objects where placed on a container and their order in the container: top, to bottom indicated which layer to overlay on the map and their visual intensity. The goal there was to visualize and understand how water will flow to predict which roads will be blocked in a near future and plan an evacuation to limit the number of person who could be stuck or in danger by the flood.
The color tracking implementation using circle detection proved dificult for light colors, which was a known limitations. The tracking is shown in figure 3. The system managed to distinguish the colors after a calibration step. However the shapes where more ellipses than circles, so for a real-world use we recommend to consider a large object as a set of multiple small colored objects. Using this, it is possible to retrieve the orientation of these objects.
The second group created a pedagogical experience to play with the Earth, Moon and Sun and see some of the impacts of their locations on Earth. The Sun is fixed in the center of the manipulation area, as seen in Figure 3. When the Earth rotates on itself the day/night cycle is updated on the top right (a). When the Moon rotates around the Earth the tides changes in the middle right part (b). When the Earth moves around the Earth, the seasons changes on the bottom right part (c).
The three celestial bodies were tracked using color tracking, each had its own color and size adjusted to be tracked on paper. In order for it work properly the Earth was simplified from blue and green to a green sphere. The Moon was changed from grey to blue. The experience could be done using this implementation for (b) and (c). However the rotation of the Earth on itself could not be achieved like this. Using our system there are two simple ways to create is: either using small IOT devices that uses an accelerometer, or using a set of dots that enable rotation tracking.