=Paper=
{{Paper
|id=Vol-2801/paper1
|storemode=property
|title=Time Navigation in a Virtual Environment using Tangible Interactions: application to the domain of History of Science and Technology
|pdfUrl=https://ceur-ws.org/Vol-2801/paper1.pdf
|volume=Vol-2801
|authors=Pierre Mahieux,Sébastien Kubicki,Sylvain Laubé,Ronan Querrec
}}
==Time Navigation in a Virtual Environment using Tangible Interactions: application to the domain of History of Science and Technology==
https://ceur-ws.org/Vol-2801/paper1.pdf
Time Navigation in a Virtual Environment using
Tangible Interactions: application to the domain of
History of Science and Technology
Pierre Mahieuxa , Sébastien Kubickia , Sylvain Laubéb and Ronan Querreca
a
Lab-STICC, CNRS, ENIB, Technopôle Brest-Iroise, 29238 Brest Cedex 3, France
b
Centre François Viète, Université de Bretagne Occidentale, 20 rue Duquesne, 29200 Brest, France
Abstract
The History of Science and Technology domain studies and describes the evolutions of human technical
activities. Historians use new technologies such as Virtual Reality to reconstruct and expose the results
of their research. In order to observe the evolutions of the studied systems inside a Virtual Environment,
the user must be able to navigate through time. We propose a model of time based on two scales and to
use tangible interactions to navigate on these two timescales.
Keywords
Models, Interaction paradigm, Tangible User Interface, Virtual Reality, istory of Science and Technology,
Cultural Heritage
1. Introduction
In the History of Science and Technology (HST) domain, Virtual Reality (VR) has notably been
used to model and represent in 3D technical and industrial systems within which we find human
activities [1]. Let’s consider for example the raising and lowering of a drawbridge inside a
harbor, the manufacturing of ship parts inside an arsenal or the routing of resources necessary
for the operating of an isolated mine. In the domain of HST, these 3D models are mainly used
for cultural mediation and the archiving of these systems. However, little research using these
tools is interested in the representation of human activities around these technical devices.
To describe the artifacts1 and the activities, historians use ontologies such as CIDOC-CRM 2
and Dolce [2]. These ontologies allow to represent and describe the flow of the activities as
procedures3 (e.g., the opening of a bridge) and period4 (e.g., the 16𝑡ℎ century).
On the one hand VR, by immersing a user in a Virtual Environment (VE), offers the possibility
to visualise and to take part in the realisation of an activity. On the other hand, to allow
Proceedings of ETIS 2020, November 16–20, 2020, Siena, Italy
Envelope-Open mahieux@enib.fr (P. Mahieux); kubicki@enib.fr (S. Kubicki); sylvain.laube@univ-brest.fr (S. Laubé);
querrec@enib.fr (R. Querrec)
© 2020 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings CEUR Workshop Proceedings (CEUR-WS.org)
http://ceur-ws.org
ISSN 1613-0073
1
Artifact: human product that has a defined function and the ability to perform this function
2
http://www.cidoc-crm.org/
3
Procedure: sequence of action allowing the accomplishment of a task
4
Period: a coherent set of phenomenons and cultural events in time. Source: CIDOC-CRM
navigation between the different periods, some research works have aimed at defining new
interactions metaphors (e.g., using time portals [3]).
As part of our work, by relying on the theoretical benefits of the Tangible User Inter-
faces (TUI) [4, 5], we hypothesise that using a TUI would improve temporal navigation and
the comprehension of the temporal evolutions of the activities represented inside a VE. For our
first demonstrator we use a Tangible Interactive Tabletop (TIT) allowing, first, to support the
interactions with a tangible object and also to show a complementary semantic view of the VE
(e.g., maps, diagrams).
Our work, therefore, aims to answer these two questions:1) Which metaphors of tangible
interactions could we offer so that a user could navigate through time in a virtual environment?
2) Which tangible interactors to use to realise these interactions?
2. Navigation through time
In order to formalise human activities asso-
ciated with socio-technical systems Laubé et
al. [6] described ANY-Artefact, an ontology
based on CIDOC-CRM 2 and Dolce [2]. It is
used to represent human activities5 , actors6
and knowledge7 . However, ANY-Artefact
does not allow the representation nor the re-
alisation of these activities inside a VE. To do
so Querrec et al. presented MASCARET [7], a
meta-model for representing human activities
inside a VE. However, there is no notion of
temporality in this model. This is why we will
rely on these two models to represent the tem-
poral aspect of human activities represented
in VR.
2.1. A model of time
In order to represent time inside an activity,
we first propose to align the notions of ac- Figure 1: Model of the representation of
tivity between ANY-Artefact and MASCARET. time (notions of P e r i o d , P r o c e d u r e
We then add the notions of P e r i o d , P r o c e d u r e and D u r a t i o n ) inside an activity,
and D u r a t i o n of action. We present our propo- based on an alignment between
sition with the figure 1. ANY-Artefact [6] and MASCARET [7].
In this figure, we represent a P e r i o d (blue
frame on the figure 1) which contains the activities defined according to ANY-Artefact (green
5
Activity: aggregation of the concepts of knowledge, actor and artifact to model human cultural and industrial
practices.
6
Actor: the person or the virtual character which has a specific function or position.
7
Knowledge: the set of technical knowledge at a given time.
oval on the figure 1) and MASCARET (orange activity diagram on the figure 1). According
to ANY-Artefact, an activity is composed of knowledge directly associated with the notion of
P e r i o d , artifacts and actors (respectively associated with the notions of resources and roles in
MASCARET ). These associations are shown in the figure 1 by the dashed lines. We add the
notion of D u r a t i o n of action, noted Δ𝑡𝑎𝑐𝑡𝑖𝑜𝑛 , as well as the notion of P r o c e d u r e which we define
as a succession of actions. The D u r a t i o n of a P r o c e d u r e , noted Δ𝑡𝑝𝑟𝑜𝑐𝑒𝑑𝑢𝑟𝑒 , is equal to the sum
of the D u r a t i o n of every action in the P r o c e d u r e . Finally, in order to navigate through time we
propose to create two timescales: the P r o c e d u r e s c a l e associated with the temporality of the
P r o c e d u r e s and the P e r i o d s c a l e linked to the evolution of knowledge (red elements on the
figure 1).
This model is the first step in our research works in order to offer a set of interaction paradigms
allowing temporal navigation in a VE.
2.2. Design workshop
We organised a design workshop with the aim of proposing tangible interactions and interactors
to navigate through time. The workshop took place on an afternoon and lasted for 4 hours. We
split the workshop in 2 parts, first a brainstorm session whose objective was to propose ways of
representing time in a tangible way, then a prototyping session during which the participants
had to propose low-fidelity prototypes.
The participants were 17, 11 of them software engineers, 2 are historians, 2 are psychologists,
1 is a designer and 1 is a mechatronics engineer.
2.2.1. Brainstorm
During the brainstorm the participants were asked how to represent time in a tangible way.
From the 20 distinct answers to this question 13 were inspired from everyday objects (e.g.:
hourglass, candle ...), 4 from cultural references (e.g.: flux capacitor from the movie ”Back to the
future”, mayan calendars ...) and 3 were metaphorical representations (e.g.: color gradient ...).
2.2.2. Prototypes
During the prototyping phase the participants could work in groups or on their own. A total of
11 prototypes were proposed, we classified them in 3 categories according to the interactions
they are proposing or their shapes.
Multi-part interactors Multi-part interactors are
composed of different parts that can be completely in-
dependent or attached to one another. The interactions
are controlled by the positions and the rotations of each
part be it relative or independent to one another.
For example the interactor in the figure 2 is composed
of 2 rotating parts. The user can navigate between
consecutive periods by rotating the 2 discs in the same
Figure 2: Picture of a prototype with 2 direction (e.g.: clockwise to go to the next period). If
rotating parts.
the user turns the discs in opposite directions he will go to the parent period (left part in
anti-clockwise rotation and right part in clockwise rotation) or to a child period (left part in
clockwise rotation and right part in anti-clockwise rotation). This interactor uses the same
movement as the rotation of the clock hands to navigate through time.
Multi-function interactors The multi-function interactors all implements multiple interac-
tions by using the position of the interactor as well as multiple buttons.
Figure 3: Pictures of the polyhedron prototype. Top: view from the top. Bottom: view of one side
showing the buttons.
The augmented polyhedron prototype in figure 3 uses a screen to display different information
to the user. When tilted on its side the interactor changes mode and can then be used to either
change period or modify the speed of time. Because the polyhedron has 4 faces on which it can
be tilted it is able to provide 4 different modes. The buttons on its sides can then be dynamically
modified depending on the selected mode.
Affordant interactors Affordant interactors’ shapes
are inspired by everyday objects related to time such
as hourglass or clock. The interactions are controlled
by using the object as if they were the real objects they
take inspiration from.
In the figure 4 we see an affordant interactor inspired
by clock. The user can navigate between periods by
moving the bigger clock hand and the smaller clock
hand can be manipulated to change the speed of time.
This prototype also proposed to add a button in the
middle of the clock to switch between different modes.
In the future we will draw inspiration from these 11
Figure 4: Picture of a prototype in- eleven low-fidelity prototypes to propose one or more
spired by clocks. tangible interactors allowing a user to navigate through
time on the two timescales we presented.
3. Implementation
In parallel with our work on the model of
time and the workshop we implemented a
prototype that we will use as a base for our
future works. However at the moment this
prototype is a proof of concept and does not
consider the results of the design workshop.
The prototype uses a Tangible Interactive
Tabletop (TIT) and a CAVE 8 and is shown
in the figure 5. It allows the user to explore
the environment in a first-person view and
to have a global point of view with the TIT.
The TIT is placed outside the CAVE to allow
another user to control the temporal naviga-
Figure 5: Picture of our prototype in which a tion while the first user is immersed in the
user is immersed inside the virtual en- environment. Using multiple displays has
vironment representing the Pont Na- been shown to have significant advantages
tional in Brest, France between 1861 compared to using a single display setup [8].
and 1944. Inside our virtual environment, we have mod-
eled a bridge crossing the Penfeld’s mouth in
Brest, France. Because Brest is a military har-
bour and the Penfeld is part of the arsenal, the bridge must move to allow the passage of ships.
Over time different bridges using different technologies were built. Our prototype currently
instantiates two of them: the Pont National (swing bridge, built in 1861 and destroyed in 1944 by
Allied bombardment) and the Pont de Recouvrance (vertical-lift bridge, built in 1954) and their
opening (respectively lifting) process. Each of these bridges correspond to a P e r i o d . The user
can select which bridge (namely P e r i o d ) he wants to observe, trigger the opening P r o c e d u r e
and control their course.
First, we integrated the 3D models of both
bridges in the environment and we then imple-
mented their respective opening P r o c e d u r e s .
We also developed the interface associated
with the TIT (cf. figure 6). The interface
is made up of three interaction areas: 1) an
aerial view of the environment (in our case
the mouth of the Penfeld river), 2) the list of
the P e r i o d s generated from our model and
3) the activity diagram associated with the
P r o c e d u r e running in the VE (cf., areas 1, 2
and 3 of the figure 6).
In order to allow the users to navigate Figure 6: Picture of our prototype’s interface.
8
CAVE: a system using projectors and stereoscopic effects to immerse a user in a virtual environment
through time on the timescale we defined ear-
lier (P e r i o d s c a l e and P r o c e d u r e s c a l e ) we implemented interaction paradigms based on two
metaphors: the time portal and the speedometer.
The selection of the observed P e r i o d (which is based on the time portal metaphor) allows the
user to control the artifacts he sees (e.g., the Pont de Recouvrance) and the realisable P r o c e d u r e s
(e.g., the lifting of the bridge). To do so, the user selects the P e r i o d by positioning the tangible
interactor on one of the elements of the P e r i o d list displayed on the area n°2 of the interface.
The P r o c e d u r e (e.g., opening the Pont National bridge) can be controlled in two ways: contin-
uous and discrete. First, the continuous mode, based on the metaphor of the speedometer, allows
to control the speed at which time is passing by. This is controlled by the angle of the interactor
beforehand placed on the area n°1 of the TIT’s interface. This navigation mode corresponds by
analogy to the fast forward mode in the control of a movie. The discrete mode, based on the
metaphor or the time portal, lets the user control the P r o c e d u r e step by step by positioning the
interactor on an action of the activity diagram shown on the area n°3 of the interface. This
mode is similar to a chapter selection while watching a movie.
4. Conclusion & Future Works
The goal of our work is to propose new tangible interactions paradigms to manipulate time
inside a Virtual Environment representing human activities. The first step in our work is to
model time in technical activities. Our model is based on the ontology ANY-Artefact and the
meta-model MASCARET. We add the notions of P e r i o d , P r o c e d u r e and D u r a t i o n . These three
notions allow the user to navigate through time on two different scales.
We organised a design workshop whose objective was to propose tangible interactions and
tangible interactors to implement the different functionalities previously mentioned. During
this design workshop 11 low-fidelity interactors prototypes were designed, we plan to draw
inspiration from them to design our interactor.
In parallel we have implemented a prototype using a Tangible Interactive Tabletop, a generic
interactor and, a CAVE. This demonstrator allows the user to navigate according to the two
timescales that our model describes.
The next step of our work is to implement an interactor taking inspiration in the result of the
design workshop. We will then evaluate our proposition in 2 conditions, first in a controlled
environment inside our laboratory and in a second time inside a museum.
References
[1] J.-B. Barreau, F. Nouviale, R. Gaugne, Y. Bernard, S. Llinares, V. Gouranton, An Immersive
Virtual Sailing on the 18th-Century Ship Le Boullongne, Presence: Teleoperators and
Virtual Environments 24 (2015) 201–219. URL: https://doi.org/10.1162/PRES{_}a{_}00231.
doi:1 0 . 1 1 6 2 / P R E S _ a _ 0 0 2 3 1 .
[2] G. Kassel, A formal ontology of artefacts, Applied Ontology 5 (2010) 223–246.
[3] R. Gaugne, F. Nouviale, O. Rioual, A. Chirat, K. Gohon, V. Goupil, M. Toutirais, B. Bossis,
V. Gouranton, Evoluson: Walking through an interactive history of music, Presence:
Teleoperators and Virtual Environments 26 (2018) 281–296.
[4] G. W. Fitzmaurice, Graspable User Interfaces, Ph.D. thesis, University of Toronto, 1996.
[5] E. Hornecker, Beyond affordance: Tangibles’ hybrid nature, in: Proceedings of the Sixth
International Conference on Tangible, Embedded and Embodied Interaction, TEI ’12, ACM,
New York, NY, USA, 2012, pp. 175–182. URL: http://doi.acm.org/10.1145/2148131.2148168.
doi:1 0 . 1 1 4 5 / 2 1 4 8 1 3 1 . 2 1 4 8 1 6 8 .
[6] S. Laube, S. Garlatti, R. Querrec, Histoire des paysages culturels industriels et humanités
numériques : le modèle d’activité humaine ANY-ARTEFACT, in: “Capitalismo en el de-
sierto: Materialidades, población y territorios en Atacama (ss. XIX-XXI)”, III Jornada de
Antropología e Historia, Instituto de Arqueología y Antropología, Universidad Católica del
Norte Centre de recherche et documentation sur les Amériques CREDA UMR7227 Labo-
ratorio de Etnografía (Le), Departamento de Antropología, Universidad de Chile Escuela
de Historia, Universidad Academia de Humanismo Cristiano, San Pedro d’Atacama, Chile,
2017. URL: https://hal.archives-ouvertes.fr/hal-01950651.
[7] R. Querrec, P. Vallejo, C. Buche, MASCARET: creating virtual learning environments
from system modelling, in: The Engineering Reality of Virtual Reality 2013, volume 8649,
International Society for Optics and Photonics, 2013, p. 864904.
[8] A. Kunert, T. Weissker, B. Froehlich, A. Kulik, Multi-Window 3D Interaction for Collabora-
tive Virtual Reality, IEEE Transactions on Visualization and Computer Graphics (2019) 1.
doi:1 0 . 1 1 0 9 / T V C G . 2 0 1 9 . 2 9 1 4 6 7 7 .