=Paper=
{{Paper
|id=Vol-1632/paper_6
|storemode=property
|title=Visual Interaction Design for Experiencing and Engaging with a Large Chronological Table
|pdfUrl=https://ceur-ws.org/Vol-1632/paper_6.pdf
|volume=Vol-1632
|authors=Kumiyo Nakakoji,Yasuhiro Yamamoto,Yusuke Kita
|dblpUrl=https://dblp.org/rec/conf/dihu/NakakojiYK16
}}
==Visual Interaction Design for Experiencing and Engaging with a Large Chronological Table==
https://ceur-ws.org/Vol-1632/paper_6.pdf
Visual Interaction Design for Experiencing and
Engaging with a Large Chronological Table
Kumiyo Nakakoji, Yasuhiro Yamamoto, and Yusuke Kita
Kyoto University, Japan
kumiyo@acm.org, yxy@acm.org, yusuke.kita@design.kyoto-u.ac.jp
Abstract
The goal of our project is to design visual interactivity for a computational
environment, which helps a user browse interconnected historical elements in a variety
of manners. We argue that a computational environment to help people explore
historical material needs to address the following three aspects: a network of changes,
intertwined lines as trajectories, and associatable multiple proximities. The paper
presents Timezoom, a prototyped environment, as a poof of concept to illustrate the
three aspects. We use the chronological table of the history of the city of Kyoto, Japan,
composed of 11,960 timestamped textual data elements.
1 Introduction
Timezoom is for users to engage themselves in the historical data by dynamically changing the
way the data is presented as their understanding and interests gradually develop. Our design is based
on our previous work on the studies of design and design provenance [4][5] as well as the inspiration
by the notion of “lines” described by Ingold [3].
Our project aims at designing the visual interactivity of a computational environment for a large
amount of timestamped data, so that users (i.e., data analysts, design researchers, and historians) in the
early stage of their research would find potentially interesting hypothesis, become aware of some
implications, come up with a few keywords for further investigation, or encounter some room for new
interpretations, by browsing the data through interacting with such an environment.
We use the chronological table as an example of the historical data, which was originally
published by the city of Kyoto in 1976 [1]. Kyoto, which used to be the capital of Japan between the
8th and 19th centuries AD, has a long history. The book consists of 505 pages of the chronological
table covering years between AD 511 and 1971. There are 11,960 timestamped (year and month
dated) textual descriptions as entries in the table. We scanned the table with the admission by the city
of Kyoto, and generated the formatted table in a CSV format by using OCR and manually fixing
recognition errors.
M. Düring, A. Jatowt, J. Preiser-Kapeller, A. van den Bosch (eds.): Proceedings of the 3rd HistoInformatics Workshop, Krakow,
Poland, 11 July 2016, published at http://ceur-ws.org
2 The Three Aspects
This section describes the three aspects we argue for in designing interactive environments for
users to explore a large amount of timestamped textual historical data in the early stage of their
research.
(1) A Network of Changes. By viewing design as “changing existing situations into preferred ones”
[7, p.111], our design studies in the domain of city development and urban design has focused on the
change a design provokes [4]. The subsequent changed situation leads to other problematic situations
and potentials in different contexts, thus resulting in re-designs. The world remains changing through
reiterating such design activities and a series of designs take place like chain reactions. Rather than
focusing on the provenance of a single designed artifact, we focused on multiple artifacts that are thus
interdependent on one another through the changes they provoke. “Expanded phenomena” may often
be observed in our history as well as in our daily life, without explicit recognition of their underlying
design activities [4].
Based on such understanding, we proposed a model and a method for depicting the history by
focusing on changes of objects [4]. The proposed framework uses a network of changes of artifacts,
persons, and situations. The temporal network structure, named C-network (Change-network),
consists of a large number of C-units (Change units), each of which is composed of the timestamped
three layers: A (Actor/Agency), B (Background) and C (Changed object, from C1 to C2). Each C-unit
may be linked to one or more other C-units through one of its three layers, forming a network of
changes.
Figure 1 illustrates one of the results of our case study that uses our model to depict the
development of streets in Kyoto. The city of Kyoto currently has double grid street system consisting
of many narrow streets (4 to 10 meters wide) and some wide streets (15 to 30 meters wide). Some of
the wider streets have been the results of widening narrower streets in order to introduce a tram
network at the time of modernization in the early 20th century (the tram systems were afterward
removed in the late 20th century). Those narrow streets emerged during the 9th to 12th century, when
people living in the city of Kyoto (then called Heiankyo) started to occupy parts of then-existing wide
streets and cultivated or built houses there without government permission due to the decline of the
governmental power caused by political confusion. At the time Heiankyo was built in late 700's,
Heiankyo had streets as wide as 12 to 84 meters because it was modelled after the Chinese cosmology
by Emperor Kanmu.
Figure 1: The Historical Development of the Streets of Kyoto Depicted as a Network of Changes
(2) Intertwined Lines as Trajectories. We are inspired by the concept of “lines” described by Ingold
[3], who uses a line to depict the life of a person, of a thing, of a city, or a wide variety of phenomena
such as music, speech, or travel. A line is drawn from left to right where the horizontal axis
corresponds to the flow of time (Figure 2). A number of lines, each of which corresponds to a person,
an object, music or a book, co-exist by sharing the same period of time (the same vertical area), and
some of them may be intertwined, producing knots. Such knots represent meeting points, encounters,
or interactions among the intertwined lines.
One of the interesting aspects of using the notion of lines to express trajectories of people and
objects is where the relationships among people and objects are represented as knots, and the
collection of the large number of intertwined lines form what Ingold calls a meshwork. In contrast to
the network of changes we described above, the notion of lines naturally expresses continuation,
duration, and co-existence.
Figure 2: Lines from a Meshwork
(3) Associatable Multiple Proximities. The goal of the DPS (Design Practice Streamliner) [5] system
was to help a designer re-experience the trace of thoughts during a recorded design meeting to
investigate how the decision had been drawn, what consequences it had led to, how the questions
being raised had been addressed, or how the problem had been resolved, by extracting the relevant
portions of recorded meeting captured in video, transcripts, and whiteboard stroke data by using
timestamps to glue segments in different media together.
When a user selects a small rectangular region on the whiteboard stroke data browser, DPS
identifies a set of timestamped strokes that are included in the specified area, and produces a short
video consisting of the video segments that have the same set of timestamps. Those timed segments
are often scattered at different time periods along the timeline, as the designers went back and forth
among different areas on the whiteboard during the meeting.
When the user specifies a phrase as a search query in the transcript browser, the system locates
one or more sentences among the timestamped transcripts that have the queried phrase, and likewise
produces a short video consisting of the video segments that have the same set of timestamps, as well
as highlights strokes on the whiteboard data browser to show which areas of the whiteboard were
drawn when uttering the queried phrase.
The interaction design of DPS is such that it would help the user search for points of concern
through spatial proximity over the whiteboard stroke data browser, symbolic proximity over the
transcript data browser, and through temporal proximity over the video browser. Starting with
exploring an area in the proximity of one type of media, the user is then presented with the associated,
possibly scattered data points in other media by DPS. Thus, DPS pulls out streams of design thinking
from the recorded design history, which may likely be intertwined, entangled, and weaved in the
design process across multiple types of representations, especially during the early stages of software
design.
3 Timezoom: A Prototyped Environment
Timezoom (Figure 3) has been prototyped as a proof of concept for the three aspects. It is
currently going through a series of revisions. Timezoom displays timestamped text data in the 3-D
space using the Web browser. The textual description of the highlighted data element is displayed at
the top of the window, preceded by the year and the Japanese calendar year, and followed by the
cluster ID’s (see below). Each timestamped text data element is diagramically visualized as a
horizontal thin blue line at the corresponding height of the time (vertical) axis; the length reflects the
length of its text description for the sake of giving visual identity to each data element.
Figure 3: Timezoom
Timezoom provides the following three ways to experience proximity:
Chronological proximity along the timeline. The environment allows a user to browse a
timestamped text data element one by one in the chronological order within the scope of the proximity,
which gives us a feeling of linearity among the successively visited data elements. The vertical axis
represents the timeline, where the time proceeds from the top to the bottom. There always a single
highlighted data element in the visualization, which is emphasized with red. The user may click on
one of the chronological-track buttons (triangles located on the right side of the window) to change
the highlight by traversing all the data elements temporally forward (i.e., the next one), or backward
(i.e., the previous one).
Symbolic proximity through the searching mechanism. The user may search for a particular word
or phrase among the text descriptions of the data elements. The identified data elements, the text
descriptions of which contain the queried word or phrase, are visually highlighted with green.
Clicking on one of the two sought-track buttons (the two triangle buttons at the bottom of the
window) traverses the located search results.
Semantic proximity through the clusters. The horizontal axis (the width in Figure 3) is currently
mapped to the clusters of timestamped text data elements in the order of their sizes. Clustering may be
done by a variety of ways, and we currently apply the Micro-Clustering algorithm [9] to identify
clusters among the timestamped text data using words composing the text description of each data
element. When one data element is highlighted, one of its belonging clusters is also highlighted, and
all the other data elements that belong to the highlighted cluster are visually stressed by changing its
color from light grey to dark grey (Figure 3). As each cluster is positioned along the horizontal axis,
the data elements of the same cluster are aligned vertically. The user may click on one of the cluster-
track buttons (triangles located on the lower right side of the window) to traverse all the data elements
belonging to the currently highlighted cluster.
While visiting data elements one by one, one may change the course of line tracking by switching
to another proximity. While following a series of the data elements within a certain cluster, one may
switch to one of the different clusters the highlighted element belongs to (by selecting the cluster ID).
Or, one may switch to simply following the data elements in the chronological order without being
bound to any clusters. One may find a phrase appeared in the textual description of the data element
interesting and search for the phrase and start browsing the searched results. Such switching proximity
context may serve as a knot to intertwine the two lines that the user has been engaging in.
4 Conclusion
Timezoom is an environment for the early stage of investigating historical data, which is likely to
precede more robust search (such as Singh et al. [6]) on the more formally represented historical data.
Timezoom is under development and we are working on evolving Timezoom in several ways. In
addition to bringing in elements of aesthetically meaningful visualizations such as works by Sugiura
[8] and Grafton and Rosenberg [2], we will associate the geographical information with each data
element so that we can use geographical proximity to experience a series of events within the spatial
context. This would especially be interesting to browse the Kyoto history data as Kyoto has very old
streets and buildings, which have been extended and demolished in a variety of manners through the
long history.
Acknowledgements
The research presented in this paper is supported by CREST, JST. We would like to thank Takeaki
Uno, Nobuto Matsubara, and Tomohiro Sakaguchi for their technical advice and support.
References
1. City of Kyoto (Eds.), The History of Kyoto, Vol.10, Gakugeishorin, 1976 (in Japanese).
2. Grafton, A., Rosenberg, D., Cartographies of Time: A History of the Timeline 1st Edition,
Princeton Architectural Press, 2010.
3. Ingold, T., Lines: A Brief History, Routledge, 2007.
4. Kita, Y., Nakakoji, K., Monnai, T., Depicting the History as Expanded Phenomena: An
Approach to Wide, Longitudinal Design Studies, Proceedings of IASDR2015 INTERPLAY,
Brisbane, Australia, 2015, 1084-1098.
5. Nakakoji, K., Yamamoto, Y., Matsubara, N., Shirai, Y., Toward Unweaving Streams of
Thought for Reflection in Early Stages of Software Design, IEEE Software, Special Issue on
Studying Professional Software Design, Vol.29, No.1, 2012, 34-38.
6. Singh, J., Nejdl, W., Anand, A., History by Diversity: Helping Historians search News Archives,
Proceedings of CHIIR 2016, ACM, New York, NY, USA, 2016, 183-192.
7. Simon, H.A., The Sciences of the Artificial (3rd ed.), Cambridge: The MIT Press., 1996.
8. Sugiura, K. et al., Experiments in Time Distance Map, Diagram Collection by Kohei Sugiura,
Kashima Shuppankai, 2014 (in Japanese).
9. Uno, T., Maegawa, H., Nakahara, T., Hamuro, Y., Yoshinaka, R., Tatsuta, M, Micro-Clustering:
Finding Small Clusters in Large Diversity, CORR abs, 2015, 1507.03067.