This paper proposes a concept to use a 2D tactile mind map tool for a better integration of blind people into co-located meetings with the goal of structuring and organizing ideas during ideation. We mainly focus on presentation techniques to present mind maps to blind users. We discuss problems of existing analogue and digital tools, which support structuring and organizing ideas, for blind meeting participants. Further we outline the concept of a sequential method for presenting mind maps [1]. Finally the design of 2D presentation technique using the “touch-sensitive tablet display for blind and partially sighted users” (HyperBraille Project http://hyperbraille.de) is presented.
In business life methods for ideation, concept development and other forms of “creative thinking” are often done in co-located meetings using tools like mind maps for structuring processes and results. Taking a closer look at co-located meetings the following sub-processes can be found which also define the basic functionalities tools like mind maps have to support:
focus and highlighting new artifacts of interests can be done in several ways. Non-verbal behavior, for instance pointing at the artifacts, or verbal expressions, for instance mentioning the place of the object or its content, can be used to put an object into focus. Often a Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
TacTT '14, Nov 16 2014, Dresden, Germany 2. 3.
combination of verbal and non-verbal cues is used.
can be in verbal form but also non-verbal cues are used to communicate ones opinion, for instance by nodding/shaking one’s head to agree/disagree to a statement or raising ones hand to point out that one likes to add something to the ongoing discussion.
discussion might be manipulated following the discussion. Examples are: sub-artifacts (ideas) are added or deleted; the position of artifact in the structure is changed; names and descriptive information might be added, deleted or changed.
Tools like mind maps or metaplan software are used to support
this process of creating, discussing, organizing, displaying and
saving ideas. Analogue techniques, for instance big white papers
(“flip-charts”) and felt-pens have been used for this purpose.
Today several computer based tools exist to support such creative
and constructive sessions (e.g. Fremind (http://freemind.
sourceforge.net/wiki/index.php/Main_Page)). Most of these tools
are developed for single use. Some provide features for networked
collaboration over distance. Such tools would allow parallel and
synchronized manipulation of the mind map by several users but
still the use of mind map tools in co-located meetings is rare. The
more and more popular use of touch sensitive devices including
large table-tops raises interest in such software tools and leads to
related research and development activities e.g. [
Using digital alternatives to the so far analogue tools (“flip-chart”) shows potential to also increase accessibility of tools and processes and thereby to allow better participation of blind users (and other groups of people with disabilities, what is not discussed here) into collocated meetings. When talking about inclusion soon the discussion goes beyond the tool itself and provokes challenging research questions as including the mentioned aspects of non-verbal communication, which play a key role in co-located meetings. For better access we need: 1. A Tracking System: To allow better supported or automated access, verbal (speech recognition) and nonverbal communication cues have to be detected. 2. Reasoning and Translation of Information: To be useful the presentation of non-verbal cues has to be accurate, has to avoid false alerts and has to be selective to avoid an information overflow. Reasoning is needed to make sense out of verbal and non-verbal information and to make the presentation selective and useful. For example the combinations of the spoken sentence “look at this bubble” and a stretched finger allows recognizing that a pointing gesture occurred. Tracking could allow to identify what “this bubble” meant and allow the blind user accessing the information when thought it is necessary or of interest. Reasoning for human being is mostly done with no explicit effort, but machine reasoning in the needed scenario is a complex and demanding task.
blind and sighted participant use different views of the artifacts, the following points have to be considered: b.
meeting participants have to be made available in
the view for the blind user. An approach of a
synchronized UI is presented in [
elements (NVCs) to blind meeting participants:
Presenting NVCs to blind participant has to be
selective, as mentioned. Information overload
would make the system more disturbing than
useful, in particular when the acoustic channel is
used. Also connections between NVCs and
artifacts have to be established (for instance
pointing to an artifact, as above). In [
Manipulation of Objects: Also the blind meeting participant should have the possibility to manipulate objects (e.g. add, delete, rename, move bubbles of a mind map), where two issues have to be considered: First the UI must provide an accessible interaction modality. Second the synchronization process between the view for sighted persons and blind meeting participants has to work bidirectional.
These aspects are addressed in the DACH project which is
presented in [
This section summarizes the main accessibility problems of tools supporting structuring and organizing ideas during ideation processes.
Originally, ideation tools have been developed to “visualize” and to allow better memorizing and manipulating the process and the results. Such tools provide methods for structuring and organizing ideas e.g.: • •
Hierarchical relations as well as cross relations of artifacts Geographical clusters to put similar artifacts together and using colors and other cues to highlight aspects as relations, attributes and properties.
Sighted people process these relations at a short glance in parallel to focusing/reading the artifacts itself. For blind participants it is impossible to access these attributes in the same short amount of time without adaption of the views for blind meeting participants. They use a sequential approach (e.g. audio or haptic) and building an efficient mental map in a fast manner is of crucial importance for participation. The mentioned hierarchical structuring for traditional access helps and with NVC tracking and reasoning for selective presentation the situation can improve. But in addition alternative presentation methods would help in developing a mental map for better “coming and staying in the discussion”. Artifacts and their attributes form already a long sequential or hierarchical list which blind persons have to navigate and manage. More challenging are the high dynamical changes of the artifacts and attributes making it hard to follow. Artifacts and their attributes are subject to change during the structuring and organizing process, and most often the more they change, the better the process. For sighted people most changes of artifacts (for instance if a cluster was moved from the left upper corner to the right bottom corner) are recognized at a short glance whereas for blind participants finding changes, without explicitly telling them such changes, is a much more complex process. They have to search through the artefacts to figure out which of the artifacts have been changed. As a consequence it is important to give blind meeting participants the possibility to get informed when artefacts are changed.
[
As mentioned in section 2 it is important to inform the blind user
of changes of mindmap bubbles. In [
This section presents tow ways (edge-projection [
In [
User testing with edge-projection [
4.2.1 Selecting of Hardware Different tactile 2D presenting techniques for blind persons exist. One possibility is to use tactile graphics as produced by Braille printers. Another possibility would be to use the Graphic Window Professional device (Maple GWP http://handytech.de/ produkte.php?produkt=58&lang=de). A third one in consideration has been HyperBraille. Comparing these approaches outlines the advantages of HyperBraille:
The resolution of HyperBraille is much higher than other existing graphical tactile devices.
Structure and attributes of the artifact are not constant during ideation but change during the ideation process. In comparison to Braille prints HyperBraille allows to adapt immediately to the changes of artefacts during ideation process.
The HyperBraille display is also touch sensitive. The
touch sensitivity on the one hand can be used to support
information presentation for the blind user (information
can be presented according to the reading positions of
the fingers) and on the other hand it can be used directly
as input device to navigate through the artefacts (for
instance to zoom in or zoom out in the mind map).
4.2.2 System Architecture
The system architecture is similar to the system architecture
presented in [
Range of View: The number of bubbles in a mind map
can get very huge. It is not possibly to present the whole
mind map at once with a reasonable resolution.
Therefore zoom in and zoom out functionalities have to
be provided. Functionalities for shifting the area in view
left, right, up and down have to be provided. For that
the existing buttons on the Hyperbraille display can be
used. Based on the touch sensitivity of the HyperBraille
device special gestures can be defined to allow a direct
zooming and shifting of the mind map on the Braille pin
b)
c)
d)
matrix array. Reasonable gestures are already defined in
[
Presentation of bubble content: If the blind user is on a
bubble the content has to be presented to him. One way
is to use speech output. Speech output would be a very
fast way to present content to the blind user. However,
much speech output in co-located meetings has the big
disadvantage to overload the blind user with acoustic
information and the blind user is no longer able to
follow the ongoing discussion. Another possibility is to
use the HyperBraille display itself and present the
content via Braille letters to the blind user. This can be
done using a separate information bar on the
HyperBraille display. This approach is based on the
Braille Window System presented in [
However, as soon as the blind participant has some idea
of the structure of the mind map, it makes sense to allow
jumping to the change by a simple gesture or a key
event, which he has to be triggered. The mind map has
e)
•
•
•
•
•
to be moved in such a way that the region of the mind
map including the last change of the artefacts is always
at the same position (for instance left upper corner). A
further consideration to be made is to inform the blind
person before the update by another person is made on
his view that he can finish the task he worked on,
provoking issues of synchronization. Another
possibility to avoid a permanent change of focus for the
blind user is to use the concept of Braille Window
System of [
Focusing elements: A specific gesture executed over the element will put the focus on the element. Focusing elements is important to give the blind user the possibility to select elements he/she wants to modify (for instance deleting).
Adding bubbles: The blind meeting participant must have the possibility to include bubbles and to place it on the mind map. Therefore a specific gesture detected by the touch sensitivity of the device can be used to specify the place where the bubble should be added. The input of content can be done via a sepparate keyboard or via the Braille – keyboard of HyperBraille. Speech Recognition might be considered, but again might be disturbing in such meetings.
Removing bubbles: A specific gesture has to be designed for the HypeBraille so that the blind meeting participant can delete bubbles.
Modifying structure: The blind user must also have the possibility, via defined gestures, to delete and add connections between the mindmap bubbles and to move bubbles.
Highlighting of bubbles: During an ongoing discussion sighted people have the possibility to illustrate focused object by pointing to them. To give blind users a similar functionality a specific gesture has to be defined to allow highlighting of bubbles. However not only the view for the blind user has to be prepared for the pointing gesture but also synchronization and visual highlighting of the depicted bubbles in the view for the sighted user have to be established.
Touch gestures either have to be designed in a clever way or have to be executed in combination with a function key to avoid unaware executing of touch gestures during browsing through the mind map. Further it seems to be reasonable that all gestures have keyboard alternative both for blind users and blind users which might have problems with executing gestures.
Comparing two dimensional presentation techniques with the sequential methods using state of the art AT devices like Braille displays and speech output, two dimensional methods have the advantage that the geographical information are presented to the blind meeting participant per se. Based on the conceptual considerations research will be done on how devices like HyperBraille can help to support better access to complex and dynamic information structures and thereby allow to support participation in co-located meeting.
This work has been partially supported by the FWF (Austrian Science Found) with the regional project number I867-N25 and it has been produced out of the D-A-CH project. The D-A-CH project is a joined project between TUD, the ETH and JKU with the respective founding organizations DFG (German Research Foundation), SNF (Swiss National Science Foundation) and FWF (Austrian Science Found).