Human-computer interaction is primarily based on communication with visuals, i.e text and pictures. On the display screen, both infoI1nation from the system to the user and feedback in respond to the users actions, are presented. Thus, good visuals are important for effective conummication. This fact has also been recognized in design of modern software, for instance operating systems, programming environments, tools and application programs, where increased use of visuals can be seen. In addition, graphical design and visual interfaces are increasingly important as marketing arguments. In existing systems, a number of graphical techniques and visual effects are used, for instance windowing, icons, visualized metaphors, and typography to mention a few of them. However, when studied from the communications point of view it can be shown that many problems still exist. Compared to how professional communicators, like graphical designers, use visuals, only part of the full potential of visualisation is utilized. A particularly promising technique for communicating information is lexivisual presentation, which aims at creating visuals that are easy to read and understand. This paper discusses and gives some exemples of how lexivisual presentation techniques can improve various aspects of human-computer interaction.
Visual communication and human-computer interaction Human-computer interaction can be viewed from many different perspectives. In om work we stress a communication perspective where we view the interaction process as an exchange of messages between the user and the system. A good system should be communicative, which means that the user can interpret and use information in the system the intended way, that the user's intentions can be communicated to the system with ease, and that the user finds it stimulating and effective to work with the system.
A model of the humall-computer CO/lllllllllicatioll process Visllal cOlIll/lIIllicatioll is esselltial, sillce both Olllplll from the system alld illplll from the IIser are presellted 011 the screell. system's model .0.6i:--"'!!>-~ medium 4: >
user's model 1::::II
presentation ::.Iilllllill interpretation ( Many problems in human computer interaction originates from bad conmmnication between the system and its user. Communication problems can have several different causes. The most conmlon are discrepancies between the model built into the system and the user's knowledge and experience, and bad communication of the model to the user due to poor visual design. Vague and confusing inf01111ation, problems of orientation, misunderstandings, mistakes, and misinterpreted icons, are examples of problems related to visual design.
Much research effort is put into developing models of users in order to adapt the system's behavior to the user's actual level of knowledge and performance. A number of problems are being studied, such as how to model the user, and how to apply this knowledge to adapt the system's behavior. (Benyon et.al 1987, Morris 1987) We have chosen an approach to improving human-computer interaction which focuses on the design of the system's model, and the communication of this model to the user. In order to avoid communication problems, different types of inf01111ation need to be communicated to the user, among others • the tasks that can be performed • how to carry out various tasks • how different tasks are related to each other • the functions available at a certain stage • the structure and content of data related to the application • the status of the system - ongoing processes etc • what help infoffilation is available
This paper is focused on the visual aspects of communication, i.e communication with text and pictures. Good visual design is essential, since both information from the system to the user and feedback in respond to the user's input, are presented on the display screen.
Although graphical interfaces are becoming increasingly popular, many systems still suffer from communication problems. One reason nught be that a majority of existing systems, employ only a limited number of available communication techniques. The communicative properties of interactive system can be improved by utilizing knowledge of professional communicators, like graphical designers, draughtsmen, publishers, authors, illusionists etc (Kindborg & Kollerbaur 1987). A particularly promising technique is called lexivisual presentation (Lidman & Lund 1972). In the subsequent sections of this paper we will present the idea of lexivisual presentation and discuss its application to interface design. Furthermore, we will give examples of how systems might be improved using lexivisual techniques.
The importance of a close interplay between text and picture for effective presentation has been pointed out by Sven Lidman, who first described the idea of lexivisual presentation. Today, lexivisual presentation is widely applied for instance in newspapers, infonnation material, school books, magazines, reference works, and at exhibitions (Bild och ord akadel11.ien et.al 1988).
A good lexivisual presentation should be a combination of text and pictures, balancing model and reality, suggestive and infonnative presentation. The presentation consists of various pennutations of text graphics and pictures. In summary the most important principles for lexivisions are the presentation should have visual totality, each lexivision should be separate and self contained, giving the reader an overview the presentation should focus on the primary message, and it should give basic guidance for presentation of more detailed information the presentation should relate detailed infol111ation to its context the presentation should be based on text and picture interplay; the text is to bring forth the content of the picture, and the picture is to clarify the information presented in the text the presentation should be attractive and stimulating A lexivisual presentation is supposed to be read as a picture. Opposite to the way we read a text, from the left to the right and from the top to the bottom, we start reading the picture as one entety. Then we focus details of interest in the picture. In a lexivision the main part of the infonnation is often presented in the center, with references and explanations around it. Each detail in a lexivision carry a piece of infonnation, a picture caption for instance, has to relate the picture to the whole, as well as introducing the picture itself. Alternative presentations of the same message, such as a photograph and a drawing, give a deeper understanding. Graphic symbols are used to relate the parts to the whole. Lexivisiual presentations often cover a double page-spread. The space and fonnat are important for clarity of the presentation. A double page-spread, is a natural unit when presenting information on paper, but lexivisions can be presented in other fonnats as well. The tool box for presenting visuals consists of different types of pictures, text and graphic markers. Pictures can be of many different kinds - drawings, maps, diagrams, pictodiagrams, chm1s, strip cartoons, pictograms, photos etc. Captions,labels, headlines, introduction text and verbograms are the textual tools. Arrows, flags, bargraphs, tint boxes and panels, and color marking are the graphic markers.
Example of lexivisual presentatioll
directly 10 a part in the pictllre.
The different parts of the type writer Anti·glaro screen
Paper guide c~O..-.~..__n ,_"""'".",..."." .I.., ~_ .,..... -.... ....."...-...,...1'..' .-.0 ..· ..... _ ~..~ ol.._"•. ........"'.CV< ...,.....w••H ",,)<,11-11 Top cover 0..1..o.O".'.« ".."'.. _ ~:~:::.-:.t..~ ---,T,""r
.
Transparent shutter .c.~.,..~.,w:.,..=".~..'"U,,M''_i·:.~_"e.y..........
Keyboard end lights, see pages 6-7 -----' ~..- --- ---......
.........:// The lexivisioll is presented on a double spread page.
Stencil cutting .u_.....~.(.._>_0 ~..."......,,~ .,.-~.. .r.>.
Lrc.wJ:)-- c__.,..,O1...t1:oJ1N(\o.. ..........._~_.......""... ,I>..~....... ',~~ """...
~ lever for paper paper _insertion ballaRd for "......... ..~ .... •~V9_ :.:.::.'.;;."~~~".."...........-,., r ".. 1)<ping scillcs .....................................) 'loNl~
(
Implications of lexivisual presentation for
human-computer interaction
We previously stated that the requirements for effective conu1ll1llicaton in printed media
also apply to human-computer communication. Principles for how to select, stmcture,
represent and layout infOimation, can be employed in the design of interactive systems.
However, there are a number of differences between communication in printed material
and computers, due to the characteristics of the computer based medium. The possibilities
and the complexity are extended beyond the presentation of static infOimation, since
dynamic and reactive infOimation also can be presented
When discussing which rules and techniques can be of interest for human-computer conmlllllication, we fU'st want to state that many of the lexivisual rules and techniques can be identified in existing systems. However, their application seems neither systematical nor conscious. In our view alllexivisual mles and techniques can be applied, and would improve human-computer interaction. Some of them are however more obvious than others, which will be further discussed in subsequent sections. It should be pointed out that though we present the lexivisual principles under separate headings, they should be applied together to achieve the best effect.
Although this paper primarily discusses presentation techniques, the perhaps most important question in all communication is what to present. It is essential to decide which infomation is necessary and sufficient for a certain task. The presentation should not be overloaded with irrelevant and distracting information. At the same time, the user should not have to remember infol111ation which is known to the system. Funhel1110re, information should be presented in a fOim that does not have to be "translated" to be understood. An example of this is the WYSIWYG-principle.
Visual totality - overview and context An important principle in lexivisual design is to communicate information as a whole. The presentation should be based on an overview and successively present and explain infol1mttion in more detail. This is easier to achive in o'aditional media than in computer based media.
Computer based information is abstract, and it is difficult to perceive the wholeness directly. Furthermore, infOimation is often fragmented due to the restricted screen space.
sO'uctmed, which different parts and levels the system consists of, where you are and
how you orient yourself around these different parts. Besides the functions of the system,
it is also important to communicate the content and the structure of data in the system.
Current research in the hypermedia field refers to this as the orientation problem
In order to maintain a visl/al/a/alil)' the subject should be limited to what can be
presented in a convient layout. In our case we are limited to the screen presentation, and
om different ways of expanding this area by the use of dynamic presentation techniques,
such as windowing, scrolling techniqnes, and dynamic menus.
One form of visual entirety is the use of Ilisualized metaphors such as the desktop in
the Star system
Pull-down mel/us can be seen as a way of giving infOlmation about the available functions in the system. The user can browse through all the menus and get an overview of the tasks which can be perfolmed. Graphic techniques, such as shading or reversed video gives further information to the user. One reason for not having menus on the screen continously is the limited presentation area. Pop-up mel1llS solve the space problem since they do not crowd the screen with infoffiJation, but they do not give a visual overview. On the other hand they are faster to use once they have been learned. Lexivisual presentation rules say that relatiouships alld comparisolls should be immediately perceivable. Several techniques can be used to fulfill these requirements. Two of the most common are to show relationships with arrows, and to start with an overview and successively reveal additional detail, or vice versa (see the illustration below).
Visualizatioll of detailed illformatioll All arrow is used to show the COli text of the exploded dialog box, though ill our view the arrow is a bit too domillallt to be aesthetically pleasillg. This example is takellfrom OpCIJ Look (Hoeber 1988), the recelltly allllOll/lCed visual illterfacefor Ullix.
,
You have unsaved edl t5.
Do yO(1 \~ant '0 S,lVe thein, 01' diSC<1r"d them before loadtng? ( Silve
( 01scarc1 ) oJ.. ----------"
Wl'1te: File
•-t::J
''''~~~IC Name: byte.doc
- - -n=-to"d (store) (In< lude rile)
Text and picture interplay
"In good lexivisual presentation reality should be seen through the picture and through the
words it should be understood." (Lid man 1972, English abSO'act)
The text and picture should be presented in close connection to one another. The purpose
of the text is to bring forth the inf0l1l1ative content of the picture, while the picture clarifies
the abstact inf0l1l1ation presented in the text. Text and picture should complement each
other. A picture without text can be interpreted in a wide variety of ways
representing functions and objects were first introduced in the Dynabook system
(Goldberg et.al 1977), and were further developed in the Star system
In the Star system icons are presented with text, which makes interpretation easier. If icons are presented without text, the user usually has to learn the correct interpretation (Lodding 1983).
Icons wiJl/Qut text can be difficult to interpret ., .... ~ ", ": [ill (We leave it to the readertofigllre olltthe possible il/telpreiatiOl/s of these icons from Microsoft Word.) Furthennore, the text should be easy to read. In graphical design there are a number of typographical rules, for instance • on paper, some fonts are regarded as easier to read than others,
for instance serif fonts are easier to read than sans-serif fonts • lowercase letters are easier to read than uppercase letters • the use of different fonts in the same lexivision can improve
communication, but too many fonts can cause distraction Layout "The presentation should not be crowded. The layout should be visually striking but at the same time logical, both arrangement- and readingwise". (Lidman 1972, English abstract) Layout is an important factor for creating presentations that are clear and pleasing to the eye. Attractive presentations will have a motivating and positive effect on the user. However, the presentation should not become obstructive and annoying. The presentation should stimulate, not distract.
Many systems can be criticized for poor layout. Traditional and historical reasons has
often led to designs where infOImation is meant to be read sequentially, as in traditionally
designed printed material. In some modern systems however, the presentation uses a
more lexivisual layout, with important infolmation focused at the center of the screen.
The limited screen space enforces more or less "crowded" designs. This has partly
been improved by the use of windows and different interaction techniques. However, the
use of these techniques introduces new problems. Overlapping windows, dialog boxes
and windows with help information, can result in crowded, messy screens. In many
systems infolmation is fragmented, giving a lack of overview. Techniques for relating
infOImation could improve design on small screens. One suggestion is to relate windows
by using different shades of color
Reactive and dynamic information As stated above, the reactive and dynamic nature of computerized media distinguishes it from traditional static media. Even films and videos m'e static in a sense, since the sequence presented is unchangeable. One can not interact with a movie. Since lexivisual presentation is designed for stalic media, an additional set of communication techniques and principles are needed when designing interactive systems.
Infonnation in an interactive system is changed as a result of the two-way communication between the user and the system. Like in a theatre play actors appear, conUllllllicate, process messages, and vanish. This dynamic communication situation leads to special problems in presenting infolll1ation, for instance when, where, and for how long a message should be presented, and by which actor. Some information should be preseuted by system, and some information should be sent by the user.
Feedback becomes paniculal'ly important when manipulating infollllalion interactively, in order to infoll11 the user of what is happeuiug. Arcade games are often mentioned as good examples of interface design (Shneiderman 1983). In most arcade games the players receive continuous visual feedback, infol1l1ing the lIser of the stale of the system. Objects in these games are sometimes animated using comic strip techniques such as speed lines, which further enhances the presentation.
An additionally important area concerns status information, communicating changes in the system's state. Switching context or changing mode should be clearly communicated, for instance using different shades of color for window borders etc.
System processes can be paniculal'ly hard to understand, especially if they are invisible to the user. It is important to show, not only whm the result of for instance a search operation is, but also how the process works and proceeds. If a process is time consuming, such as a search, a copy, or a save operation, it becomes even more important to show what is going on.
Thus, techniques for process visualization should be used. To visualize for instance a compile-link process, comic strip techniques could be employed. Each frame in the strip shows a state in the process. During execution, the frame which represents the current state can be illuminated to illustrate, what is happening. In this way the user can achive an overview of what the system is doing. A similar technique has been employed, animating graphs to show the dynamic behavior of programs (B6cker et.al 1986). Another commonly employed visualization technique is to shift the shape of the cursor during time consuming tasks. Typical symbols are an hour glass or a Walch.
Examples of lexivisual interface design In the following section we will present a couple of examples that show the practice of some of the techniques and principles inherent in lexivisual presentation. The examples are all from the word processing program Microsoft Word 3.01 (Microsoft 1987). In analyzing Word, we found that the lexivisual principles were often violated. By applying lexivisual rules and techniques we have designed some alternative solutions. These particular designs are not yet empirically verified, but previous research in this direction has been conducted (Kollerbaur et.al 1988). Om suggestions have to be funher discussed, developed and evaluated. However, we believe it is imponant to give examples which shows how lexivisual thinking can be applied.
We have not made any major redesigns, ncither of the basic Macintosh interface, nor of the Word-system. Obviously, in some cases the screen becomes crowded with inf0I1l1ation. Since large screens are not common yet, we have chosen to show the examples on a smaller screen.
Lexivisual principles applied to a full extent, using spacious layout and other techniques, would probably lead to designs different than the ones presented here, specially if using a large two-page sized screen.
Presenting information in
context •
File
Edit
Seorch
Format
Font
Document Choose Page Previe... to di,pI8\j one or t",o printed p8Qe3 from lJour document contoininq the insertion point or the fir,t charecter of a ,election. You cen preview end adju,t the document', page lalJout using the (0110....;09 icon,:
Magnifier - click to turn the pOinter into 8 ffillgnifl,li 09918". t-love the meonifying glass to the part of the pege lJou 'Want to ,ee, then click. Or. ,imply double clicle on the icon.
To ,ee other perl" 3croll the rnaqnified documen!. ...lffll p!ge vie'" - click to return to the full-pege ~ di,play. You cen abo double-click anlJ""here in the NeH!
) ( Preuious) [ [ancel The user call choose relevant iliformation direclly. by pointil/g at the icon of interest.
File
Edit
Search
Format
Font
Document F==rliiFiiEiiiiiiiii::Epcaid;9:i:e::sp;:zr::;e:iu\zi:ie!i:I.U:i::mK urs m --~
6~!e~r~i~a~1~M~D:Ijo~o~i~!i!~i~~ Q..n!...P..!9~P.!!.V icon - click to dhplev one pl* at a lime (useful for full page monitors).
The poil/ler oj 'he bill/von relates the help ill/ormfllioll to the icon it describes. The balloon discriminates the help texf/rom other types oj messages. - r:::::tZZE?- =='::::':2f= :I:!l _:.4
:ee:==- - ,
Search Format Font Document Window
Kursmaterla' MOl 08 Preferences
I8l Oisploy As Printed ~ Show Hidden Teut Keep in Memory: 0 ( File ) o (Progrom ) n OK
JI (Cancel) Manniska-datorinteraktion
(ASY 3:8, DL2:8, PKlDK:8) .. ",..
.., .- / ~
.....'. -, ..' h' M ~~ltii~F~i1ieC~ ~D
I
Search Format Font Document Window I: utl"l lIndo
.......-................... Preferences I: lJ t I: OPI.J Paste Insert Grophi
I8l Oisploy As Printed ~ Show IlIdden Teut _. _S..h.._o.w_.'_lJ.__.._--~.- - -
Glossary ... Keep in Memory: 0 ( File ) 0 (Progrom) fi
OK JI (concel) Manniska-datorinter
t i o n (ASY 3:8, DL2:8, PKlDK:8) Poge 1
illlVIziciz ,Ize dialog box appear. ( (
File
Edit Seorch
Format Font Document misunderstandings, mislnte are examples of problems which cfln be related to poor visual design. In on ideal communication situolion. the interpreted message is the some 6S the intended messoge. We con improve the communicolive properties of Interactive system by careful design of visuals nnd by utilizing knowledge of professional communicators, like graphicel ~ers, draughtsmen, publishers, authors, 1Iluslonlsts etc. (Kindborg&Kollerbaur 1967) Even though grophical interfaces are becoming incre6singly popul6r, mon s stems still suffer from communicotion --, - ' Uncle6r information, disorient6tl.~~~~~~~I1!"!1~~~"""''''''' misunderstandings, misinterpret~? ico , are examples of problems which con be reloted to poor visual Em. In on Ide61 communication situation, the interpreted message is the some 6S the intended messoge. We can improve the communicative properties of interactive system by coreful emmof vl·S!!AlS Aod b" !!t~~Zing knowledge of professional communi co ,Found: 0 of 31.. EmDars, dr6ughtsmen, I>pnullb'~Ii ls.hers, 8uNtoho ~'~i~I~1U~S~i~o~s~t~s~e~t~c'I~II~i!
..... ~., ..
'1+ ...
IIIformation is given aboIlllOtal lIumber of instances of the search string in the document.
Old Look It is not shown where ill the document the scarch currently is. You do not know how much is already searched, nor how much of the doculIlenl is leJI 10 search through.
New
Look By giving the entire dialog box a specijic look, il call be easily distinguished from olher types oj boxes and windows. Note how the scroll bar is rcdcsigncd to communicate which page and which part of the pagc, one currelllly is viewing. We have applied the lexivi.wal principles of rclmillg dctailed informatioll to its lIle)."t, ami givillg all overview,
Presenting detailed information
Whelljillillg 0111 Ihe [ormllia Ihe edilillg of the documelll is disabled. 11,e IIser has to remember or tryout what the differellt Iypes o[ [ormals alld[onls wOllld look like. .Vew
Look This presentation would be morc communicative and [ollows Ihe lVYS/lVYG principle.
When selecting a new formal or [0111 in Microso[1 Word,
Ihis dialog box is presenled.
Form~a~ts~===~~-= 0 Strikethru 0 Outline 0 Shadow 0 Small Caps 0 All Caps 0 Hidden .. File
Edit Search end the communi we focus on lhe II communiC6t ion w essential, since b and feedback in r the displey scree
Character o Bold o Italic o Underline o Word u.J. o Oouble u.J. o Ootted U.J. font Nome:
Geneva "eluetica Monaco
; ,
Times Eyen though graphict:l] interfaces are becoming increasingly c men" COelOme Still c"ff OC 'rom corom"o' . [
OK ( Cance') It: ( Apply)
)j Position o Normal By: o Superscript , . - - , o Subscript L - - J Spacing ONormo' Oy: o Condensed o o [uponded ( FOlliS appear as they will inlhe document. The IIser can increlllelllolly lesllhe dif[erenl slyles and see Ihe resull wilholll leaving Ihe dialog box.
We have chos interaction w and the camm' we focus a communi essent" ond f th
Geneva Helvetica Monaco.
, , Tinle'
Size: R;
OJ ~ fi'i Position... ) Spacing... ) o Bold o 1t8/i< o o
Towards a general application of lexivision to human-computer interface design There are many similarities between the development and the use of text and pictures in computer systems and in other information communication ru·eas. From being regarded as something other than totally serious, the use of visuals is now regru-cled as a necessity for effective communication.
A continued and increased development of knowledge and application of visuals can also be foreseen. Up to now most applications are related to personal computers and personal tools of different kinds. The investigation of the use of visuals in more traditional application areas is in its very early stages. One reason might be the limited availibity of graphics terminals. However, many lexivisual rules and techniques can be used on semigraphics terminals.
But even if we can see many prospects in the use of visuals, there are of course a number of pitfalls. Consider what happened when desktop publishing gave us tools for production of printed materials. A kind of "Las Vegas" effect occurred when people untrained in graphic design used as many of the available functions as possible in a document; fonts, graphics etc. Increased availibility of interface design tools can lead to a similar development in the design of visual interfaces. This can be seen in the use of icons, and in some systems developed with HyperCard.
One way of improving the design of visuals could be to create hand books, or even better - computer based tools. The problem, as we have tried to illustrate it, is that there are no standm·d rules for designing good visuals. In some cases more concrete guidelines can be developed, but mostly design is a more "open" task. Many solutions could be satisfactory, and it can difficult to find the ulitmate visual for a certain communication task. The conclusion has to be that design of visuals is both an art and a more fomlal process. This view of design of human-computer interfaces has also been expressed by Heckel (1984).
Good visuals implies certain requirements on the production process. Production of lexivisions in printed material is a creative, iterative work and a result of a team work between experts in their specialized fields, visualizers and editors. Examples of successfulllexivisions, fantasy, the tossing of ideas between members in the team, are important characteristics of the process. The process stlli"ts with sketches of the visual totality and the pictures, and ends with writing the texts.
We claim that a similiar process is needed in order to produce good visuals for computer systems. The design has to be be iterative with successive refinements of prototypes. Revisions of prototypes ru·e based on results from evaluations. The ideal design team should include users, computer and systems experts, human factors people, domain experts, computer graphics people and experts on communication and media. Of course the role of a team member will vary at different stages of the design.
Finally, we have to stress the need for further research about usability and the effects of the use of visual presentation in computer systems. A number of continued studies of the lexivisual approach will be pelfonned in CLEA as part of our investigations of how to improve the communicative aspects in human-computer interaction (Kollerbaur et.al 1988). An importrull part of that research is to develop methods for evaluation of human computer interfaces.
This research has been funded by the Swedish National Board for Technical Development (STU). We would like to thank our collegues Alan Davidson (Depanment of Computer and Systems Sciences, University of Stockholm) and Kjell Krona (Department of Architecture, KTH, Stockholm) for giving us their helpful advise when writing this paper.
Benyon et.al. Systems adaptivity and the modelling of stereotypes. [n Proceedings from
Backer et.al. The Enhancement of Understanding through Visual Representations. In
Heckel, Paul. The Elements of Friendly Software Design. Warner Books, 1984.
Kindborg & Kollerbaur. Visual Languages and Human Computer Interaction. In
Kollerbaur, Kindborg, Larnhed, Petterson, Jieveskog. CLEA - A Summary of Research on Interactive Systems. University of Stockholm, 1988.
Morris, A. Expert systems - interface insight. In Procedings from HC1'87. Exeter,