=Paper=
{{Paper
|id=Vol-91/paper-4
|storemode=property
|title=Towards a Development Methodology for Augmented Reality User Interfaces
|pdfUrl=https://ceur-ws.org/Vol-91/paperD4.pdf
|volume=Vol-91
|dblpUrl=https://dblp.org/rec/conf/mixer/KulasSK04
}}
==Towards a Development Methodology for Augmented Reality User Interfaces==
https://ceur-ws.org/Vol-91/paperD4.pdf
Towards a Development Methodology for Augmented
Reality User Interfaces
Christian Kulas, Christian Sandor, Gudrun Klinker
Technische Universität München
Lehrstuhl für Angewandte Softwaretechnik
(kulas,sandor,klinker)@in.tum.de
ABSTRACT
In this paper we describe why we believe that the develop-
ment of Augmented Reality user interfaces requires special 3D Designer
attention and cannot be efficiently handled with neither ex-
isting tools nor traditional development processes. A new Change User
methodology comprising both a new process and better tools Interface Design
might be the best action to take. Screen Designer
Designer
A requirement analysis on issues regarding the process, the
user groups involved, and the supportive tools for Augmented
Reality user interface development is presented. This opens
up a number of research challenges covering the tools, the
Interaction
process and the methodology as a whole. A new develop- Designer
ment process which is a first attempt to meet the newly found
challenges is briefly outlined. This process relies on high Solve tasks
parallelism and extends previously learned insights with us- Change UI using UI
Implementation
ability evaluation matters. Following, our complementary
proposed tool set gets introduced in detail. This set again Programmer
profited mostly from new tools fitting in the usability en-
gineering realm, which so far has been mostly ignored in Evaluate User
Performance
the field of Augmented Reality. First steps towards a de-
velopment methodology for the creation of Augmented Re- Usability User
Engineer
ality user interfaces, tackling the found requirements, are
thereby made. Finally, our planed future steps are shown,
meant to bring the development methodology further along, Figure 1: Use cases of all basic participating groups in
by solving important, but achievable, remaining challenges. the process (UML Use case diagram)
INTRODUCTION
Programmer The programmer changes the implementation
One of the main activities in Augmented Reality user inter- of the user interface by writing and editing low level code.
face development, which is inherently multimodal, is the ex- 3D Designer This type of designer is concerned with cre-
perimentation with different interaction techniques because ating 3D interaction / presentation elements which are to
it is such a young field. These have to be designed, imple- make an appearance in the user interface. An example
mented and evaluated. An important research issue here is to for a 3D presentation element might be the landscape for
establish a development methodology that covers these three SHEEP [11].
sub-activities and links them together more closely. Screen Designer The focus of this designer is the actual
screen layout, that is what is presented to the user at which
We believe the main groups of developers participating in location on screen.
Augmented Reality user interface development (Figure 1) Interaction Designer This designer wants to fine-tune the
are: interactions the user can experience. She will set which
multimodal inputs trigger which actions.
Usability Engineer The quality assurance of the usability
of the user interface is point of interest for the usability
engineer. This person combines all the roles of conducting
usability studies in one. He selects, briefs, and debriefs
the users for the study, prepares the evaluation materials,
conducts and logs the actual study and finally analyzes
and evaluates the results.
1
User The user actually uses the user interface by navigating Missing Tools
through it in an attempt to accomplish certain tasks. For However, the task of developing Augmented Reality UIs is
example she might want to place a roof on a building she in itself still rather cumbersome due to the lack of tools to
is constructing within an architectural Augmented Reality support the main three phases.
application.
Design Authoring tools for design would accelerate the de-
We identified several crucial requirements for each of these velopment significantly because they would allow quick
individual groups and for the development team as a whole assembling of user interface prototypes with various lev-
that are presented in this paper. We see our work as first els of functionality. Existing tools like Maya or 3DStu-
steps towards a methodology with a supporting set of tools dio can be leveraged by the 3D Designer, Adobe Photo-
for the development of Augmented Reality user interfaces shop or Microsoft Paint might be an aid for the Screen
addressing these requirements. Designer. The support for the Interaction Designer is im-
proving with projects like The Designers Augmented Re-
Building on our DWARF framework [2], we have already ality Toolkit [10], which offers a collection of extensions
successfully tested [11] a new methodology for user inter- to the Macromedia Director multimedia-programming en-
face design and implementation. The core idea was to let de- vironment, but still there is much work to be done.
signers and programmers work simultaneously in the same Implementation The actual implementation of Augmented
room. In this paper we would like to present a new usability Reality user interfaces is made easier by a few frameworks
evaluation tool that allows us to further add simultaneous us- such as DWARF [2], and Studierstube [15]. Implementa-
ability evaluation by usability engineers. The tool logs data tion usually takes place in an Integrated Development En-
about the user interactions and visualizes it to the usability vironment (IDE). Like mentioned earlier, first frameworks
engineers in real time, thus extending the work presented by for designers are starting to emerge [10], but usability en-
Lyons and Starner [9]. gineers are still kept in the dark.
Evaluation There are a number of imaginable tools, like au-
The remainder of the paper is organized as follows: In the tomatic gathering and visualization of user performance
section The Problem we describe the requirements for a new data, allowing the quick generation of usability evaluation
development methodology. The section Research Challenges results, which could then be fed back to earlier phases.
lists numerous open questions. The section Our Approach Unfortunately this class of tools is also in very short sup-
describes our prototypical methodology. Finally, the section ply for Augmented Reality applications.
Future Work discusses the next steps we intend to take.
If we had proper tools like this, the generation of intermedi-
THE PROBLEM
ate milestones would be much faster and thus make it more
bearable to encounter problems in a post implementation
The development of Augmented Reality user interfaces re-
evaluation phase, because a new iteration of design and im-
quires special attention for multiple issues. These issues are
plementation can be put together rather easily again.
presented in this section.
Additionally, tools which actually do exist, usually only ad-
Process Issues dress their specific problem domain, with no or little inte-
In traditional user interface development, a waterfall or an gration with other related tools. In Augmented Reality ap-
extended waterfall process (Figure 2) is followed [12]. Start- plications, objects are registered in 3D [1]. Therefore, after
ing with the phase design, it is proceeded to the phase imple- completing a screen design, using a 2D tool, the designer
mentation and finally a phase evaluation. These phases are usually needs to map the therein contained objects to 3D.
run highly sequential with no or little room for feedback or She might have decided to keep a presentation component,
dependencies. This type of process works well if you have keeping the user up-to-date on an important variable, like the
enough details about the design space and in general can an- amount of rescued sheep in a sheep herding application [11],
ticipate implications of design changes well in advance. in close reach, head-fixed [4] in the left right corner, all the
time.
Design Currently, the screen designer has to re-create her earlier 2D
design in 3D using a completely different tool for mapping
3D objects. It would be much more efficient if she could
instead import her 2D design into a 3D registering applica-
Implementation tion. But this is not possible without much better inter-tool
integration.
Evaluation Unclear Design Space
For traditional desktop software, vast amounts of knowledge
on usability data exist, which created extensive and complete
Figure 2: Basic waterfall process (UML Activity dia- standard guidelines [16] which limit the design space to a
gram) known usable and working subset. Since Augmented Real-
2
ity applications are a comparably new development, such a On the process the main challenges are:
knowledge base is still to be built. So for now we are con-
fronted with a vast design space and a big uncertainty which • Limit to parallelism? By conducting multiple different
designs will work and which will not. development phases at the same time much better feed-
back can be attained. But how parallel can the process get
Unclear Non-Functional Requirements without losing reasonability? The different phases of the
Traditional software can also benefit from clear non-functional process have undeniably certain dependencies which will
requirements. For example a web-site has to be navigate- probably not allow a total parallel execution.
able, which meaning is defined in web-style guides together
with all other non-functional requirements which are proven • Formal process? Only by obeying a formal process simi-
to be sufficient. But which non-functional requirements do lar to Extreme Programming [3], built on reasonable rules
we have to impose on Augmented Reality user interfaces? and process patterns [5], a highly parallel development
The graphical portions of the UI should probably be concise can be accomplished. But which practices do apply the
but what exactly does this mean? best on Augmented Reality user interface development?
• Persistence of UI experiments? It is in the nature of rapid
Summary proto-typing to experiment with different variations of the
Because of the lack of tools and uncertainties, traditional wa- UI in quick succession. However, after testing a number
terfall cycle processes are not suitable for developing Aug- of UIs, it is very desirable to be able to roll-back into a
mented Reality user interfaces efficiently. But even a new previously evaluated UI iteration since it may have turned
process cannot make up for the lack of suitable tools. So a out to be best suited after all. It is a challenge to build
new methodology based on both a better process and usable the process in a way to ensure the results of these prior
tools is needed. experiments are not lost.
RESEARCH CHALLENGES Finally, on the methodology as a whole:
The previously identified problems result in a number of
research challenges on all areas tools, process and on the • Limits? Is the new methodology only suited to create pro-
methodology as a whole which are the focus of this section. totypes for temporary experimentation or might it actually
yield usable products which can be deployed at the site of
On tools the main questions are: the customer?
• Which tools? There are numerous paths to take in sup- • Validation? Does the methodology actually fully meet all
porting the main three user groups, resulting in a large requirements we impose on it? Answering this is also
design space for tools. It is a challenge to gain clarity re- a challenge, since the exact definition of requirements is
garding which type of tools will have the largest benefit. still a non trivial task.
• Tool integration? By integrating tools with each other, a Going deeper into the research challenges, we will now take
much better work-flow between these tools can be lever- a closer look at the tool questions. The issues regarding
aged building on tool chains. Where are the limits to inte- the process and the methodology as a whole cannot be con-
gration and which integrations are reasonable at all? sidered in any more detail until more future work has been
• Tool mapping? Some tools might be useful to more than done.
one user group thanks to a high level of integration. The
presentation of multiple tools simultaneously to certain Tool Combination Design Space
user groups might have more value, than the sum of each In an attempt to tackle some of the tool research challenges,
single tool on it’s own merit. It is a challenge to figure out it is helpful to correlate a list of likely supporting tools with
which tool combinations map best to which user groups. all three groups in a matrix like found in table 1. Following
• Tool automation? The more knowledge on UI design is this, the value of the different pairs can be assessed to learn
accumulated, the more ideas for automation features in which challenges are the most worthwhile to be confronted
tools can be generated. For example, basic clear cut de- first.
sign principles which have been shown to apply in cer-
tain scenarios could be enforced in design tools. Since we IDE or Authoring
still lack knowledge in this area, it is unclear which au- A 2D Paint Tool is probably only beneficial to the screen
tomations will be indeed feasible in the future. Instead of designer such as a 3D Modeller also probably cannot serve
testing against known usability problems, there have been anyone but the 3D designer. Basically these are already au-
interesting approaches in web interface development, like thoring tools. An authoring tool for interaction could of
WebRemUSINE [13], which try to automatically identify course also help the interaction designer putting together
new usability problems. This is done by looking at the new interactions. Generally, any integrated development en-
level of correspondence between how users actually per- vironment or authoring tool should be a great benefit to all
form tasks and the intended system task model. This idea three user groups if they are adopted enough to their re-
might also be applicable to Augmented Reality user inter- quirements. For any programmer an Integrated Develop-
faces. ment Environment (IDE) is already a standard tool to rely
3
Tool Designer Programmer Usability all components should obviously be useful for a program-
Engineer mer. The usability engineer could also profit from such a
2D Paint Tool + - - tool, if it was integrated with performance visualization. He
3D Modeller + - - would use the tool to indicate which data he is currently most
IDE or + + + interested in, which is then visualized.
Authoring
Performance Logging - o +
& Visualization
Wizard of Oz + + + Interaction Graph
Automatic Testing + + + In multimodal interaction, inputs from different media chan-
Monitoring Tool - + + nels trigger defined actions. For example, by combining a
Interaction Graph + + + speech token with a gesture, a wall could be deleted in an
architectural Augmented Reality application. A tool could
Table 1. Tool combination design space matrix visualize this interaction graph, display received tokens and
in general show the progress of the user in his current task.
Such a tool could be beneficial to all three user groups. The
on. Likewise, the usability engineer could use an authoring
interaction designer could use such a tool to visualize his
tool to put together user performance visualizations or to de-
work and even create new interaction paths with it. The pro-
fine tasks for the user to attempt which performance is then
grammer could use it to learn at which interaction his code
automatically measured.
got stuck to ease debugging. Finally the usability engineer
could use such a tool to learn in which interaction the user
Performance Logging & Visualization is currently struggling, if this is not obvious through other
User performance data is only directly interesting to the us- means.
ability engineer and only has an indirect impact on the de-
signer and programmer. However, the programmer might Summary
benefit from this feature, too. If it was automated enough to As a result, we are confronted by a large amount of research
do some initial rough tuning concerning usability, the pro- challenges covering multiple areas, of which we only had
grammer might be able to later skip implementation code chance to look closer at tool questions for now. Even here it
changes fixing obvious usability flaws. is still unclear if the list of tools is complete and how much
the tool integration can cover. By evaluating tool with user
Wizard of Oz group correlations, initial ideas were gained which missing
A Wizard of Oz [14] tool could actually benefit all three user tools seem to be the most promising. Although our focus is
groups. The interaction designer could use it to test in ad- mostly on tools, we will now briefly cover our process ap-
vance if certain interactions pan out or not, before actually proach in the next section after which our attempt at meeting
prototyping them. The programmer could leverage this tool a number of tool needs is detailed, too.
for feature dummy implementations to make early integra-
tion tests between partially incomplete features. The usabil- OUR APPROACH
ity engineer could conduct simulated full-featured usability To make up for the lack of tools, a better process offering
studies by faking yet missing functionality to gain insights much more feedback between phases is necessary. In fact
into usability aspects, which would normally not be attain- we believe that only maximizing this feedback can offer us
able until much later into the implementation phase. enough efficiency until our knowledge base is large enough
to allow older, slower paced, sequential processes.
Automatic Testing
Automatic testing tools of various sorts would again bene- To maximize this feedback, we propose to run all three phases
fit all three user groups. There could be a tool to check for design, implementation and evaluation in parallel (Figure 3).
conformity of standard design guidelines, which would ease
the life of the designers by avoiding trivial design errors. A Design Implementation Evaluation
similar tool to JUnit could quickly double check mandatory
functionality, after code refactoring has taken place by the
programmer. Once enough usability knowledge in the area
of Augmented Reality has been accumulated, hard usabil-
ity guidelines might crystallize themselves. Their confor-
mity could be again tested against by automatic tools, which
would remove the strain of the usability engineer to evaluate
these then trivial usability parameters. This enables him to
focus on the still unclear and less studied usability questions. Figure 3: Proposed parallel process (UML Activity dia-
gram)
Monitoring Tool
A monitoring tool visualizing the state of the distributed Aug- We already learned a few valuable lessons regarding the pro-
mented Reality application and the communication between cess within the earlier SHEEP project [11]. In Jam ses-
4
sions, development takes place at run time obeying same Multiple peers or the evaluation monitor himself might at the
time & place principles. This allows a crowded group work- same time observe internal system behavior and even fine
ing with peer code reviews and on-the-fly insertions of al- tune the system on the fly while observing usability implica-
tered system components, for quick integration tests. These tions immediately.
sessions proved to increase the development speed signifi-
cantly. This process also allows playful exploration, because This lab-based approach is usually considered as “Local Eval-
sub-components which have an impact on the user experi- uation” with both the user and the usability engineer in the
ence can be swapped during run-time effortlessly, enabling same place at the same time. We believe, this is still the
quick assessment of different variations. Iterative, continu- best way to capture qualitative usability data on Augmented
ous development is an implication of this. Reality user interfaces. However, when Augmented Real-
ity systems are used on a more frequent basis globally, a
When a high level of tool integration is achieved to support remote evaluation approach using Remote Usability Evalu-
the efforts of all user groups in all three development phases ation Tools [7] might be more reasonable. Here the system
a very fast, feedback-driven and parallel process to develop usually presents a wizard-based dialog to the user, asking
Augmented Reality user interfaces like proposed might be her details about her opinion on the usability problem after
indeed realizable. recognizing a critical usability incident [6] automatically or
after the user triggered the dialog herself. By design, this
Now, our newly developed supportive tools for the usabil- requires quite a lot of effort on the part of the user her-
ity engineer are presented in detail, after which a few other self. Additionally, great care has to be taken regarding issues
older tools are also briefly introduced. Finally, a possible of user privacy when passing on collected data without her
tool combination for the usability engineer is presented. prior consent.
With this in mind, the mentioned software components are
Usability Evaluation Tools now covered in more detail. The core component is a fully
At Technische Universität München we have developed a automated logging tool to capture events from the running
framework for usability evaluations in the field of Augmented Augmented Reality system. For this it is important to men-
Reality, covering both process as well as software issues, ap- tion that Augmented Reality systems, which are built mod-
plicable on applications based on DWARF [8]. ularly leveraging the DWARF framework, communicate in-
ternally mostly by means of CORBA based events running
Our momentary focus lies on the therein newly developed through event channels which can be effortlessly tapped into
software tools, but before presenting these in detail it is worth- by any logging tool interested in doing so, such as the newly
while to give a quick overview of the intended process for developed logger.
conducting usability evaluations by looking at a possible room
setup (Figure 4). A manual data entry tool (Figure 5) can be leveraged to take
quick written notes for later review, which are also directly
passed on to the data logging component.
Figure 4. Room setup for usability evaluations
The setup might look like this at crowded group working Figure 5. Dialog to enter usability data manually
when even an end user is taken into account while debugging
the system. The user is placed at a suitable distance of the
usability engineer / evaluation monitor who is busy entering All performance measurements can finally be visualized in
observations (Data Entry) into the usability logging system real-time during usage with a number of highly flexible and
(Data Logging) while also monitoring what the user actually adaptable scripts.
sees on screen (Action Visualization) and while monitoring
real-time visualizations of measured usability data (Data Vi- Figure 6 shows a number of different sample visualizations.
sualization). It was decided to base the visualization off the GNU General
5
Public licensed (GPL) third party tool ploticus 1 for multiple completion times outside of the standard deviation. Finally
reasons. Its’ scripting language proved to be well suited for below each task a number is printed, which depicts the num-
rapid prototyping of new visualizations while maintaining a ber of averaged tasks, which is equal to the amount of users
high level of ease of use. Additionally, it already had all the who took this task.
2D plotting support we required, that is it supports out of the
box all standard 2D plotting styles including line plots, filled We also prepared a median version which renders a big dot
line plots, range sweeps, pie graphs, vertical and horizontal at the median time for each task while the box-plot extends
bar graphs, time lines, bar proportions, scatter plots in 1D or to the 25th and 75th percentile. Error-tails extend to the bor-
2D, heat-maps, range bars, error bars, and vectors. der values while smaller light dots show the individual task
completion times for all users.
Numerics, alphanumeric categories, dates and times (in a
variety of notations) can be plotted directly. There are ca- All range visualization parameters can be easily adopted on
pabilities for curve fitting, computing linear regression, and a case-by-case basis.
Pearson correlation coefficient r. There is a built-in facil-
ity for computing frequency distributions. Means, medians,
Value Timeline
quartiles, standard deviations, etc. can also be computed out
of the box meeting our needs for default statistical functions. This visualization has the same parameter requirements like
the Relative Error visualization. Here it is merely shown
For the first sample study we conducted [8], the four shown which event type value (y-axis) occurred at what time (x-
visualization types have been assembled. axis).
Before elaborating the details of these different types, the Figure 6 actually shows a slight modification of this basic
usability data log file format must be exposed. It is a stan- visualization. An additional line visualizing a study specific
dard ASCII file in which each line represents exactly one additional event type and value development was added ef-
log file entry. Each entry must have six components to gain fortlessly to be able to better spot usability flaws of a specific
unambiguous data. The first mandatory component encom- nature [8].
passes the detailed current date & time of the log entry for
later time dependent data mining. The second component Absolute Bars
stores the study name, since multiple studies are to be con- Requiring the study, task and type parameters, absolute to-
ducted, which are not to be mixed up. For similar reasons tals (y-axis) of all different values (x-axis) are rendered in
and to facilitate intra-user comparisons and task time taking, horizontal bars. If no user is specified, it will output aver-
user and task names are stored in the third and fourth com- age bars together with a specification on how many users the
ponent. The most interesting components are the two last script averaged over. However, if a user name is given, it
ones since they store the logged event type which might be will output bars using data from this specific user only.
e.g. a Button-click and its’ corresponding value e.g. Hit or
Miss. Sample usability study results [8] using the above tools are
out of scope for this paper but it should be mentioned that
Leveraging this log file format, the visualization types from our sample study was very promising.
top left to bottom right shown in Figure 6 are:
Relative Error Other Tools
This script is the least flexible, since it requires the value In this section other older tools developed by us, which sup-
fields to be exactly Hit or Miss for the to be analyzed study, port the first two phases are briefly introduced because they
user, task, and type combination. It visualizes the resulting will offer insights on future integration.
accumulated hit-ratio (y-axis) over time (x-axis). The final
hit-ratio is additionally printed separately in a box. Our framework for multimodal interactions is an UI archi-
tecture described by a graph of multiple in-/output and con-
trol components (User Interface Controller (UIC)) ([11], Fig-
Task Time Range
ure 7 bottom-left).
Requiring only the specification of the to be analyzed study,
a range of all task (x-axis) completion times (y-axis) aver- The UIC can be visualized, and since it shows the complete
aged over all participating users is visualized. These times user interface interaction graph, it is very useful for interac-
are extracted from the log file by filtering for special event tion designers. This tool is a very close approximation of the
types, which mark the begin and end of any given task. interaction graph tool, mentioned earlier. However, it is still
nowhere feature complete.
The actual ranges can be easily visualized in different ways.
Shown is the mean and standard deviation. The biggest dot The arbitrary event streams within DWARF, as well as all
indicates the mean times while the error bars extend to the participating communicating components can be visualized
standard deviation. The smaller light dots show the individ- by our general-purpose monitoring tool DWARF Interactive
ual task completion times of all users. The stars denote task Visualization Environment ([11], Figure 7 bottom-right) which
1
http://ploticus.sourceforge.net currently serves as a debugging tool for programmers.
6
Figure 6. Sample real time visualizations of logged usability data
Given all these tools, a combination which should hopefully e.g. install a data filter for better usability, thereby in effect
prove to be useful to the usability engineer in future scenar- overhauling the design.
ios is now presented.
FUTURE WORK
Usability Engineer Tool Combination There are still many challenges to solve providing us with
In Figure 7 the UIC, the monitoring tool and the user perfor- multiple objectives for future work. Of course it is future
mance real-time visualizations are combined on one screen. work to implement the missing tools and achieve a high
level of integration to be able to better follow the proposed
The monitoring tool (bottom right) shows raw unfiltered event methodology. One of the first easiest integrations to do is to
communication between service components while at the add Wizard of Oz functionality to our UIC.
same time showing all running services with full details on
their states. The current version of our monitoring tool is Additionally, the monitoring tool could be integrated with
really only useful for the programmer, but extensions are the user performance visualizations to make this tool feasi-
imaginable which make this worthwhile for the usability en- ble to the usability engineer. Currently, a fixed set of scripts
gineer after all (see section Future Work). The UIC (bottom for visualization have to be pre-selected prior to the study by
left) might help the usability engineer to understand where the usability engineer which will then be constantly updated
the user is currently within the interaction graph. with live data. However, it would be much preferable if the
usability engineer could change these visualizations on-the-
Finally the real-time user performance measurement win- fly by e.g. clicking on a map representing the system state
dows (top row) should enable the evaluation of the actual and exchanged events, similar to what the monitoring tool
usability at the same time. While the first two tools could already offers to DWARF.
although reveal that a certain action was successfully trig-
gered by the user, it does not become apparent how many An authoring tool for the interaction designer is another very
tries there were, at which time frame, or how many errors critical next step, since this type of user still has clearly the
there have been until this final tool is taken into considera- worst tool support. Being overly visionary, this tool could
tion. even reach bootstrapping proportions. That is the designer
starts out with a very basic authoring tool based on Aug-
Observations in the top windows will likely usually lead to mented Reality and uses this tool to extend itself by build-
implementation fine tuning to e.g. trigger actions differently ing new widgets which can create ever so bigger interactions
or they might reveal the need for a whole new service to slowly creating a full-blown user interface.
7
Figure 7. Usability engineer tool setup mockup
Currently we only aim at mastering a better process of man- 5. A. G RANLUND AND D. L AF, A pattern-supported approach to the
ually designing, implementing and evaluating user interfaces user interf ace design process, 1999.
for Augmented Reality applications, but in the future we will 6. H. H ARSTON AND J. C ASTILLO, Critical Incident Data and Their
Importance in Remote Usability Evaluation, in Human Factors and
also want to invest in proactive UIs. Here the application Ergonomics Society 44th Annual Meeting, pp. 590–593.
evaluates itself during runtime and changes its’ own user 7. N. KODIYALAM, Remote Usability Evaluation Tool, Master’s thesis,
interface design, and corresponding implementation, auto- Virginia Polytechnic Institute and State University, 2003.
matically on-the-fly. For example, by observing user behav- 8. C. K ULAS, Usability Engineering for Ubiquitous Computing,
ior patterns over time, it would be possible to take note of Master’s thesis, Technische Universität München, 2003.
9. K. LYONS AND T. S TARNER, Mobile Capture for Wearable
never used functionality which could be hidden to generate Computer Usability Testing, in Proceedings of IEEE International
a less obstructing UI. The process itself needs much more re- Symposium on Wearable Computing (ISWC), October 08-09 2001,
finement which will be achieved by conducting more exper- Zurich, Switzerland, pp. 69–76.
iments gradually accumulating hopefully in a formal model. 10. B. M AC I NTYRE , J. D. B OLTER , E. M ORENO , AND B. H ANNIGAN,
Augmented Reality as a New Media Experience, in Proceedings of
the 2nd International Symposium on Augmented Reality (ISAR
In summary, traditional development processes and current 2001), New York, USA.
tools are ill-suited for Augmented Reality, and only by im- 11. A. M AC W ILLIAMS , C. S ANDOR , M. WAGNER , M. BAUER ,
proving on both the process as well as developing new or G. K LINKER , AND B. B R ÜGGE, Herding Sheep: Live System
integrating existing tools a more suitable platform for creat- Development for Distributed Augmented Reality, in Proceedings of
ing Augmented Reality user interfaces can be established. ISMAR 2003.
12. D. J. M AYHEW, The Usability Engineering Lifecycle, Morgan
Kaufmann Publishers, 1991.
REFERENCES 13. L. PAGANELLI AND F. PATERN Ò, Intelligent Analysis of User
1. R. T. A ZUMA, A Survey of Augmented Reality, Presence, 6 (1997), Interactions with Web Applications, in ACM Symposium on
pp. 355–385. Intelligent User Interfaces, San Francisco, CA, 2002.
2. M. BAUER , B. B RUEGGE , G. K LINKER , A. M AC W ILLIAMS , 14. T. R EICHER AND T. KOSCH, Software Design Issues for
T. R EICHER , S. R ISS , C. S ANDOR , AND M. WAGNER, Design of a Experimentation in Ubiquitous Computing, in The Second Workshop
Component–Based Augmented Reality Framework, in Proceedings of on Artificial Intelligence in Mobile Systems (AIMS 2001), Seattle,
the 2nd International Symposium on Augmented Reality (ISAR Washington, USA, August 4, 2001.
2001), New York, USA.
15. D. S CHMALSTIEG , A. F UHRMANN , G. H ESINA , Z. S ZALAVARI ,
3. K. B ECK, eXtreme Programming Explained: Embrace Change,
L. M. E NCARNAÇ ÃO , M. G ERVAUTZ , AND W. P URGATHOFER,
Addison-Wesley, 1999.
4. S. F EINER , B. M AC I NTYRE , M. H AUPT, AND E. S OLOMON, The Studierstube Augmented Reality Project, Presence, 11 (2002).
Windows on the World: 2D Windows for 3D Augmented Reality, in 16. B. S HNEIDERMAN, Designing the User Interface, Addison-Wesley
ACM Symposium on User Interface Software and Technology, Publishing, 1997.
pp. 145–155.
8