=Paper=
{{Paper
|id=None
|storemode=property
|title=Spreadsheets: from data interfaces to knowledge interfaces
|pdfUrl=https://ceur-ws.org/Vol-1010/paper-17.pdf
|volume=Vol-1010
|dblpUrl=https://dblp.org/rec/conf/mkm/Kohlhase13
}}
==Spreadsheets: from data interfaces to knowledge interfaces==
https://ceur-ws.org/Vol-1010/paper-17.pdf
Spreadsheets:
From Data Interfaces to Knowledge Interfaces
Andrea Kohlhase
Jacobs University Bremen
Abstract. Documents of type “spreadsheet” are considered user interfaces to nu-
meric data as they allow authors to create, modify and display these data in dis-
tinct layouts like tables or diagrams and readers to interpret them. We tend to be-
lieve that enhancing software semantically means that we are lifting its value. In
particular, if we enhance spreadsheets semantically can we lift their data interface
status to a knowledge interface status? We used the repertory grid methodology
to conduct a study on the difference between spreadsheets and spreadsheets se-
mantically enhanced with the SACHS extension. Our research shows that, indeed,
from the perspective of users adding semantics turns spreadsheets into knowledge
interfaces.
1 Introduction
The document type “spreadsheet” is indeed very successful: tens of millions profes-
sionals and managers create hundreds of millions of spreadsheets according to [15].
Spreadsheet documents are used to create, modify, and visualize numeric business and
science data, therefore they are mathematical user interfaces. Their complexity and im-
pact increased at the same time: this intensity yields wide-impact errors on the data level
(up to 90% [15], see also [17]) and on the apprehension level (e.g. [16]). To address this
problem, in the past we have extended spreadsheets semantically with the “SACHS (sx)”
system [12, 13].
In this paper, we ask what difference SACHS really makes: Do people perceive a
difference when offered SACHS functionality in spreadsheets1 ?
To better understand what users of spreadsheets perceive commonly as information
units, what meaning users assign to these units and what meaning users assign to the
additional SACHS information units, and finally, how users discriminate between them,
we conducted a study using the Repertory Grid Interview (RGI) Technique [11, 7,
10]. RGI explores personal constructs, i.e., how persons perceive and understand the
world around them. It has been used as a usability/user experience method to research
users’ personal constructs when interacting with software artifacts (see [9, 19, 8, 4,
21] for examples). RGI is a semi-empirical method that has the grand advantage of
not depending on a high amount of study subjects and nevertheless delivering valuable
insights into the perception of users.
1
At this point we still use the terms “spreadsheets” and “spreadsheet application” unspecifically,
because the concepts involved will only be clarified by the research reported on in this paper.
As a pre-study we used a smaller RGI to elicit which “semantic objects”, i.e., mean-
ingful objects, are commonly perceived by users in spreadsheets. From these we iden-
tified those semantic objects, which were considered information objects. For our main
study we added information units provided by SACHS. Besides getting a better grasp
on human-spreadsheet interaction in general, we were specifically interested to find out
which of these external information objects were perceived to deliver similar or differ-
ent information compared to traditional spreadsheet information objects.
First we present the setup of our repertory grid interviews. Then we analyze the
RGI and interpret the results. Finally, we conclude by drawing conclusions from our
RGI study.
2 The RGI Study
The aim of the study is a better understanding of information quality within a spread-
sheet and whether existing applications already cover the information offerings of the
SACHS extension. If such additional information were to be perceived by readers at all,
we were especially interested in what ways these new qualities would be perceived.
A repertory grid is a grid consisting of “elements”, i.e., the objects under consid-
eration, and “constructs”, i.e., pairs of antithetical properties that separate elements.
The constructs serve as a bipolar dimension on which the elements are evaluated. As
the property elicited first in a construct is the more salient one, RGI calls it the “implicit
pole” and the other one emerging in the reflection of the dimension of comparison the
“emergent pole”.
Elements as well as constructs can be elicited by the test persons themselves or can
be provided by the interviewer. Comparison of multiple repertory grids is simplified
if the individual ratings are given on a fixed set of elements or constructs, but a free
elicitation explores the perception space. For our main RGI we decided to fix the set
of elements to be “common and additional information objects in spreadsheets”, but to
elicit individual constructs to better understand the information space.
In a first RGI we extracted spreadsheet elements that are considered common in-
formation objects for the main RGI. Then we selected six additional spreadsheet infor-
mation objects from SACHS offerings, that are not contained in a common setup with
spreadsheets. We used both element sets together to collect personal constructs used to
evaluate these elements. Concretely, we asked the interviewees to tell us in what ways
the selected elements considered as information objects were similar and different (ac-
cording to traditional RGI).
2.1 Common Information Objects in Spreadsheets
We asked users to mark those elements from their self-created list of elements in the
first RGI study, which they consider to be “information objects”. We explained that we
understand information objects to be objects carrying information identifiable to the
reader.
Even though the resulting principal components given by a factor analysis and an
accordingly weighed clustering were interesting per se, they wouldn’t give us a gener-
2
ally valid assessment, as the constructs were not directly comparable. But we found six
information elements in this small RGI to be consistently listed:
Title A phrase describing the content of the spreadsheet
Headers A (short) phrase supporting the interpretation of values of a
regionally close range of cells (e.g. a column header)
Legends A list of content properties and resp. layouts (as in a map
legend)
Values The content of a cell container
Formulae A computational rule that yields a cell value
(sx:)Color The use of color hinting at additional information
Coding
Furthermore, two subjects also identified:
Tables A possibly multidimensional homogenous structural layout
of cells, that is perceived as an object of its own
Note that “diagrams” are missing, which may be due to the fact, that they were not part
of our standard spreadsheet example.
We selected these 7 information objects of common spreadsheets as part of the set
of elements for the main RGI.
2.2 SACHS Information Objects in Spreadsheets
The SACHS system contains semantic spreadsheet-related information not usually avail-
able to spreadsheet users. It aims at providing user assistance for spreadsheet users
based on a background ontology. As “cells” are the important semantic objects in
spreadsheets, SACHS acts cell-oriented.
In a spreadsheet like the one
in Fig. 1 a user clicks for exam-
ple on a cell that contains “1,878”
as information of Values. Com-
mon information objects tell the
user about the context (e.g. by Ti-
tle “Profit and Loss Statement”,
Headers “Profits” and “1986”,
or Legends “in Millions”). But
what if the user doesn’t under-
stand how “Profits” are calculated?
When using SACHS this cell might
be linked to a concept in a back- Fig. 1. A Simple Spreadsheet after [20]
ground ontology that covers the
domain knowledge of this partic-
ular spreadsheet. If the user likes to retrieve this linked concept from the ontology, then
she can do so by opting for a “look-up” option provided by SACHS and by selecting the
wanted cell. A pop-up close to the selected cell will appear with this additional infor-
mation — e.g. “A profit is the difference between revenues and expenses.” — together
with the header information “Profits [1986]”.
3
For this study, we extracted the following information objects that seemed to carry
additional information benefits compared to the common information objects:
sx:- A local look-up (data and text) of relevant information for
Localized cells on a by-cell-click basis
Info
sx:Functio- A local border indicating all cells functionally associated to
nal Block the currently selected cell
sx:Dependen- An overview graph (in a different window) of concepts show-
cy Graph ing on which the corresponding (selected) cell is ontologi-
cally dependent
sx:Relatio- An arrow indicating a dependency relation between concepts
nal Arrows in sx:Dependency Graph
sx:Concept A node in sx:Dependency Graph representing a depen-
Nodes dent subconcept, that additionally serves as a link to corre-
sponding spreadsheet cells
2.3 Data
For our study we interviewed 14 people, of which 10 were male and 4 female. The
age mean was 29,3 years. Participants reported an average of 8.2 construct pairs (SD
= 1.4) ranging between 5 and 11 pairs. 5 subjects were familiar with authoring spread-
sheets, the other 9 only had occasional contact. The rating scale for the 115 elicited
constructs was essentially binary: it consisted of -1,0,1 but the interviewees were only
told about their option to use “0” as a rating when they otherwise would have discarded
the construct in question as inapplicable.
We performed a Generalized Procrustes Analysis, as it can be used when data “have
arisen from one type of scaling of the same stimuli as perceived by different individuals” [5,
p. 33]. In particular, in our RGI we can compare the individual natural language con-
structs rated on our fixed set of information objects. We follow Grice’s example pro-
cedure for a Generalized Procrustes Example [6]. In particular, after having produced
a “consensus grid”, i.e., a best fit grid for a number of grids that are equal in one di-
mension but not in the other, we conducted a Principal Components Analysis (PCA)
on it yielding components {P Ci=1,...,11 }. The first two components explain ca. 56% of
the variance in the data, the first three even 71%. Hereafter, a Multiple Group Com-
ponents Analysis was performed, which allows to map the specific construct/element
distribution into the PCA results.2 .
3 Analysis and Interpretation
We now give an overview of element clusters we identified and the outcome of the
reconstruction of elements/constructs via the Multiple Group Components Analysis of
all elicited repertory grids followed by an interpretative discussion of the findings.
2
The original subject and computed data are available at http://kwarc.info/ako/
ProcrustesAnalysis
4
We focused each repertory grid by swapping the construct poles to optimize the
amount of applicable poles for the set of common spreadsheet information objects.
This way we could identify the characteristic construct poles for this and the remnant
element set and their pole distribution.
For the analysis of multiple repertory grids we found the “Idiogrid”3 tools of analy-
sis most helpful. Especially the availability of the Generalized Procrustes Analysis [5]
resulting in a consensus grid, yielded a valuable input for other analyses. The constructs
of the consensus grid are created based on the underlying statistical analysis, thus, they
are an artificial gold standard for real elicited constructs.
In Fig. 3 we can see the outcome of the Multiple Group Components Analysis
for P C1 and P C2 (with only the more salient constructs, in particular with 0.85%
suppression of labels) run by Idiogrid. Emergent Poles are marked by a “(-)” prefix.
3.1 Elements
Let us first look at the elements themselves. As the set of elements were fixed in this
RGI, we could build the
“concatenated grid” con-
sisting of the given ele-
ments and their ratings on
all elicited constructs. For
this concatenated grid we
ran a cluster analyis in
OpenRepGrid4 . Its dendro-
gram visualization (to be
seen in Fig. 2) identifies Fig. 2. Element Cluster Dendrogram for the Concatenated Grid
three clusters. Note that
these clusters can also be easily spotted in Fig. 3. They can be described as follows:
Local Cluster (L) The first cluster mainly contains the elements in quadrant II and III
of Fig. 3. Concretely it consists of the elements Headers, Title, Legends,
sx:Localized Info, Values, and Formulae. This cluster contains all lo-
cal spreadsheet information objects — those elements whose content depend on
their position. Therefore, we can categorize this cluster as the locally perceived in-
formation objects group or in short the “local cluster”. Headers, Title, and
Legends build a nested cluster as well as Values together with Formulae.
Both are linked in the main cluster via sx:Localized Info.
Visual Cluster (V) The second group contains all elements in the first quadrant of
Fig. 3, specifically the elements (sx:)Color Coding, sx:Functional Block,
and Tables. As all of them communicate information visually, we call it the “vi-
sual cluster”.
Meta Cluster (M) In quadrant IV of Fig. 3 we find the remaining objects: sx:Rela-
tional Arrows, sx:Dependency Graph, and sx:Concept Nodes. The
3
http://www.idiogrid.com/
4
http://www.openrepgrid.uni-bremen.de/wiki
5
most salient pole nearest this cluster that does not refer to their property of being
external to MS Excel reads “meta level”, therefore we call this group the “meta
cluster”.
Fig. 3. Multiple Group Components Analysis for P C1 and P C2
3.2 Constructs
To approximate the meaning of the principal components, we looked at actual, similar
constructs, especially at the more salient ones close to the axes in Fig. 3. Then we tried
to find categories that can serve as common denominator constructs. As this content
analysis was qualitative, the reliability was ensured by following the procedure given
in [10, 155ff.].
Principal Component P C1 The constructs coming closest to the first principal com-
ponent (depicted by the horizontal axis in Fig. 3) are:
6
Implicit Pole Emergent Pole
“Knowledge Tool” “Data Tool”
meta level object level
UX outside Excel UX in Excel
implicit info explicit info
relevant for analysis relevant for understanding
represents relational info represents contextual info
Here, the black entries are more salient than the gray ones (cited for clarification). The
main property which the poles of this component share is that they classify the elements
according to how they are used by a spreadsheet user, i.e., according to their purpose.
Probst et al. suggested in [18] a knowledge management model positing that glyphs,
data, information, and knowledge can be seen as stages of a pipeline as in Fig. 4. This
Fig. 4. Knowledge Management Model after [18]
model differentiates what we have simply called “information” so far into four distinct
traits. Glyphs are just a set of characters without any structure, combined with a syn-
tax they become data, additionally enriched by context they become information, and
finally, they turn into knowledge if a semantic net or a global context is present.
Even though Brown & Duguid don’t speak of knowledge as such in their model,
they also suggest to go beyond information in [1]. What this knowledge might be for
knowledge workers is discussed in [2]. In the following we use the four distinct traits
of “information” established by Probst et al.. We will use the term “information” in its
generic (naive) form as before.
For the descriptions of the implicit poles this means that the information objects
were used as “Knowledge Tools”, i.e., that the communicated information aims at the
knowledge level. In contrast, the descriptions of the emergent poles indicate that the
according information objects are being used as “Data Tools”: they handle informa-
tion as pure data. We can say that in this categorial construct a macro perspective on
information objects is captured.
Principal Component P C2 The second component (vertical in Fig. 3) can be de-
scribed best by the following constructs:
7
Implicit Pole Emergent Pole
“Represented Data” “Implicit Knowledge”
visual information cognitive information
project-specific meaning globally defined meaning
When we looked for properties which the implicit resp. emergent poles of the second
component shared, we were surprised to find the elements of Probst et al.’s knowledge
management model again. Here, a micro perspective is taken up in the interviewees’
constructs. They target what the information object itself uses and makes use of, that is,
the categorial construct is concerned with what the content of the elements pertains and
how it does so. The implicit poles we categorized as “Represented Data” whereas the
emergent poles could be summarized as “Implicit Knowledge”.
Principal Component P C3 To also touch the third principal component we analyze
the constructs closest to P C3 in the biplot of P C1 and P C3 resp. P C2 and P C3 , which
are at suppression of labels at 0.78:
Implicit Pole Emergent Pole
“Creator” “User”
computation explanation
data processing semantics
Both, “computation” and “data processing” are concerned with the spreadsheet as in-
telligent application, but also with the author who uses the application to handle her
input data. Thus, the implicit pole can be summarized under “Creator” as either appli-
cation or author produces the spreadsheet by computation and by data processing. On
the other hand, “explanation” and “semantics” apparently refer to the meaning of the
document. To contrast it with the implicit pole we therefore named the emergent pole
“User”, which is justified since meaning of software artifacts can only be evaluated and
experienced by its users. Please note that creators are applications as well as authors,
whereas users can be authors as well as readers.
3.3 Discussion
Let us now combine the findings about the elements and the constructs. Note that the
term “versus” in the subtitles does not signify opposition, but is supposed to enhance
users’ distinct context experiences implied by our subjects’ construct elicitations.
Common Spreadsheets versus SACHS Extension If we look at the element space
with the coordinate system slightly shifted according to the construct vectors nearest to
the axes, then all common spreadsheet information objects are on the left side and all
SACHS ones are on the right side (except for the hybrid (sx:)Color Coding which
is located very close to the separating axis). The construct “object level - meta level”
is one of the constructs for which rating of the elements differ: common spreadsheet
objects are considered to be at the object level, whereas SACHS objects are considered
to be at the meta level. Similar constructs directly distinguish standard objects from
8
the additional ones (e.g. “UX in Excel - UX outside Excel” or even “Excel - SACHS”),
thus the term “meta” could indicate a mere “going beyond”. Therefore we looked at
more distinguishing constructs nearby by looking at Fig. 3 from within Idiogrid with
suppression 0.70 and found e.g. the following:
Implicit Pole Emergent Pole
“Common Spreadsheet” “SACHS”
(−→ “Data Tools”, P C1 ) (−→ “Knowledge Tools”, P C1 )
structures info structures meaning
represents contextual info represents relational info
explicit info implicit info
relevant for understanding relevant for analysis
not formal info dependency info
global context info interpretative info
visual interpretation help structural interpretation help
Thus, we can conclude that our subjects perceived a qualitative difference between com-
mon spreadsheet- and SACHS information objects, where former are concerned with
supporting the data interface qualities, i.e., on the object level, whereas latter aim at
providing a global context as interpretation help for the data interface, that is on the
meta level.
Fig. 5. Interpretation of Fig. 3 and Fig. 2
9
Spreadsheet Player versus Spreadsheet Document At first glance it was troubling
that the properties of the first and the second principal component both concerned “data”
and “knowledge”, as they were supposed to stress differences between elements. But
a closer look (presented subsequently) reveals that the macro perspective “Knowledge
Tool — Data Tool” rates the elements as information objects of the player whereas a
view from the micro perspective “Represented Data — Implicit Knowledge” consid-
ers the elements as information objects of the document. A spreadsheet player is the
application (e.g. MS Excel) which acts as a smart interface between the spreadsheet
workbook and the human reader. This player e.g. renders the workbook on screen, inter-
prets cells as value containers, runs computations for “cells”, and so on. In a sense it is
an interpreter/compiler for input data. This input is given by a spreadsheet document.
The document is an intermediate data layer as it presents a prepared view on the real
data e.g. stemming from a database. Note that players “play” such a pre-compilation
of data, whereas documents “document” (= “to furnish documentary evidence of, to provide
with factual or substantial support for statements made or a hypothesis proposed” [14]) data.
The information objects perceived as “Data Tools” are Title, Headers, For-
mulae, Legends, and Values. They either describe data coming from some input
source or work on such. They serve the purpose of enriching these input data with con-
text, thus elevating them into information. The spreadsheet player supports the transi-
tion from data to information e.g. by simply distinguishing between various cell formats
(like “text” for Headers or “date” for Values) and by computational tools for For-
mulae. As “Data Tools” these information objects thus belong to the document player.
With the same reasoning the elements perceived as “Knowledge Tools” are informa-
tion objects viewed as spreadsheet player objects (turning data into knowledge). For
example, if we look at the visual cluster V from the player perspective, they serve as
“Knowledge Tools” as they help the user to elevate information into knowledge.
In Fig. 5 we visualized the element distribution according to the Principal Compo-
nent Analysis as in Fig. 3. The only difference is that we enhanced the distance between
the element clusters L, V, and M determined in Fig. 2 to allow for a horizontal grid
to depict the distinctions discussed in the following. In light of the discussion above
we interpret the first PCA component dimension as spreadsheet player dimension. In
particular, we used the knowledge management model components glyph, data, infor-
mation, and knowledge (Fig. 4) as scale for this axis. The exact location of these on the
x-axis of Fig. 5 was determined by observing the specific transformation function of the
spreadsheet player’s information objects in terms of the model (see discussion above).
Now, if we reconsider that a spreadsheet player is commonly considered as an in-
terface for input data, we can clearly see that there are feature groups on the way from
non-interpretable glyph to desirable knowledge e.g. as foundation for financial deci-
sions based on the data.
Another aspect under which information objects are perceived is given by the sec-
ond principal component construct “Represented Data — Implicit Knowledge”. This
categorial construct deals with the spreadsheet as a document, with which available
knowledge is transformed into distributable data. According to this axis the elements in
Fig.3 are spread as information objects of the document, because the elements are rated
with respect to their communication capacities. For instance, Tables are the most ex-
10
pressive in terms of representing data, whereas the elements of the meta cluster M are
the most expressive in representing implicit knowledge.
The perceived distinction between spreadsheet player and spreadsheet document is
even more remarkable as clarification discussions with some of the interviewees re-
vealed that this distinction was not an explicit one. Prompted to differentiate between
the two, the subjects were surprised and not able to distinguish the concepts continu-
ously.
Spreadsheet Author versus Spreadsheet Reader In Fig. 5 we took the element clus-
ters from Fig. 2 into account and stressed their clustering according to the second prin-
cipal component P C2 , that refers to information objects of the document. Now we can
interpret the distribution of information objects as document features as a communi-
cation timeline starting from retrieving available knowledge up to compiling it into a
message. Note that the individual clusters thus show an ordering. Depending on what
the document author wants to stress, she makes use of the offered information objects.
From this we can cautiously assume that the distinction between a spreadsheet’s author
and a spreadsheet’s reader is also perceived. To express the author’s intention the de-
grees of liberty are highest for the elements of the visual cluster V, then follows the
local cluster L and are lowest for the meta cluster M. A reader on the other hand can
abstract from the author’s design by using more and more information features along
the data interface axis (Fig. 5). We suspect the underlying reason for the controversial
location of Headers and Title in Fig. 3 with respect to this differentiation to be that
the content of headers is implicitly already specified by the choice of data to be shown,
whereas the title is more general and can thus be more freely chosen by the author.
As the document view is strongly correlated with a spreadsheet author’s view, and
the player view correlates with the reader’s perspective, we can reformulate the di-
chotomy of “document versus player” as “author versus reader”. This gives us an inter-
esting second view on the phenomena involved.
Local Objects versus Global Objects in Spreadsheets As we have seen above our
subjects perceived the spreadsheet application objects as different from the SACHS in-
formation objects. But it is noticeable that sx:Functional Block and especially
sx:Localized Info are close to the separating line in Fig. 3. Only these elements
together with (sx:)Color Coding (which can also be identified as an MS Excel
information object) are also local to where the (user-)action is in the application, and by
that perceived to belong to the application. Even though sx:Dependency Graph,
sx:Relational Arrows, and sx:Concept Nodes are also triggered by local
actions of the user, they are far more distant. This is a clear indication that this attribute
makes a difference for a spreadsheet user. Together with the dichotomy of “author ver-
sus reader” above, we can interpret that the position of an information object relative to
the spreadsheet is relevant to readers.
11
4 Conclusion
In this paper we presented a repertory grid study that investigates properties of informa-
tion objects in spreadsheets in order to better understand the information qualities given
by the SACHS extension for human-spreadsheet interaction. We found four different
evaluation schemes for information objects in spreadsheets (see Fig. 6).
Fig. 6. Quality Dimensions of Information Objects (in Spreadsheets)
The strongest quality dimension is the distinction between spreadsheet player and
spreadsheet document. In particular, information is experienced differently, when asso-
ciated with the spreadsheet player or with the spreadsheet document. A good example
is the use of Formulae: as a player object the resulting values are considered fac-
tual data, as a document object these values are considered to contain implicit knowl-
edge. We conjecture that the distinction between a document player and its document
is strongly perceived by users in general, even though this study shows this only for
spreadsheet applications. If so, then this has extensive consequences for future design
options. For instance, we can distinguish between a presentation player like MS Pow-
erPoint and a *.ppt file. Most services though support only document-independent fea-
tures and miss out on the document-dependent opportunities. “Grouping of shapes” for
example is a nice feature to ease copying or moving of a set of shapes. The fact that they
“belong together” is only appreciated on the object level, whereas it very often targets
a meta level as well: their semantics.
Another extracted dimension is the distinction between information objects in com-
mon spreadsheet applications and ones in the semantic spreadsheet extension SACHS.
This is especially pleasing, as we conclude that such additional semantic services are
12
new wrt. common spreadsheet features. In particular, the analysis of the RGI shows that
features of the semantic extension do not duplicate already existing features of spread-
sheet players. They are perceived by users as additional services targeting implicit gen-
eral information provided by the player and specific information anchored in the docu-
ment. They are conceived as features that transform information into knowledge. This
doesn’t prove the usability of SACHS itself, but it opens a new area of spreadsheet user
requirements. Note that this can be generalized beyond spreadsheets as well. For in-
stance, the sx:Dependency Graph can also be used for lectures in a presentation
application like OO Impress or in a CAD application like Autodesk Inventor.
The distinction of local and non-local positioning of information objects is also of
consequence for general application extensions. For instance, this is made use of in [3].
Last, the perceived differentiation of spreadsheet users into authors and readers al-
lows a much better fine-tuning of services. Even though the existence of both groups
has been recognized, the interface design for players has not yet seriously taken this dis-
tinction into account. A *.ppt document e.g. serves divergent needs of a lecturer versus
a student, but basically only former are paid attention to.
All in all, this work has shown that semantic extensions of spreadsheets are per-
ceived as offering distinct functionalities, that turn spreadsheets from data interfaces to
knowledge interfaces.
Acknowledgement This work has been funded by the German Research Council under
grant KO-2484-10-1.
References
[1] John Seely Brown and Paul Duguid. The Social Life of Information. Harvard
Business School Press, 2000.
[2] Thomas H. Davenport and Laurence Prusak. Working Knowledge. 2000th ed.
Harvard Business School Press, 1998.
[3] Catalin David et al. “Semantic Alliance: A Framework for Semantic Allies”. In:
Intelligent Computer Mathematics. Ed. by Johan Jeuring et al. Vol. 7362. Lecture
Notes in Computer Science. Springer Berlin Heidelberg, 2012, pp. 49–64. ISBN:
978-3-642-31373-8. DOI: 10 . 1007 / 978 - 3 - 642 - 31374 - 5 _ 4. URL:
http://dx.doi.org/10.1007/978-3-642-31374-5_4.
[4] Ari Ginsberg. “CONSTRUING THE BUSINESS PORTFOLIO: A COGNITIVE
MODEL OF DIVERSIFICATION[1]”. In: Journal of Management Studies 26.4
(1989), pp. 417–438. ISSN: 1467-6486.
[5] J. Gower. “Generalized procrustes analysis”. In: Psychometrika 40 (1 1975),
pp. 33–51. ISSN: 0033-3123.
[6] James W. Grice. Generalized Procrustes Analysis Example with Annotation. Manuscript
at http://psychology.okstate.edu/faculty/jgrice/personalitylab/
GPA_Idiogrid_Example.pdf. 2007.
13
[7] Marc Hassenzahl and Rainer Wessler. “Capturing Design Space From a User
Perspective: The Repertory Grid Technique Revisited”. In: International Journal
of Human-Computer Interaction. 3rd ser. 12 (2000), pp. 441–459. ISSN: 1044-
7318. URL: http://www.informaworld.com/10.1207/S15327590IJHC1203&
4_13.
[8] Stephanie Heidecker and Marc Hassenzahl. “Eine gruppenspezifische Repertory
Grid Analyse der wahrgenommenen Attraktivität von Universitätswebsites”. In:
Mensch & Computer. Ed. by Tom Gross. Oldenbourg Verlag, 2007, pp. 129–138.
[9] Morten Hertzum and Torkil Clemmensen. “How do usability professionals con-
strue usability?” In: Int. J. Hum.-Comput. Stud. 70.1 (2012), pp. 26–42.
[10] Devi Jankowicz. The Easy Guide to Repertory Grids. Wiley, 2003. ISBN: 0470854049.
[11] George Kelly. “International Handbook of Personal Construct Technology”. In:
John Wiley & Sons, 2003. Chap. A Brief Introduction to Personal Construct
Theory, pp. 3–20.
[12] Andrea Kohlhase. “Towards User Assistance for Documents via Interactional
Semantic Technology”. In: KI 2010: Advances in Artificial Intelligence. Ed. by
Rüdiger Dillmann et al. LNAI 6359. Karlsruhe, Germany, 2010, pp. 107–115.
[13] Andrea Kohlhase and Michael Kohlhase. “Compensating the Computational Bias
of Spreadsheets with MKM Techniques”. In: MKM/Calculemus Proceedings.
Ed. by Jacques Carette et al. LNAI 5625. Springer Verlag, July 2009, pp. 357–
372. ISBN: 978-3-642-02613-3. URL: http://kwarc.info/kohlhase/
papers/mkm09-sachs.pdf.
[14] Merriam-Webster. documentz — Merriam-Webster. [Online; accessed on 2012-
09-18]. 2012. URL: {\url{http : / / www . merriam - webster . com /
dictionary/document}}.
[15] Raymond R. Panko. “Spreadsheet Errors: What We Know. What We Think We
Can Do.” In: Symp. of the European Spreadsheet Risks Interest Group (EuSpRIG
2000). 2000.
[16] Stephen G. Powell, Kenneth R. Baker, and Barry Lawson. “A critical review of
the literature on spreadsheet errors”. In: Decision Support Systems 46.1 (2008),
pp. 128–138.
[17] Stephen G. Powell, Barry Lawson, and Kenneth R. Baker. “Impact of Errors in
Operational Spreadsheets”. In: CoRR abs/0801.0715 (2008).
[18] G. Probst, St. Raub, and Kai Romhardt. Wissen managen. 4 (2003). Gabler Ver-
lag, 1997.
[19] Felix B. Tan and M. Gordon Hunter. “The Repertory Grid Technique: A Method
for the Study of Cognition in Information Systems”. English. In: MIS Quarterly
26.1 (2002), pp. 39–57. ISSN: 02767783.
[20] Terry Winograd. “The Spreadsheet”. In: Bringing Design to Software. Ed. by
Terry Winograd et al. Addison-Wesley, 2006, pp. 228–231.
[21] Hsin-Hung Wu and Jiunn-I Shieh. “Applying repertory grids technique for knowl-
edge elicitation in quality function deployment”. In: Quality Quantity 44 (6
2010), pp. 1139–1149. ISSN: 0033-5177.
14