=Paper=
{{Paper
|id=None
|storemode=property
|title=A Model of Relevance for Reuse-Driven Media Retrieval
|pdfUrl=https://ceur-ws.org/Vol-680/paper_4.pdf
|volume=Vol-680
}}
==A Model of Relevance for Reuse-Driven Media Retrieval==
https://ceur-ws.org/Vol-680/paper_4.pdf
A Model of Relevance for Reuse-Driven Media
Retrieval
Tobias Bürger
Salzburg Research, Salzburg, Austria
tobias.buerger@salzburgresearch.at
Abstract. An often criticized fact in multimedia retrieval is, that user
needs are not appropriately taken into account. Both knowledge about
how end users search and how they assess the relevance of retrieved mul-
timedia objects can provide invaluable hints for the design of multimedia
retrieval systems. This paper reports on an end user study on multime-
dia retrieval behavior of media professionals who intend to reuse me-
dia objects in media productions. We present a conceptual model which
contains empirically validated information on how users in the media
production domain search for content to be reused and how relevance is
assessed by them. Finally we sketch how this information can be used to
improve ranking of media objects in multi-faceted retrieval scenarios.
1 Introduction
The amount of multimedia content available on the Web and the amount of
professionally produced content stored in local or commercial databases grows
every day: While there is a steady growth of professionally produced content
available on the Web, a continuous blurred shift happens between consumers
and producers of content, which share huge amounts of user generated content.
This ever growing amount of content offers a great potential for reuse.
Reuse of multimedia content, i.e., every kind of use of content which has
been used in a certain context before, is an ongoing challenge and is mostly
not very well supported by existing tools and approaches. Supporting reuse can
however provide significant improvements in the way how content is created,
including increased quality and consistency, long-term reduced time and costs
for development, maintenance or adoption for changing needs [25]. As our recent
observations in the domain of media production reveal, only approximately 30
percent of the produced content is based on already existing content. We further-
more revealed barriers leading to this low figure which include reasons such as
“relevat content cannot be found”, that “it is sometimes faster to build content
from scratch”, that “content is not adaptable to new situations”, or that “the
legal situation is either unclear or does not allow reuse”.
One of the identified barriers of reuse includes the problem of findability of
content which maps the problem of reusability to the solution space of multime-
dia retrieval. One ongoing problem there is the gap between the research done
2
and the practical end user needs in different contexts as many approaches take
a system-centric approach focusing on technical aspects of multimedia indexing
and retrieval [11] and lack a theoretical background of the characteristics of users
and their needs for the design of these systems [24]: In order to support efficient
retrieval, matches to a request have to be presented in an appropriate order,
minimizing the distance between actual features of the content and expected
features by the user.
To bridge the aforementioned gap, user-oriented studies were conducted which
analyzed the indexing practices and retrieval needs of typical end users. Some of
these studies resulted in analytic models which formalized the characteristics of
user requests and search patterns of these users (cf. Section 2). While the main
aim of these studies was to conceptualize and bridge the Semantic Gap [4, 5],
the judgement of relevance for the selection of media objects has so far not been
researched to a great extent. In order to overcome this situation, we present a
conceptual model which contains empirically validated information on how users
in the professional media production domain search for content to be reused and
how relevance is assessed by them.
In this paper we examined a typical retrievel task in the studied environ-
ment: A media professional is engaged in a design task and intends to search for
images to reuse in his current production. He starts with formulating his needs
in an image request and receives a result set of images. After that, he checks
the topicality of the images in the retrieved result set and starts browsing. If
either the topicality of the returned images does not match his needs or if he
is unsatisfied with the results investigated during browsing, he reformulates his
query. Otherwise he applies his relevance criteria and finally selects an image
which he uses in his design task.
Understanding how and why users search for and select multimedia content
to be reused can provide invaluable hints for the design of multimedia retrieval
systems. Therefore the research leading to this paper aimed to address the fol-
lowing research questions:
1. “Which factors do users use to search for reusable media objects?” and
2. “Which relevance criteria do users apply when searching for media objects
to be reused?”
In order to answer these questions, we built a basic model containing factors
used in search and relevance assessment. To do so we analyzed prior literature
and conducted interviews with design professionals. Subsequently we empirically
validated the model through an end user survey and assessed the validity and
importance of the factors in both tasks.
The remainder of this paper is structured as follows: Section 2 presents the
background and motivation for the model. Subsequently the model is discussed
in detail in Section 3: We present insights from prior literature and the basic
mode. Section 4 details the validation of the model and presents the results of
the conducted survey. Finally, Section 5 concludes the paper.
3
2 Research Background and Motivation
Since the 1960s research has been reported which analyzed the indexing practices
and retrieval needs of typical end users. The work in this area can be divided into
conceptual frameworks of image indexing (cf. [5, 9, 14, 22, 27]), which are mainly
situated in cognitive psychology and models of user’s multimedia retrieval needs
(cf. [1, 2, 10, 15, 16, 19, 29]). The intention of these conceptual frameworks was
to provide groundings for the manual and automatic image indexing and the
description of the semantics of multimedia content in general and images in
particular.
The earliest model was provided by Panofsky [22] who recognized three types
of subject matter, for instance, primary subject matter which requires no inter-
pretative skills, secondary subject matter which necessitates an interpretation,
and tertiary subject matter (“iconology”) demanding high-level semantic infer-
encing done by the user. Subsequent work by Shatford [27] simplified the three
levels of Panofsky into generic, specific and abstract. Additionally Shatford intro-
duced the distinction between “of-ness” and “aboutness” of a picture. A simpler
model was provided by Greisdorf [9] who recognized three levels which corre-
spond to visual primitives (e.g., color or shape), logical features (e.g., objects or
events) and inductive interpretation (e.g., abstract features). Joergensen et al.
have further refined the model by Shatford which resulted in the so-called visual
indexing pyramid [14]. A newer model by Enser et al. builds on Joergensen’s
notion of semantic facets of images and furthermore takes the combination of
semantic content of an image and its context into account [5].
Besides the development of these models, studies were conducted which inves-
tigated user retrieval needs. Their intention was to inform other research strands
which type of semantics can be extracted from multimedia content. Most of these
studies revealed, that a user is typically interested in high-level semantics which
are hard to derive based on automated approaches and which are often highly
subjective. An analysis of early studies in this area by Jörgensen revealed a wide
variation in subject foci and terminological speciality and also that the majority
of requests were for specific events or objects, especially for specific, named fea-
tures [16]. This observation was also made in [1, 19]. Other studies reported an
emphasis on generic or affective visual features (cf. [2, 10, 15]). Validations and
comparisons of these studies can be found in [1] and [29].
Especially in multimedia retrieval, uses and needs vary considerably, as media
objects are used in a variety of domains (e.g., media production, art, journalism,
or medicine) for different purposes. Furthermore, needs of professionals and needs
of end users are in many cases different: End users are motivated by leisure,
while professionals search for images for inspiration, reuse, or other reasons. As
relevance differs considerable based on the situation of the user and his needs,
domain specific investigations have been made: User needs in domain specific
collections and for specific user groups have been conducted, e.g. for web images
(cf. [6, 7, 15, 23]), for historical images (cf. [1, 2]), for medical images (cf. [17]),
or for image retrieval in a journalistic context (cf. [11, 12, 18, 19, 29]). User needs
of media professionals such as graphic-, or game- designers, and especially the
4
influence of the intention to reuse, were, however, rather unexplored up till now.
Our aim was therefore to develop a conceptual model that dimensions relevance
in reuse-driven multimedia retrieval in which people search for content to reuse.
3 A Model of Relevance in Media Reuse
This section presents a conceptual model which reflects factors which influence
the relevance of multimedia content for end users in the particular situation
in which they look for content to reuse. The model is based on insights from
existing literature, on motivation and barriers for reuse, and on user studies
which investigated multimedia retrieval in professional domains. Prior insights
were validated and supplemented with expert interviews conducted with media
professionals.
3.1 Insights from Existing Literature
Relevance is a central concept in information retrieval and there used as a mea-
sure for retrieval and to judge the effectiveness of an information system. Ingw-
ersen and Järvelin suggest that relevance is a multidimensional cognitive concept
whose meaning is largely dependent on searcher’s perceptions of information and
their own information need (cf. [26] as cited in [13]). While content features are
in most situations the most appropriate indicators for relevance, non-content
features of documents can give valuable hints, too. This is especially true for
multimedia retrieval in professional environments in which relevance is not only
based on topicality but also on visual, qualitative, situational and other con-
textual factors as reported in previous studies (cf. [1, 2, 15, 18, 19, 21, 28, 29]):
Markkula investigated retrieval of images in a journalistic context [19]. His ob-
servations clearly indicated the diversity of relevance criteria which were applied
and the situational nature of their relevance judgements. The primary criteria
which is applied by journalists to assess relevance is topicality. Secondary crite-
ria are technical properties, technical quality and biographical criteria. Images
which are technically good, are current or were not recently published are be-
ing considered as relevant in this domain. The cost of images has also been
identified as an important criterion. Even though expressive and also aesthetic
criteria, such as color and composition, were used for search by journalists, they
played the most important role in the final selection phase. The critical criteria
to reject or to accept an image depended on earlier selections: An image already
chosen for a page and nearby pages or used recently in a different or the same
newspaper restricted the possibility to use other similar images. According to
the journalists, the goal was to make the illustration of the page attractive, bal-
anced, and dynamic. This was achieved by using images of different types (e.g.,
horizontal and vertical photos, portraits, group photos, action, or themes) and
with different visual features.
Another study was conducted by Choi and Rassmussen who investigated
relevance criteria in image retrieval of historic art images [2]. They identified
5
nine relevance criteria together with 8 non-visual (descriptive) attributes for
highly relevant images:
– Time frame: The time period of the image.
– Accuracy: The image accurately presents what the user is looking for.
– Topicality: The image is related to the user’s task.
– Completeness: The image contains the necessary details.
– Accessibility: The availability of the image, as in the ease of obtaining the
image and the means by which image information can be accessed.
– Appeal of information: The image is interesting and appealing to the
user.
– Novelty: The image is new to the user.
– Suggestiveness: The image generates new ideas and insights for the user.
– Technical attributes: These attributes include mood, emotion, point of
view, or color.
Furthermore the following descriptive attributes were identified as being impor-
tant in the assessment of relevance in image retrieval in a historical art context:
creation date, notes, subject descriptors, the title of the image, the source repos-
itory, the source collection, the medium, and the name of the creator.
Eakins’ study done in 2004 investigated features used in image search but
not how these features affect relevance [3]. His results showed that despite of
topicality, technical quality is the most important criterion in search. Besides
that, he identified the following features as the most relevant:
– Low level features such as colour, texture or shape.
– Technical quality features such as sharpness.
– Semantic content containing general and specific semantic terms.
– Abstracted features such as contextual abstraction which refers to non-
visual information derived from the knowledge of the viewer, cultural ab-
straction which refers to aspects which can only be inferred based on a
cultural background, Emotional abstraction which refers to emotional re-
sponses triggered by the image, and technical abstraction which refers to
aspects requiring specific technical expertise to interpret.
– Metadata such as the image type (e.g., photographic, painting, or scan).
Othman was the first to investigate image retrieval in a creative media-related
context [21]. The relevance criteria she discovered for the domain of media pro-
duction were similar to the ones by Choi and Rasmussen [2] but with a different
mean importance of each criteria: Technical attributes were considered the most
important, followed by completeness and topicality. Furthermore relevant images
had to be qualified for processing and should not require any authentication.
Most images in her study involved analysis and image manipulation, and thus
the majority of users rated technical attributes as the most important relevance
criteria. Technical criteria included resolution, size, color, and dimension. Top-
icality and completeness ranked second and third which indicated that images
must be right on the topic and have all the objects specified. Time frame was
6
an important criterion for specific tasks. A further novel insight from her study
was that the images which were judged as relevant and met their intended use
ranged from one object in the image retrieved to the whole image itself.
The most recent study which reports on end user needs in image retrieval in a
journalistic context was published by Westman and Oittinen [29]. It featured 47
criteria for image selection which were partially based on insights from Markkula
[19]. The criteria were grouped along the following dimensions:
– Information and content (e.g., information content or story of the pho-
tograph)
– Visual and compositional features (e.g., visual features, composition, or
lighting)
– Technical features (e.g., technical error, sharpness, or physical size)
– Abstract and affective factors (e.g., movement and dynamicity, mood,
expression of the person)
– Metadata and associated information (e.g., recentness, source, or im-
portance)
– Publication context (e.g., compatibility with the headline, publication sec-
tion, or importance of the article)
– Workflow and other actors (e.g., timetable or possible print quality)
– Practices and feedback (e.g., image selection practices or feedback from
readers)
According to Westman’s and Oittinen’s studies, several types of criteria were
used in the relevance assessments made. Contextual factors (such as publishing
section or layout of the page) formed a selection frame for suitable images. Top-
icality was identified as a necessary but insufficient criterion for relevance, used
mostly as a starting point. Compositional and informational criteria followed in
later stages of the process. The final selection criteria were dynamic, activated by
comparisons of retrieved images and based on the characteristics and differences
between them such as dynamic elements or sharpness. Final selection criteria
also were preferential or reactive in some situations; selections were based on
personal impressions of images being, for instance, more interesting than others.
Furthermore several implicit criteria were employed in the image selection pro-
cess. Unless otherwise asked, the image retrieved was as recent as possible and,
if search is carried out across multiple archives, retrieval from the own archive
was preferred. Constraints such as price, previous publication, recentness and
presence of other images in a product also influenced the selection. Their re-
sults revealed that the most important relevance criteria were related to the
informational content of the image. Several abstract and affective criteria also
influenced the selection strongly. Least important were feedback and reactions
from others. Various factors related to the eventual publication context of the
image were considered important which means that often not the best matching
image according to a query was used but the one matching the context most. A
large number of individual criteria affected image selection strongly. Technical
factors were identified as not being as crucial as previously thought by Markkula
and Sormunen [19].
7
3.2 Basic Model
Fig. 1. A Conceptual Model for Reuse-Motivated Relevance Assessment
Our conceptual model, which is depicted in Figure 1, contains factors which
are typically used by media professionals for search and/or to assess relevance of
a media object. To create it, we refined and extended prior literature, validated
and supplemented it with context specific interviews:
1. First interviews with designers, art directors, and researchers with experience
in multimedia content creation and reuse were conducted.
2. Secondly, the qualitative data gathered from the expert interviews and prior
observations were analyzed by transcribing and coding into meaningful ex-
pressions which were then classified.
3. Thirdly the results were systematically analyzed according to statistical the-
ory.
4. Finally the analysis resulted in a cluster of themes, each containing a grouped
set of factors that influence reuse. All of these factors were constantly em-
phasized by media professionals.
The derived factors are arranged into clusters which are partly based on the cat-
egorization schema from Westman [29] which we further extended and tested in
our survey. The clusters are further arranged into intrinsic and extrinsic features
and environmental factors. The following clusters are part of the model:
– Intrinsic Features
1. Information and content (I) which includes features such as topical-
ity, completeness, or conveyed message.
2. Technical Quality (TQ) which contains features such as sharpness, or
technical error.
3. Technical Properties (TF) which contains intrinsic technical features
such as resolution or size.
4. Visual and compositional features (V) contain features such as
composition, color, angle, or other visual features.
– Extrinsic Features
5. Abstract and affective features (A) include expression, dynamicity,
eye catching ability, or mood
8
6. Provenance and bibliographic metadata
(PM) includes features capturing previous uses of the media object such
as popularity, recentness, previous publishing, etc.
7. Rights and licensing (RM) includes information regarding the rights
holder(s), permitted use, etc.
– Environmental Factors
8. Production workflow (PDW) captures features related to the overall
production and its requirements.
9. Processing workflow (PCW) bundles features regarding the actual
processing of the media objects, such as if it is adaptable or how it can
be processed.
10. Publication context (PC) refers to the concrete location into which
the media object should be integrated to and by that provides addi-
tional constraints such as available space, consistency with the layout, or
publishing history of the item to be reused.
11. Practices and feedback (PF) refers to work related factors such as
typical habits of the designer itself, typical guidelines from the company,
or social recommendations from colleagues, customers or experts.
Table 1 presents the dimensions investigated in the different models in different
domains as reported in the literature. In order to compare previous studies,
we mapped it in the clusters used by our model:M 1 refers to the model from
Markkula [19], M 2 to the model from Choi [2], M 3 to the model from Othman
[21], M 4 to the model from Westman [29], and MR to our model. An “x” means
that the category has been confirmed to be relevant in the domain investigated
in the respective model.
Category M1 [19] M2 [2] M3 [21] M4 [29] MR
Information and content (I) x x x x x
Visual and compositional Features (V) x x x x
Technical features (TF) x x
Technical quality (TQ) x x x x
Abstract and affective factors (A) x x x x x
Provenance and bibliographic Metadata (PM) x x x x x
Rights and licensing Metadata (RM) x x
Publication context (PC) x x x
Publication workflow (PUW) x x
Processing workflow (PRW) x x x x
Practices and Feedback (PF) x x
Table 1. Dimensions investigated in different relevance models
4 Validation
To test the proposed research model and to gain insights on the degree of influ-
ence of the different factors, we adopted the survey method for data collection,
9
validated it using statistical methods, and compared our insights to the results
derived from previous related work in this area.
4.1 Methodology and Data Collection
We conducted an end user survey to collect data and designed a questionnaire
reflecting the factors in order to assess the conceptual model. The resulting ques-
tionnaire was first tested in a small group of members of a media design online
forum. After analyzing the results from the test phase one item from Abstract
and affective factors has been dropped because it has not been used by any of
the participants. A further refined version was then sent as a self-administrated
questionnaire to 150 media design professionals in Europe. The data was col-
lected via an online survey in two languages (German and English)1 . The back
translation approach was applied in order to ensure consistency between both
language versions of the questionnaire [20]. The questionnaire consists of five
parts: The first part contains general questions regarding reuse such as how
much content is reused on a personal, company- and production-oriented level,
and asked for reasons and barriers for reuse. The second part contains questions
regarding factors used in search for media objects to reuse and the third part
contained questions regarding selection criteria of content for reuse. In parts two
and three the participants were asked to indicate the frequency of the use of
the factors in search and for the assessment of the relevance of a media object
on a five-point scale. The fourth part includes questions about the actual use
of reused content (e.g., if it is adapted, or used as is). The fifth part contains
concluding questions regarding the demographics of the participants.
31 responses which makes a response rate of 21 percent were returned, from
which two responses with incomplete data were eliminated from further analysis.
The gathered data reflects habits of media professionals spanning the domain
of print and Web design over game design to the design of learning material.
The majority of the participants had an experience from 3 − 5 years in their
domain (32.14%), followed by 25% which had more than 10 years of experience
and 21.34% which had 5 − 10 years of experience. The remaining respondents
had up to 3 years of experience.
In order to check the internal consistency of the model, it was assessed using
factor analysis [8]. Further structured relationships between the variables were
examined.
4.2 Survey Results
The gathered data revealed several interesting insights on barriers and motiva-
tions for reuse of media, how people search for and assess the relevance of media
objects in a particular situation (cf. Section 4.2 and 4.2), and how they finally
use the selected media objects (cf. Section 4.2).
1
The online questionnaire used in the survey is available at http://www.
tobiasbuerger.com/reusesurvey/
10
Factors Affecting Search In this section we provide answers to the question
”Which factors do users use to search for reusable media objects?” based on
insights from the conducted survey.
In the first part of our questionnaire, participants were asked to indicate
the importance for each feature on a five-point scale: 1 means that the factor
is never used, 2 that it is used very infrequently, 3 that it is used infrequently,
4 that it is used frequently, and 5 that the factor is used very frequently. Based
on that, Figure 2 shows the mean importance of the factors used in search by
media professionals grouped into the relevant clusters:
Fig. 2. Mean Importance of Factors in Search
Not surprisingly, the cluster with the highest frequency is Information and
Content (I) meaning that users use keywords or classification information to
search for content very frequently. Following this cluster are six clusters having
almost equal frequency. The first one is Technical Quality (T) which includes
factors such as clarity of structure, sharpness, brightness, or technical error.
This is followed by Practices and Feedback (PF) including feedback from experts
and colleagues which has the highest impact. After that, Rights and Licens-
ing (RM), Technical Features (TF), Visual and Compositional Features (V) and
finally Provenance and Bibliographic Data (PM) follow which are used rather
infrequently. Users are typically not using affective factors, context or workflow
information for search. A further observation from related work which has been
confirmed by the expert interviews is, that media professionals typically start
with a keyword query and then extensively use browsing facilities. This confirms
11
earlier insights from [21]. Furthermore it seems to be appropriate to present
images with rather differing visual properties in some situations.
The results from this part of the survey are in line with results from Eakins [3]
who identified topicality and technical quality as the most important criteria
used in search.
Factors Affecting Selection and Assessment of Relevance Our survey
revealed, that users use different factors from all clusters of the model to assess
the relevance of media objects. Figure 3 shows the mean importance of the factors
used for the assessment of relevance grouped into the clusters of the model:
Fig. 3. Mean Importance of Factors in Relevance Assessment
Our results show only small differences to the study done by Westman et
al. [29] in which the authors reported similar mean values for different factors
in relevance assessment. Their study revealed that the cluster Information and
Topicality (I) is the most important one, followed by Abstract and Affective
Factors (A) and by Visual and Compositional Features (V). Our results how-
ever suggest that Abstract and Affective Features (A) are most important even
before Information and Topicality (I). This can be explained by differences in
the domain that we investigated, as a media object in media production only
seems to be relevant if the aesthetics and other abstract features are compatible
with the intended usage. This comes even before Information and content (I).
This observation can partly be explained by the fact how media professionals
reuse media objects; media professionals retrieve media objects for inspiration
very frequently (in 30% of all cases) meaning that they may create media objects
12
which reflect their thought topicality based on an aesthetically pleasing artwork.
The other clusters had a similar mean importance as reported in previous work:
Technical Features (TF) and Technical Quality (TQ) is followed by Publication
Context (C) and other metadata. The smallest mean value is assigned to Work-
flow Related Issues (W) and Practices and Feedback (PF).
It should be noted that some of the factors from clusters which are ranked
lowest such as Rights and Bibliographic Metadata (RM) are in the top-10 of most
important factors such as price or usage rights (cf. Table 2.)
Factor (Cluster) Mean
Aesthetic compatibility (A) 4.67
Topical compatibility with the usage context (I) 4.46
Mental associations (I) 4.39
Technical quality (TQ) 4.32
Price (RM) 4.15
Technical adaptation possibilities (PCW) 4.11
Consistent layout (PC) 4.11
Usage rights (RM) 4.08
Technical format compatibility (TF) 4.04
Adaptation effort (PCW) 4.00
Table 2. Mean Importance of Relevance Criteria (Top-10 Factors)
This explains the difference to the clusters from Westman [29] in that cate-
gory. The importance of rights can be explained by the fact that the Internet is
the most frequent source for reusable media objects followed by the local hard-
disk as our study indicated; on the Internet stock image sites are used most
frequently followed by specialized image search engines such as Google image
search2 . Company wide content management systems are ranked even after so-
cial media sharing sites such as Flickr3 . The importance of adaptability can be
explained by the differences in the domains investigated. In the journalism do-
main, which was investigated by Westman, images or photos are typically used
as is and only marginally adapted, whereas in media production aesthetics and
other abstract features have to be compatible with the intended usage. Further-
more novelty of created media objects is a very important criterion especially in
games, animation or film production, which makes the need for bigger adapta-
tions evident (cf. Section 4.2).
Usage of Selected Media Objects The fourth part of our study revealed
interesting insights into how people reuse media objects that they select. The
types of reuse can be grouped according to the definition provided in Section 1:
2
http://images.google.com
3
http://www.flickr.com
13
(i) content is either reused as is, (ii), only parts of it are reused, (iii) it is reused
after being adapted, or (iv) it is only reused for “inspiration”.
In most cases, media objects are only retrieved for inspirational purposes
which can be explained by the fact that work has to be original in the investigated
domain. A media object as is is only reused infrequently, parts of media objects
are in contrast to that reused frequently. Results of our study furthermore reveal
that content is being adapted very frequently before use. The adaptations range
from basic features like resolution, contrast, brightness to the extraction of the
background or parts of the content such as objects or areas.
5 Conclusions and Future Work
In this paper we reported on a conceptual model which dimensions relevance
assessment in multimedia retrieval scenarios in which people search for content to
reuse. The model is grounded on prior literature, completed with insights gained
from expert interviews and validated based on empirically gathered results from
an end user survey. The model captures factors used for search and relevance
assessment, proposes a clustering of these factors, and assigns a mean importance
value to each factor based on the results of the reported survey.
Our next steps contain the realization of a hybrid image search engine which
integrates content based search with semantic search and which takes the results
reported in this paper into account in order to re-rank the fused result lists from
both search engines. We believe that the values assigned to the factors can
be used to rank results which were retrieved in multi-faceted search including
keywords related to the topic of images but also metadata such as rights, pricing
information, or visual features. We plan to perform a second validation and
calibration of the model based on end user experiments using the search engine.
References
1. H. Chen. An analysis of image queries in the field of art history. Journal of the
American Society for Information Science and Technology, 52(3):260–273, 2001.
2. Y. Choi and E. M. Rasmussen. Users’ relevance criteria in image retrieval in
american history. Information Processing & Management, 38(5):695–726, 2002.
3. J. P. Eakins, P. Briggs, and B. Burford. Image retrieval interfaces: A user perspec-
tive. In Image and Video Retrieval, LNCS, pages 628–637. Springer, 2004.
4. P. Enser. The evolution of visual information retrieval. Journal of Information
Science, 34(4):531–546, 2008.
5. P. Enser, C. Sandom, J. Hare, and P. Lewis. Facing the reality of semantic image
retrieval. Journal of Documentation, 63(4):465–481, 2007.
6. A. Goodrum and A. Spink. Image searching on the excite web search engine.
Information Processing & Management, 37(2):295–311, March 2001.
7. A. Goodrum, M.Bejune, and A. C. Siochi. A state transition analysis of image
search patterns on the web. In Image and Video Retrieval, Springer, 2003.
8. R. L. Gorsuch. Factor Analysis. Lawrence Erlbaum Publ., 1983.
14
9. H. Greisdorf and B. O’Conner. Modelling what users see when they look at images:
a cognitive viewpoint. Journal of Documentation, 58(1):6–29, 2002.
10. L. Hollink, A. T. Schreiber, B. J. Wielinga, and M. Worring. Classification of user
image descriptions. Int. J. Hum.-Comput. Stud., 61(5):601–626, 2004.
11. T.-Y. Hung. Search moves and tactics for image retrieval in the field of journalsim.
J. of Educational Media & Library Sciences, 42(3):329–346, 2005.
12. T.-Y. Hung, C. Zoeller, and S. Lyon. Digital Libraries: Implementing Strategies
and Sharing Experiences, chapter Relevance Judgments for Image Retrieval in the
Field of Journalism: A Pilot Study, pages 72–80. LNCS. Springer, 2005.
13. P. Ingwersen and K. Jaervelin. The Turn: Integration of Information Seeking and
Retrieval in Context. Springer, 2005.
14. C. Joergensen, A. James, A. B. Benitez, and S.-F. Chang. A conceptual framework
and empirical research for classifying visual descriptors. J. Am. Soc. Inf. Sci.
Technol., 52(11):938–947, 2001.
15. C. Joergensen and P. Joergensen. Image querying by image professionals. J. of
the American Society for Information Science and Technology, 2005.
16. C. Joergensen. Image Retrieval: Theory and Research. The Scarecrow Press, 2003.
17. L. Keistler. User types and queries: Impact on image access systems. In R. F. et al.,
editor, Challenges in Indexing Electronic Text and Images, pages 7–22. Medford,
NJ: Learned Information Inc., 1994.
18. M. Laine-Hernandez and S. Westman. Image semantics in the description and
categorization of journalistic photographs. In Proc. 69th Annual Meeting of the
American Society for Information Science and Technology, Austin (US), 2006.
19. M. Markkula and E. Sormunen. End-user searching challenges indexing practices
in the digital newspaper photo archive. Information Retrieval, 1(4):259–285, 2000.
20. M. R. Mullen. Diagnosing measurement equivalence in cross-national research.
Journal of Intemational Business Studies, 26(3):573–596, 1995.
21. R. Othman. A model for an image retrieval tasks for creative multimedia. Perfor-
mance Measurement and Metrics, 6(2):115 – 131, 2005.
22. E. Panofsky. Studies in iconology. Harper & Row, New York., 1962.
23. H.-T. Pu. An analysis of failed queries for web image retrieval. J. Inf. Sci.,
34(3):275–289, 2008.
24. E. M. Rasmussen. Indexing images. Annual Review of Information Science and
Technology, 32:169–196, 1997.
25. A. Rockley. Managing Enterprise Content. New Riders, 2002.
26. T. Saracevic. Relevance reconsidered. In Information Science: Integration in per-
spectives. Proceedings of the Second Conference on Cepnceptions of Library and
Information Science, pages 201–218, 1996.
27. S. Shatford. Analyzing the subject of a picture: a theoretical approach. Cataloging
& Classification Quarterly, 6(3):39–62, 1986.
28. S. J. Westerman et al. Creative industrial design and computer-based image re-
trieval: The role of aesthetics and affect. In Affective Computing and Intelligent
Interaction, chapter Creative Industrial Design and Computer-Based Image Re-
trieval: The Role of Aesthetics and Affect, pages 618–629. Springer, 2007.
29. S. Westman and P. Oittinen. Image retrieval by end-users and intermediaries in a
journalistic work context. In IIiX: Proceedings of the 1st international conference
on Information interaction in context, pages 102–110, New York, NY, USA, 2006.