Many modern goods have both factual and perceptual features. While factual features such as technical specifications can easily be handled by existing database technology, perceptual features such as design or usage experience are very hard to deal with. However, with the huge success and growing market share of online shopping, retailers face the need to provided detailed and structured information about perceptual product features to their customers. In this paper, we analyze why dealing with perceptual product features in databases is difficult and summarize our current efforts on tackling this problem.
Marketing theory distinguishes between two types of product
features: Factual and perceptual ones [
While factual product features can easily be represented and managed by existing database technology (e.g., by introducing a database attribute per feature), working with perceptual product features is much more complicated. This is mainly because perceptual features tend to be vague and defy precise definitions (e.g. the borders of literary genres). However, paradoxically, there are established ways to express perceptual features using natural language (e.g., sporty car or clunky cell phone), which surprisingly mostly are not a matter of taste but are based on general agreement. Therefore, we strongly believe that established database technology is indeed able and suited to store, process, analyze, and answer queries based on perceptual product features. We just have to find out how this can be done in practice.
In this paper, we survey existing approaches to handling perceptual product features in databases, point out their limitations, and summarize our own recent work towards solving this problem. In particular, we present a series of use cases illustrating the benefits of our approach.
In this following, we use movies as a running example. Movies
are particularly suited for this task as they appeal to a wide range
of people and provide a large variety of both factual and perceptual
features. In addition, movies perfectly illustrate the problem of
lacking support for perceptual features in databases: It has been
shown that, when selecting movies, consumers rely far more on
perceptual movie features (funny, romantic, scary, . . . ) than factual
ones (actors, directors, release year, . . . ) [
In this section, we take a close look at existing approaches to handling perceptual product features in information systems. We identified three different groups of approaches: those based on explicit data provided by experts, those based on textual data provided by users, and special-purpose approaches that implicitly deal with perceptual product features. In addition, in domains where products can be represented in digital form (e.g., music or movies), (low-level) features can be extracted automatically. 2.1
Besides the traditional classification of movies into a small
number of major genres [
Although this approach looks rather straightforward and seems to be easy to implement in practice, it comes with many problems. First of all, clearly identifying and narrowing down the most relevant 1http://www.netflix.com 2http://www.allmovie.com 3http://www.clerkdogs.com individual perceptual features tends to be difficult. However, even if a comprehensive and generally understandable classification system has been developed and experts have been trained how to use it correctly and consistently, manually classifying all movies is a huge amount of work.
An even worse problem is the actual consistency of these movie
classifications. We recently compared the genre judgments made by
three major movie databases and found that the agreement among
them is moderate at best, being just slightly less directed towards
to completely random genre assignments than to perfect agreement
[
An alternative approach to making perceptual movie features available to movie databases has been adopted by movie portals such as the Internet Movie Database4 (IMDb) or Rotten Tomatoes5 (RT). Instead of trying to represent movies in a structured fashion by means of explicit database attributes, they focus on textual descriptions, usually in the form of reviews provided by arbitrary users (IMDb) or (semi-)professional critics (RT).
Although textual descriptions give users a comprehensive and helpful characterization of each individual movie, it is difficult to search for movies or provide targeted movie recommendations given only textual data. One the the rare services offering movie search based on movie reviews is Nanocrowd6, which applies information retrieval methods to extract so-called nanogenres from textual data. Each movie is characterized by a set of nanogenres, where each nanogenre is represented by a three-word group (e.g., sports/ballpark/loves or chemistry/adorable/formulaic). However, these nanogenres tend to be much less informative and understandable than explicit database attributes that have been manually created by experts.
Another drawback of text-based movie descriptions is the lack of data. While blockbusters are commented by a large number people, less popular movies often receive just a very small number of reviews, which tend to provide only a partial movie description and are too short to effectively apply methods of text analysis. 2.3
The third major approach is collaborative filtering as used in the
area of recommender systems [
So far, this kind of data has only been used for special problems such as similarity search (finding those movies that are most similar to a given one) or recommendations (providing a list of movies that are likely to appeal to a given user). Here, the basic idea is to analyze the ratings for systematic patterns indicating similar taste across a group of users or similar properties in a group of movies. For example, to provide recommendations to some user u, one might first look for other users who rated most of the movies rated by u in a similar way, and then recommend those movies to u that have been
liked by most of these other users. In a way, movie features and user tastes are modeled implicitly when using collaborative filtering.
Recently, a series of recommendation algorithms has been
developed that try to decompose the rating matrix (movies are rows,
users are columns, and ratings are entries) into the product of two
smaller matrices [
Our own analysis of the semantic spaces produced by recent
recommender algorithms showed that these spaces indeed capture
major perceptual features of movies [
In some domains, one can provide a (near-)complete description of each product in digital form. Prime examples are images, music, and movies. In these cases, it is possible to automatically derive so-called low-level features from the products itself, thus avoiding any dependence on external product descriptions. For example, common low-level features of images are color histograms, symmetry properties, and measures for contrast. Low-level features are contrasted by high-level features (concepts), which describe those aspects of content objects a user is interested in.8 For example, high-level features of images are the types of objects (sun, beach, mother, child, . . . ), events (playing, talking, . . . ), or abstract concepts (family, fun, . . . ) associated with a photo. The multimedia content description standard MPEG-7 defines a large number of lowlevel features and also provides a language to annotate multimedia content with custom-defined high-level features.
In state-of-the-art content-based multimedia retrieval systems,
low-level features are usually extracted automatically from the
available content, whereas the use of high-level features tends to require
a significant amount of human interaction. Although there are initial
approaches to automatically derive selected high-level features from
low-level features, there is still a large discrepancy between the
limited information that one can extract from the available multimedia
data and the interpretation that the same data has for users [
When comparing content-based feature extraction to the three approaches discussed previously, we see that low-level features loosely correspond to semantic spaces and high-level features to explicitly modeled attributes. However, there are important differences:
As low-level features must be extracted by means of spe
8Sometimes, the distinction into low-level and high-level features
is refined to a 10-layer pyramid structure for classifying different
feature types of multimedia content [
Semantic spaces are derived directly from human feedback (e.g., star ratings), which is turn is based on the most relevant perceptual product properties. Low-level features only capture statistical properties of the data representation such as color histograms. Therefore, the semantic gap between semantic spaces and user perception can be expected to be lower than the semantic gap present in current content-based multimedia retrieval systems.
For these reasons, we decided to put aside content-based feature extraction for the moment and focus on the three remaining approaches discussed above. However, in future work we plan to compare the ideas presented in this paper to existing methods from content-based multimedia retrieval where this is possible. 2.5
We can draw the following conclusions from the findings presented in this section:
Modeling perceptual movie features by explicit attributes requires a huge amount of manual work but still leads to data of questionable quality. However, users can easily understand the meaning of these attributes.
Capturing perceptual movie features by means of textual descriptions is helpful for users when looking for information about each individual movie. However, this kind of data is difficult to process automatically, cannot be understood as easily as explicit attributes, and the amount of available data is scarce for less popular movies.
Semantic spaces created from a large number of user-provided ratings capture major perceptual features of movies. However, semantic spaces as such do not offer any intuitive interpretation and thus cannot be used to communicate with users. 3.
At first view, the result of our above analysis is rather disillusioning. Intuitively understandable models of perceptual movie properties are expensive to create and lack data quality, semantically meaningful models cannot be understood, and the third option seems to combine both disadvantages.
However, there is still hope. In [
A. attributes describing factual movie properties, B. attributes making a selected number of perceptual properties explicit (manual classification), and C. attributes containing the movie’s coordinates in some semantic space.
This approach brings several advantages. Probably most
important is that the three different types of attributes can work together to
reduce the weaknesses of each of them. In the following, we show a
series of examples illustrating this idea (for technical details, please
see [
Enhancing the data quality in type-B attributes.
By aligning the manual classification of movies as expressed in type-B attributes to the semantic space, we are able to detect are large number of possibly misclassified movies. The basic idea is that movies that are classified into the same category should also be located close together in the semantic space. If we find a movie m that has the same value with respect to some type-B attribute but is very different from other movies having this value with respect to the semantic space, then m is likely to be misclassified by the experts. By identifying such movies and giving human experts
a chance carefully re-check problematic movies, the data quality
can be increased. In our experiments on genre classifications [
Saving manual work in creating type-B attributes.
To significantly reduce the amount of work required to manually
classify all movies with respect to the type-B attributes, automatic
classification can be applied. Here, given a binary type-B attribute
(e.g., the genre Action), a human experts provides a small number
(e.g., 10) of clearly positive examples (i.e., typical Action movies)
and the same number of clearly negative examples (i.e., obvious
non-Action movies). Using a support vector machine classifier that
categorizes all remaining movies based on the training data and the
type-C semantic space representation of movies, we have been able
to produce results being only of slightly lower quality than those
created by human experts [
Again, by comparing type-B attributes to the semantic space
represented by type-C attributes we are able to determine to what degree
a type-B attribute value applies to each movie. For example, IMDb
only assigns binary genre judgments to its movies, which leads
to the classification Drama/Romance/Sport for the movie Rocky
(1976) and Action/Adventure/Mystery/Sci-Fi for the movie Star
Trek (1976). Although this classification is justified, there are
several problems: Rocky contains romantic elements but it is a highly
untypical Romance movie. It is most well-known for being a typical
sports movie with dramatic activities. Similarly, Sci-Fi is widely
recognized as Star Trek’s most prominent genre, while it is a rather
untypical Mystery movie. By analyzing the semantic space for
where typical movies of genre X are located, we are able to judge
how typical an assigned genre for each movie really is. To illustrate
this, Figure 1 depicts a “genre clouds” for the above two movies. We
automatically generated it from IMDb’s binary genre assignments
(type B) in combination with a semantic space extracted from ratings
(type C) [
Enabling conceptual queries.
When describing their movie preferences, users often refer to
factual movie properties as means attributes that approximately
characterize an intuitive concepts that they are unable to express
otherwise. For example, movies in the style typically associated
with the director Quentin Tarantino could be called Tarantino-ish
movies. In fact, Google counts 4530 Web pages mentioning this
term. We refer to database queries in this style as conceptual queries.
We are able to answer such queries by first finding out where movies
directed by Quentin Tarantino are typically located in the semantic
space (by identifying a small continuous region in space), and then
looking for other movies that are located close to the center to this
region. By applying a simple weighting scheme, we are able to
produce a, say, top-10 list of the most Tarantino-ish movies. To
give an example, Table 3 shows our results for Tarantino and two
popular actors. Here, we used a support vector machine to learn
where movies directed by Tarantino tend to be located in semantic
space and used this information to find very similar movies that have
not been directed by Tarantino [
In this paper, we have discussed the problem of representing perceptual product features in databases. We concluded that each existing approach alone does not provide an acceptable solution to this problem as it comes with severe disadvantages. However, by combining several methods into a joint data model, we have been able to reduce the weaknesses of each individual approach and boost its strengths. Our examples show promising results, which we are going to analyze in detail in future work. In addition, as already indicated in Section 2.4, we plan to compare our work to approaches from content-based multimedia retrieval. For example, for genre classification tasks, it would be interesting to compare semantic spaces derived from ratings to low-level features extracted from the actual movies. Tarantino-ish
I am very grateful to Prof. Dr. Wolf-Tilo Balke for providing valuable guidance and supervising my doctoral thesis, which will be based partly on the work presented in this paper.