Introduction
Machines' lack of outside world knowledge is one of the biggest problems on our
way to achieving a human-level arti cial intelligence. Even if we can equip robots
with sensors that humans do not posses, cognitive architectures are far from
acquiring the same quality of perception. When we read the famous example of a
grammatically proper English sentence \Colorless green ideas sleep furiously" [
2
]
we e ortlessly realize what is wrong with its meaning, but for arti cial agents
it is not so obvious because they lack common sense. For Japanese language,
which we use in our applications, there is not much choice when it comes to
common sense knowledge resources. The only database available is ConceptNet
ontology [
8
] which growth relies mostly on knowledge from http://nadia.jp
service where users \teach" a child named Nadya through a guessing game. The
problem with this approach is that users get bored quickly and try to be
original or humorous which leads to assertions as \music CreatedBy me" or \words
CreatedBy expectations" which are not completely wrong in a semantic way but
di cult to be utilized for reasoning about physical world. For that reason we
try to automatize the knowledge acquisition process [
17
] and concentrate on
acquiring semantically new entries in opposition to methods utilizing similarity [
9
].
Our approach to this problem is to support sensing technology with text-mining
techniques [
16
] and applying them for various AI tasks as moral reasoning [
12,
15
], arti cial therapists [
13
] or poetry generation[
14
]. One of the biggest
problems with \textual sensing" approach is that words used in text are used both
literally and guratively. In this paper we use colors as an example of physical
features of objects which, when shallow NLP techniques are used, can lead to
learning wrong commonsense knowledge which often happens in automatic
approaches [
1
]. Automatic color recognition method in text was introduced in [
3
],
where English ConceptNet was used. Because of a very small size of Japanese
version, implementing the same vector-based method was impossible, therefore
we decided to develop an original method utilizing a corpus. Another di erence
is that the work on English language has limited single word to single color, but
in our method one noun can have multiple colors.
2
System Overview
Colors in ConceptNet are usually stored under HasProperty relation, for example
\ink HasProperty black" or \cayenne pepper HasProperty red", however they
can be also found as edges of other relatios as in "blood SymbolOf red" or
\apple IsA edible red fruit". In our experiments we concentrated on HasProperty
relation which comprises only 3.6% of Japanese ConceptNet 4 and needs to be
extended. There are 140 entries regarding 6 colors (red, blue, yellow, brown,
white and black) which we chose for being an adjective (for example \green"
is expressed by a noun in Japanese language), while the YACIS blog corpus we
developed [
10
], contains 748,078 sentences using these colors. We retrieved these
sentences under the condition that color adjectives are accompanied by nouns
labeled by the morphological analyzer [
6
] as \usual nouns" (this helps excluding
proper nouns as names) and that are not noun phrases consisting more than one
noun. Then the acquired nouns are counted and if they appear less than twice
in the whole set, they are deleted as rather unusual ones. Finally, the ltered
sentences are used for retrieving nouns stored in six colors categories generating
a database. For example, \There was a red apple shaped key case in my bag"
put \apple", \key case" (one word in Japanese) and \bag" as the related nouns
candidates under the \red" category. When we analyzed the most frequent 10
nouns for each color, except natural associations (\blue sky", \white rice" or \red
ower" we found numerous errors as expected (\brown part", \yellow color",
\white lover", which is a popular cookie brand name or \yellow elephant" which
is a cartoon character). Error analysis showed that there are many examples
which are true but hard to be unequivocally categorized as common sense about
the physical world as \black feeling". Many nouns appeared in di erent color
categories (\eyes", \ owers", \birds", \men", etc.) but we believe they should
not be treated as errors and might be stored as separate concepts. To determine
which noun is described by a given color in a sentence more accurately, we
implemented dependency parsing with CaboCha tool [
5
]. If a color adjective was
not immediately followed by a noun, we also collected nouns from dependency
chunk ending by a particle suggesting existence of subject (wa, ga, mo1, etc.) For
1 Original Japanese words are written in italic throughout the paper.
example from Tomato-wa mi-mo akai (\Tomatoes, and also their seeds, are red")
both \tomatoes" and \seeds" are retrieved because subject indicating particle
wa followed \tomato". This addition allowed to eliminate some ambiguous nouns
but many problematic pairs still existed. Next we decided to extend our method
with corpus frequency checks and eliminated pairs with signi cantly smaller hit
rates, however the overall quality did not improved as much as expected. For that
reason we decided to limit nouns to these which exist in Japanese ConceptNet 4,
what decreased number of color related words (see Table 1) but visibly improved
the quality of retrievals.
To investigate how the above mentioned quality improved, we performed an
experiment with nouns chosen randomly from the ConceptNet and to showed
them to ve judges (3 male and 2 female Japanese students in their twenties).
They labeled the words with six previously mentioned colors and if the majority
agreed on the same colors for a noun, it became the part of \color" set, giving
60 nouns in total. To prepare balanced counter-set, we randomly chose 60 other
nouns for which the judges did not choose a color. For example, judges agreed
that roses are red, corn is yellow, snowman is white and pianos are black and
white. Words as \book", \skirt", \fruit" or \scarf" were marked with all six
colors.
Then we input the nouns into the system and generated ve sets of results for
1, 5, 10, 15 and 20 retrievals as thresholds to see how strict the process should
be. The results (see Tables 3) show that 10 retrievals threshold is the most
balanced one, however if we focus on precision, for example when gathering
new concepts for the ontology, it seems that the higher threshold the higher
quality noun-color(s) associations can be obtained. Adding dependency parsing
was obviously e ective but limiting output to high occurrences improved the
results only minimally.
In the second part of our experiment we wanted to see how good is our
system in recognizing non-physical nouns without colors (e.g. \youth", \leadership",
\energy-e ciency", \universal gravitation" or \disapproval"). Experimental
results for this recognition are presented in Table 3. This time high precision was
achieved and the method showed capability of distinguishing objects with and
without colors (physical and non-psychical objects), though there are
transparent physical object which should be dealt with separately.
The rst problem we have spotted with automatic color assignment was that
proper names often had unproportionally high frequencies in the corpus. Japanese
language does not have any equivalent of big letters, so to tackle this problem, we
added a step to the algorithm which checks if there is a Wikipedia page for any
color-noun pair. If it is found, this straight phrase is then not counted, leaving
only counts of retrievals like \noun was color". For example it eliminated rather
unnatural in Japanese language associations as \White Lover" (cookie brand) or
\Red Fox" (instant noodles brand), but also \White Scarf" (music piece title),
\Yellow Book" (comic book title), \White Book" (cartoon title) or \Blue Sky"
(book title), which represent natural colors. It appears that a deeper semantic
processing is needed, because there are much more natural color-noun phrases
in proper names than we expected. Another concern was to see if setting more
strict thresholds will further improve the results but it appeared that after 40
retrievals limit recall drops causing f-score to decrease below the baseline level
because high recall of the baseline is not limited by occurrences and accepts even
very peculiar color-noun pairs (see Figure 1).
Automatic color recognizing is not a new research topic but image processing
eld tends to concentrate on speci c tasks as licence plates recognition [
18
]. It
is often a part of wider applications as locating faces in complex backgrounds
[
7
] but until recently gathering knowledge through images was di cult. The
latest image understanding tasks using Deep Learning [
11, 4
] open a wide range of
possibilities for enriching common sense knowledge bases. However, statistical
approaches need often costly annotation process and any additional support is
needed. We think that commonsense knowledge ontologies as ConceptNet could
help with adding weights to such algorithms. For example Karpathy's
experiments with Deep Learning2 (see Figure 2), suggest that statistical methods
2 cs.stanford.edu/people/karpathy/linear_imagenet/
are not perfect (\green gorilla", \pink ambulance", \pink bucket", etc.). In our
opinion combining both ontological and stochastic approaches could decrease
numbers of classi cation mistakes as background knowledge might provide
simple rules as leaves are green / yellow / red and branches are brown / black
to avoid mixing both colors as a \tree" representation. Our preliminary tests
showed that without laborious and costly annotations it is possibile to predict
colors of objects and distinguish concepts with color features from ones which
do not posses them. For example, for our metaphor generation project we need
a solid knowledge base about physical and abstract words and their features.
Preliminary tests presented in this paper show promising level of precision for
text-based approach in this task, however there are obvious problems remaining
when it comes to proper color assignment. There are two basic directions in which
the method could be used, to concentrate on precision given by threshold and
to minimize manual check before adding data to the ConceptNet or maximizing
the number of automatically acquired concepts for further automatic re ning.
In the next step we plan to experiment with image processing and to extend
the method to adjectives representing di erent types of cognitive perceptions as
shapes, sizes or sounds.