While still influential, the view that concepts are organized as a hierarchical taxonomy as proposed by Rosch (1973) has been challenged on several occasions. For example, some studies have attributed a larger role to thematic relations (Gentner and Kurtz, 2005; Lin and Murphy, 2001), whereas others have stressed the role of affect in structuring word meaning (Niedenthal et al., 1999). A comprehensive account of how these different principles shape and structure meaning in the lexicon is missing, and most studies continue to be biased towards concrete noun categories that fit into hierarchical taxonomies (Medin and Rips, 2005). To capture mental or psychological properties that organize the lexicon for a wide range of concepts and semantic relations, we propose a large-scale semantic network derived from word associations as the basis to uncover what the structural principles are.
Since this is one of the first times the mental
lexicon is mapped in its entirety using an extremely
extensive word association corpus, an exploratory
approach is warranted. To achieve this, network
clustering was used as a way to study how the
mental lexicon can be structured at different scales
and what type of semantic relations dominate its
structure. At the basis lies a semantic network
derived from a large scale word association corpus
including over 12,000 cues and 3.77 million
responses
Applying OSLOM to the semantic network resulted in a solution with five hierarchical levels. An overview of this solution is shown in Table 1. There was a large degree of variability in the number of clusters across the five hierarchical levels ranging between 2 and 506 clusters. On average, the p-value of the extracted clusters was low across all levels, indicating that the obtained clusters were unlikely to arise in a comparable random network1. There were few homeless nodes at any level, indicating that most words were reliably attributed to a specific cluster. There was also a considerable degree of overlap at all levels relative to the size of the clusters; clusters were more distinct at the more precise levels, where more clusters were obtained. For instance, at the lowest level 1,676 words appeared in multiple clusters, compared to 5,943 at the highest level.
Figure 1 illustrates the obtained clusters with
the most prototypical examples of each cluster at
various levels. At the most general level, Figure 1
shows two distinct clusters, with one of them
containing highly central words with a negative
connotation. In order to verify whether this
interpretation is supported statistically, we used the valence
judgments reported by Moors et al. (2012), which
1Default parameters were used in the OSLOM algorithm,
except for the p cut-off value. Setting this value depends on
the task as it affects the size of the clusters
At Levels 2 to 4, the meaning clusters become increasingly more concrete. For instance, Level 2 shows that the “negative” cluster in Level 1 includes clusters with abstract words or words related to human culture (school, money, religion, time,...) which are now differentiated from a purely negative cluster with central members like negative, sad, and crossed. The subdivisions of the “positive” cluster involve the central nodes nature, music, sports, and food, which might be interpreted as covering sensorial information and natural kinds.
At the lowest level, 506 clusters were
identified, with an average size of 25 words. A total of
1,676 words occurred in multiple clusters; at least
a part of them because of homonymy (e.g., bank)
or polysemy (e.g., language, assigned to clusters
about nationality, speech, language education, and
communication). Most importantly, inspection of
the content of all clusters exhibited a widespread
thematic structure: the clusters were often
composed of both nouns (racket), adjectives (loud),
and verbs (to sound), which does not reflect a pure
taxonomy of entitities, but also includes properties
and actions.
To test whether the clusters provide evidence for
a hierarchical taxonomic view along the lines of
Rosch and colleagues
A taxonomic-like organization would be evident in clusters with high precision and recall, resulting from many true positives and few false positives and false negatives. For instance, if the cluster corresponding to the category BIRDS contained robin (a true positive) and did not contain spoon (a true negative), that would increase the F-score. Conversely, if it contained guitar (a false positive) or did not contain ostrich (a false negative), that would decrease the F-score. This way, high Fscores should reflect categories that are not overly specific (many false negatives) or general (many false positives).
On average, the best matching clusters were found at Level 5. The results for each category are shown in Table 2. The average number of members in the exemplar generation task was on average 41 for the seven natural kinds categories, which is in the same range as the average best matching cluster size of 42. For artifacts the generated categories included on average 55 members, which was somewhat larger than the obtained average cluster size of 37.
The resulting F-values were on average 0.48 for the natural categories and 0.28 for the artifacts, indicating only limited support for the presence of taxonomic categories. The highest values were obtained for FISH (F = .57) and REPTILES (F = .65) where most items in the clusters were true category members.
Inspecting the false positives for each of the clusters in Table 3 confirms the validity of the approach as in the majority of the cases the superordinate label (e.g., fruit, tools, etc.) was the most central member of each cluster. The remaining intrusions were thematic in nature (e.g., FRUIT: pick, BIRDS: nest), thus confirming our earlier exploratory findings.
One potential response to the previous analyses relates to the nature of the data upon which they are based. Perhaps the word association task simply fails to capture taxonomic information, and if so, the results of these analyses are simply an artifact of the choice of task. Alternatively, perhaps the “failure” arises because the word association task is more general than the tasks typically used to study taxonomic categories.
There is some evidence that a different choice of
task would produce different results. For instance,
much of the work on taxonomic organization
relies on tasks in which participants are asked to list
features of entities
To test this idea, we used the word association data to construct a network that included only those 588 words that belonged to one of the taxonomic categories. Moreover, in order to approximate the “shared features” measure that is more typical of feature generation tasks, we computed the cosine similarity between pairs of words. That is, words that have the same associates are deemed more similar, and this similarity was used to weight the edges in the restricted network.2 We then applied the clustering procedure to this restricted network and repeated the analysis from the previous section. The F -statistics from this analysis are reported as the F 0-values in Table 2. This time, the results of the clustering show a high degree of agreement with the taxonomic organization, with an average F -value of 0.79. The only exception was REPTILES, which upon inspection appears to reflect a failure to distinguish REPTILES from INSECTS.
The success of this analysis suggests two things. First, the word association task does encode taxonomic information, as evidenced by the fact that we are able to reconstruct taxonomic categories.
2Note that one could also derive such a similarity-based network for the complete lexicon, which would reflect the similarity between cues rather than their weighted associative strength. We did in fact do this. It produced similar results to the original analysis. 3 pit puree nest beast rod tail animal blouse stove fanfare carpentry vehicle blade
4 pick sausage whistle crawl slippery pen tail collar cooker hood orchestra wood motor point
5 summer hotchpotch
egg animal
water marten amphibian zipper burning harmony drill circuit stake However, the fact that the only way to do so is to mimic all the restrictive characteristics of a feature generation task (e.g., limited word set) is revealing. Taxonomic information is not the primary means by which the mental lexicon is organized: if it were, we should not have to resort to such drastic restrictions in order to uncover taxonomic categories.
In summary, even at the most detailed level of
the hierarchy, only limited evidence for a
taxonomic view along the lines of Rosch was found,
even for typical taxonomic domains like animals.
These results suggest that in much of the
previous work the pervasive contribution of affective
and thematic or relational knowledge structuring
might be overlooked by a selection bias in terms
of the concepts (nouns, mostly concrete) and
semantic relations (predominantly taxonomic). This
finding is in line with previous results
indicating that network derived similarity estimates
account better for human thematic relatedness
judgments than for taxonomic relatedness judgments
(De Deyne et al., in press). In priming studies,
the dominance of thematic over taxonomic
structure can also explain facilitation when thematic but
not coordinate prime-target pairs are used