=Paper=
{{Paper
|id=Vol-481/paper-7
|storemode=property
|title=Symbol Emergence in Design
|pdfUrl=https://ceur-ws.org/Vol-481/paper-6.pdf
|volume=Vol-481
|dblpUrl=https://dblp.org/rec/conf/nesy/MukerjeeD09
}}
==Symbol Emergence in Design==
https://ceur-ws.org/Vol-481/paper-6.pdf
Symbol emergence in design
Amitabha Mukerjee, Madan M Dabbeeru
Indian Institute of Technology Kanpur
amit@cse.iitk.ac.in, mmadan@iitk.ac.in
Abstract flow of energy, information, etc, and proceeds downward into
detailed design. With its roots in value engineering ideas
A key step in mapping the more conceptual stages from the 1940s, these notions were seeded by the analysis
of design onto computational systems involves in Pahl and Beitz [Pahl and Beitz, 19881996] and a partic-
identifying a vocabulary and ontology. While a ularly influential study by Welch and Dixon [Richard and
number of high-level ontologies have been pro- Dixon, 1994], leading to modern ontological models like
posed, these are difficult to ground in terms of ac- the widely used functional basis model [Hirtz et al., 2002]
tual design instances, and manual definitions of the or implementations on ontology tools [Nanda et al., 2007;
symbols are often incomplete and difficult to main- Szykman et al., 2001].
tain. As an alternative, we propose an ”infant de- The above represents the human-engineered approach to
signer” paradigm which abstracts patterns for the defining symbols. This type of approach is initially tempt-
”functionally feasible regions” (FFR) while evalu- ing because it tends to meet immediate applications, but a
ating many individual configurations in the design long history in knowledge-based systems has shown it to be
space. These learned FFR patterns (which may brittle, i.e. subject to failure under even minor deviations in
arise due to minimal levels of functional accept- the domain. In general, it may be that symbols are more
ability, or from optimization) often embody depen- meaningfully developed by abstracting from existing data.
dency relationships among the design parameters, The novel contribution of this paper is to show that at least
i.e. the good designs lie along lower-dimensional in certain types of design tasks, lower-dimensional surfaces
manifolds in the design parameter space. We show are revealed by multi-objective optimization. The intrinsic
how such manifolds exist in several design situa- dimensions in these pareto-surfaces might constitute one ap-
tions; each combination of the original design pa- proach to obtaining “symbols” directly from experiential data
rameters may be thought of as a ”chunk”; the space as opposed to engineering them by programming definitions /
of these chunks models only the ”good designs”. rules. These approaches are detailed further in section 1.2 and
Next, we show how the patterns defined based on section 3, but first we look more closely at the term “symbol”,
these chunks constitute image schemas, which may and what is understood by its semantics.
be implicit (e.g. the pattern for an FFR), or ex-
plicit (where the relationship is observable). These 1.1 The semantics of design symbols
patterns or image schemas are incipient semantic
model leading to symbols. We present examples Unfortunately the term “symbol”, as it is used in the logic
of how such image schemas are arrived at with the and computational theory is considerably different from its
help of universal motor design. usage in cognitive linguistics and in everyday life. In the
latter usage, symbols are imbued with meaning grounded on
experience, whereas in the formal usage, it is merely a to-
1 Efforts towards standardizing the design ken constructed from some finite alphabet, and is related only
to other such tokens. If we present an analogy, a blind man
vocabulary knows “red” is a different color from “blue” and “green” but
Evolving a standardized vocabulary for design has emerged his understanding of red is dramatically different from that of
as an important focus in engineering design. Possible appli- a sighted person, because the semantic pole is not connected
cations include developing design repositories [Bohm et al., to direct experience.On the other hand, “symbol” has come to
2005], computer assisted conceptual design [Gero and Fu- be understood in cognitive science (and also traditionally in
jii, 2000], etc. It is clear that vocabularies are structured, linguistics, e.g. de Saussure ( [De Saussure, 19161986]), as
that is there are considerable relations between terms. Of- the tight binding of the of the psychological impression of the
ten, this is viewed as an ontology or as a structured rela- sound (the “phonological pole”) with the mental image of the
tionship that captures a part of the semantics of these terms. meaning (the semantic pole) [Langacker, 1986]. The mental
One popular view of the engineering system considers the image or image schema includes all sorts of associations and
28
car High
Communication level
Design
Pattern 1 (Phonological pole) Image 4
Exp.1
is-a is-a
is-a
Design term
Pattern 2 Chunk Image schema jeep sedan hatchback Low
Exp. 2
(Semantic Pole) Image 1 Image 2 Image 3
level
Image schema
Design
Pattern 3 Symbol
Exp.3
Instance level
Figure 1: Emergence of symbols based on experience: Often
the same abstract pattern (or chunk) appears in many experi-
ences (e.g. the notion of “containment” for peg in hole, bolt in
latch, plug in sink, etc.). If a chunk is valuable in compactly Figure 2: Abstraction starts with ground instances: Sym-
representing many situations, it has a higher likelihood of be- bols like “hatchback”, “sedan”, or “jeep” may correspond an
ing communicated, thus acquiring a phonological pole and abstract pattern or “image schema”, which is used to iden-
becoming a symbol. A symbol can then form other associa- tify instances as belonging to a symbol category, but also in
tions besides the initial chunk, all of which together constitute composing symbols, and in interpreting higher abstractions.
its semantic pole or image schema. Primitive design ontologies like is-a arise when instances al-
ready known as sedans or hatchbacks are also labelled as
“car” by a trusted user. Similarly, other relations e.g. “jeeps
can drive over rough terrain” would also be learned through
is somewhat different for each user, though social convention usage and become part of the image schema. The number
ensures a degree of overlap between mental images within the of such associations for each symbol is often very large, and
language community. limiting these to a few user-determined definitions is a major
contributor to brittleness in knowledge systems.
However, the notion of symbol is more far-reaching than
communication. It turns out that to some extent, the sym-
bols help divide up the world into classes, and eventually, it
may reflect changes in how we think. For instance, Korean 1.2 Bottom-Up Semantics in design
language makes a distinction between spatial tight-fit situa- An alternative that has been proposed for modeling design
tions, kkita, (as in “put the cap on the pen”, “hand in glove”) concepts is to attempt to move more towards the human pro-
from other usages of “in” or “on”. Infants growing up in En- cess, to learn symbols based on design experience[Gero and
glish and Korean linguistic environments were sensitive to Fujii, 2000]. The human design process is a constant, moti-
both contrasts, but English children appear to lose this sen- vated exploration of the design space, e.g. through sketching.
sitivity around the time they start acquiring language, sug- All the while, the designer is focusing on the designs that are
gesting that the language construct may have weakened their “good” in some functional sense, and eventually, some kinds
sensitivity to these changes [McDonough et al., 2003]. of patterns emerge as the common characteristics of these de-
On the other hand, incompatibility of design vocabulary is signs. This is one sense in which sketches “talk back” to the
rarely a problem between humans (that’s why exceptions of- designer [Goldschmidt, 2003]. These patterns result in con-
ten become memorable). If designers A and B are talking, straints whereby many of the initial design variables can be
and A does not have a particular symbol λ, its image-schema combined, a process cognitively known as chunking [Gobet
may emerge through a small amount of discussion; in many et al., 2001].
cases, just a single example may be enough to stretch an ex- For example, in designing a padlock, we may learn that the
isting concept λ0 in A to the current one. Of course, the new shackle diameter increases roughly in proportion with body
symbol λ0 remains imprecise, and designer A is aware of it, size. Thus these two parameters can then be brought down to
and subsequent uses of λ0 will serve to ground it. All this is a single chunk. These chunks, which limit the choices used
possible because the semantic pole for the human is a com- in “good designs”, may be what are used by expert designers
plex, elastic set of associations that cannot be defined in terms [Gross, 1986].
of a single predicate or even a range, it is the set of all situ- An early attempt at discovering patterns in the design space
ations where the symbol may be encountered (figure 2). All of shapes may be seen in relation to 2D shapes in the work of
these associations need to be learned, and cannot be inferred [Park and Gero, 1999]. [Moss et al., 2004] have developed
based on a single definition (not to mention issues such as a system in which a design observer agent considers trends
nonmonotonicity); hence the programmer-given single defi- among good designs and try to extracts chunks. Similarly
nition, usually created to demonstrate the example at hand, is a recent approach by [Sarkar et al., 2008], considers Singu-
a hopelessly inadequate semantics for a design symbol; and lar Value Decomposition (SVD) on a co-occurance matrix of
that is why we need bottom-up symbol discovery in order to matrix of variables and constraints to identify the relations
ground a design vocabulary. between different variable groups.
29
However, none of these proposals attempt to learn their 1
symbols in a grounded manner, and therefore lack the flexi- 0.8
bility of the human designer. By grounded, we refer to the 0.6
t
progressive manner in which a human designer learns her 0.4
concepts - the more abstract ones are based on earlier, con- 0.2
crete concepts, but are still presented through instances. In 0
0 0.5 1
the end, many concepts are grounded in terms of a number w
of experiential instances. For a human designer, this learn-
(a) Latch-in-Slot assembly (b) 10 instances
ing cannot be limited to the years of training as a designer,
1 1
but must include all of her knowledge about the world, the so
0.8 0.8
called commonsense knowledge. Thus, the fact that a fat peg
will not go into a thin hole is part of her prior knowledge. In- 0.6 0.6
t
t
deed, it is likely that the process by which she acquires these 0.4 0.4
patterns, built upon many layers of pre-existing knowledge, 0.2 0.2
may be similar in some salient ways with her earliest learn- 0
0 0.5 1
0
0 0.5 1
ing. w w
In this work, we propose to take the first step towards build- (c) 50 instances (d) 200 instances
ing such a grounded semantics, which we call the birth of
symbols. In a human design scenario, say while “talking” to Figure 3: Learning through experience that latch-must-be-
a sketch, a designer may get a conscious awareness of a con- smaller-than-slot (w > t). (a) A latch of thickness t is fit-
straint without verbalizing it - this is referred to as reification, ted to a slot of width w. The learned patterns are shown in
becoming real - and is a key step in forming new symbols. (w, t)-space in (b)-(d). The quality of the learned pattern
Sometimes, amorphous implicit schemas, which are formed varies greatly with degree of experience: results shown for
well before we are aware of them [Gladwell et al., 2005] are a multi-layer perceptron after experiencing 10,50, and 200
incipient symbols, but they need to prove their mettle before design instances.
they become true symbols. This interpretation is in line with
a long tradition in psychology and linguistics, that symbols
are “aware” or conscious [Mandler, 2004]. the slot. The function is defined in terms of the degree of fit
- how much does it wiggle? Defining the wiggle in terms of
2 Infant designer the area of the free-space in the configuration space, we see
that if the wiggle desired is very small, we get the situation
A system learning symbols is like a baby who is first discov- on the left, and if it is very large, we get the situation on the
ering regularity of object behaviour in the world. She can right. Eventually, the learner learns the concept of “fit” as
make various choices, and evaluate them based on some no- a chunk (composed as w − t) - thus, given a level of fit, it
tion of function. Considering the peg-in-hole task just alluded imposes a constraint where w and t are related in a manner
to, we see how she might learn the concept that a peg must be where they constitute a one-dimensional chunk instead of two
smaller than a hole. independent variable.
The functional model considered is simple - the design is Of course, from a machine learning perspective, both these
functionally feasible if the peg can go in (actually our system examples are rather elementary. Our objective in presenting
computes the configuration space - the penetration region dis- it is merely to emphasize the role of even the earliest knowl-
appears when w > t). We consider a horizontal version of the edge in many advanced design situations. These two con-
peg-in-hole - a latch is entering a slot on a bolt, say. Figure 3 cepts are also among our earliest knowledge achievements;
shows how after evaluating a number of instances in the de- typically, infants learn containment (peg in hole) by about
sign space of latch-widths w and slot-widths t; in (w, t) space, 3 months, and tight vs loose by 5 months [Casasola et al.,
a clear 45 degree line emerges, separating the “good designs” 2003]. Many cognitive scientists believe that our concepts of
from the bad. abstraction, including the is-a crucial to constructing hierar-
Does this constitute symbolic knowledge for the infant de- chies, is a metaphorical extension of containment [Lakoff and
signer? Most likely not. However, it is something that might Johnson, 1999].
become a symbol as she acquires other concepts that she
can refer to. What is interesting in the results of figure 3
is how, after experiencing just a few instances, the pattern is
3 Symbol emergence
inchoate, so the baby keeps trying to insert the fat square into As the designer matures from infancy, we can consider the
the smaller circle, filling up the negative (black) area of the more general process by which symbols form. These may
figure. Eventually the defining boundary becomes sharper, correspond to the stages shown in figure 5. At first, the de-
and at some point it can be said to knows the principle, at signer explores with instances in the design space, distin-
least implicitly. guishing the good designs from the bad. Eventually a sub-
At the next step for our infant designer, we consider the set of the design space emerges as the Functionally Feasible
concept that a designer knows as “fit”. By now our infant region (FFR), or the space of “good designs”. Often, FFRs
learner will attempt to insert pegs only if they are smaller than correspond to narrow bands of functional feasibility. This
30
Functional
Pattern Learned
Ω feasibility
learning
Design Space Ω FFR FFR Pattern
Tight fit Medium fit Loose fit
10 10 10
w=t w=t w=t
feasible feasible feasible
infeasible infeasible infeasible FFR
D
5 5 5 Dimensionality
t
t
t
w 0
−0.1430 and γC = 0.0007. g2 (v) ≡ 5000 − H > 0,
(1)
This reduction of the two design parameters to a single γ g3 (v) ≡ 2.0 − πM ass ≥ 0,
represents the first stage of symbol formation. If, later, this g4 (v) ≡ 0.5 ≤ πtorque ≤ 5.0,
γ chunk is discovered in other situations, then a label, say g5 (v) ≡ 300 ≤ πP ower ≤ 600
“gavagai”, may attach to it. Then as the term “gavagai” may g6 (v) ≡ πef f iciency − 0.15 ≥ 0
spread in the design community, and might occur in many
32
3 the design space as well. These two dimensions possibly re-
2
flect inter-relations between the original eight parameters that
D 1
pertain to the better designs in the design space. In terms of
4 C C symbol formation, these two dimensions (“XX” and “YY”,
torque
0
say), if they are found repeatedly in other domains as well,
2
D E
may eventually become symbols. With sufficient experience,
π
−1
2
E 1.5
0
1 1
−2 the relation between these two parameters and the design may
0.8
0.6
0.5
π −3
eventually be encoded into design rules: e.g. “higher YY is
πefficiency
−2 −1 0 1 2 3 4
0.4 0 mass
usually associated with the more efficient designs”. Subse-
(a) (b) quent experience may also alter the way we understand these
chunks, and therefore rules like the above that are built on it;
Figure 7: The non-dominated front for the Universal mo- through this demonstration we are primarily arguing that by
tor. (a) The non-dominated solutions (pareto-front) in the keeping these symbols grounded, it would be possible to keep
3-objective space of mass, efficiency and torque. (b) The updating their semantics and their inter-relations (the rules),
manifold space corresponding to the map from the high- thus providing a truly flexible symbol system, in contrast to
dimensional design space D = 8 to low-dimensional design static symbol systems.
space d = 2 obtained by LLE. Note that the distribution of We must be careful to point however, that in general a k−1-
colours are non-uniform in the two maps, but they remain dimensional pareto-surface in objective space may not map
segregated (with some noise). to an equivalent manifold in design space - there are a large
number of situations where the performance metrics mapping
from design space to objective space are not so well-behaved,
1
and such results may not hold. Nonetheless, even if a subset
Normalized Residual Error (rd)
0.8 of design parameters are well-behaved, at least some dimen-
sionality reduction may occur in these spaces. To obtain an
0.6
estimate of the dimension of the manifold for our data set,
0.4 we use the technique based on the idea that a dimensionality
reduction algorithm should preserve information on a global
0.2
scale, so that the inverse mapping error should be minimal.
0
0 2 4 6 8 10
For a given input dataset X = {X1 , . . . XN } ⊂ RD , the
dimension (d)
dimensional reduction algorithm such as LLE provide a re-
duced dimensional representation Y = {Y1 , . . . YN } ⊂ Rd
Figure 8: Dimensionality of manifold for Universal Motors of the original data set X. How to determine the reduced-
based on . The FFR data is mapped onto manifolds of differ- dimensionality d is not clear; one approach may be to con-
ent dimensions, and then mapped back to the original design sider several d’s and select
space and the error is estimated. The error drops sharply from P that which minimizes the residual
bijection error (rd ) = i ||fd−1 (fd (Xi − Xi )||,[Martin and
1-D to 2-D manifold, and then less sharply. The knee of the Backer, 2005] wherefd : X → Y is the map produced by
curve at “2” is indicative of the intrinsic dimensionality of the LLE. By observing the behavior of rd for different values of
space. d shown in Fig. 8 we can suggest the intrinsic dimension
for the universal motor is most likely 2; i.e. the initial space
of 8 parameters can, given these optimization conditions, be
We use the well known NSGA-II [Deb, 2001] evolution- reduced to two incipient “symbols”.
ary algorithm, with population size 2000, and probability of
crossover 0.8, mutation probability 0.33 and 0.1 (for real/ bi-
nary). The estimated pareto front for maximizing both the
4 Conclusion
torque (πtorque ) and efficiency (πef f iciency ) while minimiz- The main contributions of this work is the proposal that non-
ing the mass (πmass ) is shown in Fig. 7(a). The designs in linear manifold learning may constitute an important step in
this non-dominated front in objective space are identified in discovering latent relationships among the many parameters
the original 8-parameter design space. We now attempt to that define how the world works. A key constraint is our in-
see if these 8-D points actually constitute a lower dimension- complete characterization of the situations in which such a
ality manifold, by considering the reconstruction error when lower-dimensional characterization would exist.
mapped to differing dimensionalities from 2 to 8 (figure Fig. Among the work that would need to be done next is to the
8; the sharp knee at 2 indicates considerable information ab- conjoints of more than one symbol; i.e. given the design ele-
straction, and Fig. 7(b) shows the mapping to a 2-dimensional ments each as an individual symbol, we need to be able to say
space obtained by LLE. This mapping reveals that neighbours what the conjunction of these elements (the syntax) will do,
in the high dimensional space remain nearby in the lower- and whether the resulting object - a design instance - will be
dimensional space at least for this universal motor problem. adequate to meet the design task or not. Again, depending on
The results here signify that for the universal motor, ob- the “good designs” that emerge in the process, a combination
taining the FFR as a 2-dimensional non-dominated surface in of symbols may come to be designated as a symbol on its own
objective space can lead to a dimensionality reduction to 2 in right, leading to the birth of abstract symbols.
33
The argument presented here implies that in the long run, [Langacker, 1986] Ronald W. Langacker. An introduction to cog-
to create viable computer vocabularies for design or AI, we nitive grammar. Cognitive Science, (10):l–40, 1986.
must train the systems to learn these relationships, by expe- [Mandler, 2004] Jean Matter Mandler. Foundations of Mind : Ori-
riencing many design and real world situations. This may be gins of conceptual thought. Oxford University Press, New York,
done in an accelerated manner, but the system must be ex- 2004.
posed to something like the vast array of experiences of a [Martin and Backer, 2005] S Martin and A Backer. Estimating
human - or possibly many more, since the abstraction pro- manifold dimension by inversion error. In ACM Symposium on
cesses as computationally available today may not be as effi- Applied Computing, pages 22–26, 2005.
cient. As different systems are deployed in solving different [McDonough et al., 2003] L. McDonough, S. Choi, and J.M. Man-
problems, their somewhat differing input sets would result in dler. Understanding spatial relations: Flexible infants, lexical
somewhat different abstractions for the same symbols. These adults. Cognitive Psychology, 46(3):229–259, 2003.
resulting design agents may therefore be somewhat less pre- [Moss et al., 2004] J. Moss, J. Cagan, and K. Kotovsky. Learning
dictable than current computers, but such is the price of flex- from design experience in an agent-based design system. Re-
ibility. search in Engineering Design, 15(2):77–92, 2004.
[Nanda et al., 2007] J Nanda, H.J. Thevenot, T.W. Simpson, R.B.
References Stone, M. Bohm, and S.B. Shooter. Product family design knowl-
[Belkin and Niyogi, 2002] M. Belkin and P. Niyogi. Laplacian edge representation, aggregation, reuse, and analysis. AI EDAM,
Eigenmaps and Spectral Techniques for Embedding and Cluster- 21(02):173–192, 2007.
ing. Advances in Neural Information Processing Systems, 1:585– [Pahl and Beitz, 19881996] G Pahl and W Beitz. Engineering
592, 2002. Design:A Systematic Approach, pages 199–400. The Design
[Bishop, 2006] C.M Bishop. Pattern recognition and machine Council/Springer-Verlag, London/Berlin, 1988/1996.
learning. Springer, 2006. [Park and Gero, 1999] SH Park and J.S. Gero. Qualitative repre-
[Bohm et al., 2005] M.R. Bohm, R.B. Stone, and S. Szykman. En- sentation and reasoning about shapes. In Visual and Spatial Rea-
soning in Design, volume 99, pages 55–68, 1999.
hancing Virtual Product Representations for Advanced Design
Repository Systems. Journal of Computing and Information Sci- [Richard and Dixon, 1994] Welch V Richard and John R Dixon.
ence in Engineering, 5:360, 2005. Guiding conceptual design through behavioural reasoning. Re-
search in Engineering, 6:169–188, 1994.
[Casasola et al., 2003] Marianella Casasola, Leslie B. Cohen, and
Elizabeth Chiarello. Six-month-old infants’ categorization of [Roweis and Saul, 2000] S.T. Roweis and L.K. Saul. Nonlinear Di-
containment spatial relations. Child Development, 74:679–693, mensionality Reduction by Locally Linear Embedding, 2000.
2003. [Sarkar et al., 2008] Somwrita Sarkar, Andy Dong, and John S
[De Saussure, 19161986] F. De Saussure. Course in general lin- Gero. A learning and inference mechanism for design optimiza-
guistics. Open Court, 1916/1986. tion problem (re)-formulation using singular value decomposi-
tion. In Proceedings of DETC’08, ASME Design Engineering
[Deb, 2001] Kalyanmoy Deb. Multi-Objective imization using Evo- Technical Conferences and Computers and Information in Engi-
lutionary Algorithms. Chichester, John Wiley and Sons, Ltd., 1 neering Conference, August 2008.
edition, 2001.
[Saul and Roweis, 2003] L K Saul and S T Roweis. Think globally,
[Gero and Fujii, 2000] JS Gero and H. Fujii. A computational fit locally: unsupervised learning of low dimensional manifolds.
framework for concept formation for a situated design agent. The Journal of Machine Learning Research, 4:119–155, 2003.
Knowledge-Based Systems, 13(6):361–368, 2000.
[Simpson, 1998] Timothy W. Simpson. A Concept Exploration
[Gladwell et al., 2005] M. Gladwell, H. Finder, and C-SPAN (Tele- Method for Product Family Design. PhD thesis, Georgia Tech
vision network. Blink: The power of thinking without thinking. University, Dept Mechanical Engineering, 1998.
Penguin Books, 2005. [Szykman et al., 2001] S. Szykman, R.D. Sriram, and W.C. Regli.
[Gobet et al., 2001] F. Gobet, P.C.R. Lane, S. Croker, P.C.H. The role of knowledge in next-generation product development
Cheng, G. Jones, I. Oliver, and J.M. Pine. Chunking mechanisms systems. Journal of computing and information Science in Engi-
in human learning. Trends in Cognitive Sciences, 5(6):236–243, neering, 1:3, 2001.
2001. [Tenenbaum et al., 2000] J.B. Tenenbaum, V. Silva, and J.C. Lang-
[Goldschmidt, 2003] G. Goldschmidt. The backtalk of self- ford. A global geometric framework for nonlinear dimensionality
generated sketches. Design Issues, 19(1):72–88, 2003. reduction, 2000.
[Gross, 1986] Mark Donald Gross. Design as Exploring Con-
straints. PhD thesis, Department of Architecture ,Massachusetts
Institute of Technology, February 1986.
[Hirtz et al., 2002] J. Hirtz, R.B. Stone, D.A. McAdams, S. Szyk-
man, and K.L. Wood. A functional basis for engineering design:
Reconciling and evolving previous efforts. Research in Engineer-
ing Design, 13(2):65–82, 2002.
[Lakoff and Johnson, 1999] George Lakoff and Mark Johnson. Phi-
losophy in the Flesh: The embodied mind and its challenge to
Western thought. Basic Books, New York, 1999.
34