=Paper=
{{Paper
|id=Vol-1388/DeCat2015-paper2
|storemode=property
|title=Creativity Profiling Server: Modelling the Principal Components of Human Creativity over Texts
|pdfUrl=https://ceur-ws.org/Vol-1388/DeCat2015-paper2.pdf
|volume=Vol-1388
|dblpUrl=https://dblp.org/rec/conf/um/PanagopoulosKKK15
}}
==Creativity Profiling Server: Modelling the Principal Components of Human Creativity over Texts==
https://ceur-ws.org/Vol-1388/DeCat2015-paper2.pdf
Creativity Profiling Server: Modelling the Principal
Components of Human Creativity over Texts
George Panagopoulos, Pythagoras Karampiperis, Antonis Koukourikos, Sotiris Kon-
stantinidis
Computational Systems & Human Mind Research Unit, Institute of Informatics &
Telecommunications, National Center for Scientific Research “Demokritos”, Greece
{gpanagopoulos,pythk,kukurik,skonstan}@iit.demokritos.gr
Abstract. Within the field of Computational Creativity, significant effort has
been devoted towards identifying variegating aspects of the creative process
and constructing appropriate metrics for determining the degree that an artefact
exhibits creativity with respect to these aspects. However, the formalization of a
person’s creativity (i.e. a creativity user profile) as a derivative of such crea-
tions is not straightforward, as it requires a transition to a space reflecting the
core principles of creativity as perceived by humans. This becomes a necessity
in domains where personalization goes beyond timely and personalized
knowledge provision, targeting the encouragement and fostering of creative
thinking. Thus, it becomes essential to develop methodologies for modelling
creativity to support personalization based on creativity aspects / characteristics
of users. The paper proposes a user modelling framework for formulating crea-
tivity user profiles based on an individual’s creations, by transitioning from tra-
ditional computational creativity metrics to a space that adheres to the principal
components of human creativity. Furthermore, the paper presents the Creativity
Profiling Server (CPS), a system implementing the aforementioned user model-
ling framework for computing and maintaining creativity profiles and showcas-
es the results of experiments over storytelling educational activities.
Keywords: Human Creativity Modelling, Creativity Profiling, Computational
Creativity
1 Introduction
Human creativity is a multifaceted, vague concept, combining undisclosed or para-
doxical characteristics. As a general notion, creativity adheres to the ability to move
beyond traditional and established patterns and associations, by transforming them to
new ideas and concepts or using them in innovative, unprecedented contexts and set-
tings [1]. The usage of computational methods for producing creative artefacts, as
well as, unveiling the essence of human creativity and using computers understanding
it, is the subject of extensive debate [2]. Along with such philosophical approaches,
research results from neuroscience should also be considered in the process of reveal-
ing/ understanding the human creative process. In general, the creativity of a person
can be expressed qualitatively by taking into account its origin in psychometric or
cognitive aspects of their thinking process [3]. An example of the former is the work
of [4], who examine how the human mind perceives complex auditory stimuli e.g.
music. In this case, they look at the brains of improvising musicians and study what
parts of the brain are involved in the kind of deep creativity that happens when a mu-
sician is really in the groove. Their research has deep implications for the understand-
ing of creativity of all kinds. In any case, while machines can mimic human creativity,
or provide the necessary stimuli for encouraging and promoting the production of
creative ideas and artefacts, it is not straightforward to assess the exhibited creativity
by using automated techniques. Rather, most efforts have been focused on analyzing
creativity on different aspects and producing different metrics, based on the nature of
the examined artefacts.
Hence, the core assumption for building a user’s creativity profile, is that his/her
creativity is showcased by his/her creations, named Creativity Exhibits. These exhib-
its can follow different modalities, corresponding to the aforementioned reasoning
patterns, e.g. texts, diagrams/pictures, actions etc.
The calculation of a creativity profile, constitutes the process of (a) measuring the
creativity expressed by given creativity artifacts; (b) associating these measurements
with dimensions of human creativity corresponding to the given dimension.
For achieving (a), we employ creativity metrics derived from computational crea-
tivity and formulate them in accordance to the characteristics of the examined exhib-
its. A number of different creativity metrics are proposed from the literature on com-
putational creativity.
More specifically, Novelty reflects the deviation from existing knowledge/ experi-
ence and can be measured as a difference metric between what is already known and
the given piece of content. Novelty is a generally accepted dimension of creativity
within the area of computational creativity and an essential candidate for measuring
elements of creativity within the human-created content when interacting with the
machine. It has been used as a heuristic for driving the generation of novel artefacts in
exploratory creativity [3] known as novelty search, an approach to open-ended evolu-
tion in artificial life [5]. Surprise is another essential characteristic which may be
represented as the deviation from the expected [6]. The higher the deviation the high-
er the perceived surprise. Surprise offers a temporal dimension to unexpectedness [7].
Likewise, impressive artefacts readily exhibit (ease of recognition) significant design
effort and may be described via two heuristics, Rarity (rare combination of properties)
and Recreational Effort (difficult to achieve) [8]. These four metrics will be used to
construct the creativity profile of a human user, as expressed by the artefacts that this
user has been constructed alone or as a participating member of a group of users. In
the case of Textual Exhibits, examples of such artefacts include a written story, a
dialogue and any other textual creation.
In our previous work [9] we presented the formulization of the Computational Cre-
ativity Metrics for Novelty, Surprise, Rarity and Recreational Effort over textual arte-
facts. In the present work, we use these text-based metrics for the core aspects of crea-
tivity and examine their conformance with the human perception of what constitutes a
creative artefact. We proceed to identify the deviations between these two perspec-
tives (computational metrics and human judgment) and propose a model for trans-
forming the automatic measures to a space that more accurately reflects the human
opinion. In this way, the constructed human creativity profiles can be used for provid-
ing personalized material / content that is suitable for a specific user or addresses
his/her limitations regarding creativity.
The rest of the paper is structured as follows. We examine the correlation of the
proposed metrics with the human perception of creativity. Afterwards, we build on
these observations to propose a transition model from computational metrics to a two-
dimensional orthogonal space which aims to closely reflect the way human beings
perceive creativity. We present the experiments for assessing the effectiveness of the
proposed model towards this goal, describe the architecture and functionality of the
Creativity Profiling Server, a system that incorporates the proposed model and report
on the experiments for a preliminary evaluation of the system. Finally, we summarize
the present research and report on our next steps.
2 Correlation of Computational Creativity Metrics With the
Human Perception of Creativity
In order to assess the adherence of the proposed metric formulization with the human
perception for creativity, we organized and conducted an experimental session based
on storytelling activities. For the execution of the experiment, we employed forty (40)
human participants, split in ten (10) teams of four (4) members each. All teams were
asked to construct a story, on a specified premise, the survival of a village’s habitants
under a ravaging snow storm. The stories were created incrementally, with twenty
(20) fragments produced for each story.
Following the completion of the stories, the teams were organized in two groups,
each consisting of five teams. Without any interaction between the groups, each team
was called to rate the stories of the remaining four teams belonging to their group,
using a rank-based 4-star scale (i.e. the best story received 4 stars, the second-best
story received 3 stars etc.). In this way, we obtained a ranked list of the five stories in
each group. The goal of our experiment was to determine if, using the ranked lists of
one of the test groups and a formalized representation of the computational creativity
metrics, we can identify their correlation and examine if the distribution of values for
the metrics follow the pattern of human judgment. To this end, we define a con-
strained optimization problem over functions of the aforementioned metrics, which is
described below.
2.1 Extracting a Model for the Human Perception of Creativity
Each artefact (story) 𝑆𝑆𝑛𝑛 is characterized (via the application of the computational
creativity metrics presented in the previous section) [9] by a set of 4 independent
properties 𝑔𝑔 𝑆𝑆𝑛𝑛 = (𝑔𝑔1 𝑆𝑆𝑛𝑛 , 𝑔𝑔2 𝑆𝑆𝑛𝑛 , 𝑔𝑔3 𝑆𝑆𝑛𝑛 , 𝑔𝑔4 𝑆𝑆𝑛𝑛 ) where 𝑔𝑔1 stands for “Novelty”, 𝑔𝑔2 for “Sur-
prise”, 𝑔𝑔3 for “Rarity” and 𝑔𝑔4 for “Recreational Effort”. We define as partial creativi-
ty function (PCF) related to artefact property 𝑔𝑔𝑘𝑘 a function that indicates how im-
portant is a specific value of the property 𝑔𝑔𝑘𝑘 when calculating the creativity of an
artefact 𝑆𝑆𝑛𝑛 . This function is defined by the following formula:
𝑐𝑐𝑔𝑔𝑘𝑘 ∗�1−𝑑𝑑𝑔𝑔𝑘𝑘 � 𝑑𝑑𝑔𝑔𝑘𝑘
𝑃𝑃𝑃𝑃𝑃𝑃𝑔𝑔𝑘𝑘 (𝑔𝑔𝑘𝑘 𝑆𝑆𝑛𝑛 ) = 𝑤𝑤𝑔𝑔𝑘𝑘 ∗ � 2 + �, where 𝑔𝑔𝑘𝑘 𝑆𝑆𝑛𝑛 ∈ [0,2]is the value of
� 𝑎𝑎 ∗𝑔𝑔 𝑆𝑆𝑛𝑛 + 𝑏𝑏𝑔𝑔 � 2
𝑒𝑒 𝑔𝑔𝑘𝑘 𝑘𝑘 𝑘𝑘
property 𝑔𝑔𝑘𝑘 for the artefact 𝑆𝑆𝑛𝑛 , and 0 ≤ 𝑎𝑎𝑔𝑔𝑘𝑘 ≤ 5, −4 ≤ 𝑏𝑏𝑔𝑔𝑘𝑘 ≤ 4, 0 ≤ 𝑐𝑐𝑔𝑔𝑘𝑘 ≤ 1,
0 ≤ 𝑑𝑑𝑔𝑔𝑘𝑘 ≤ 2 are parameters that define the form of the partial creativity function,
whereas 0 ≤ 𝑤𝑤𝑔𝑔𝑘𝑘 ≤ 1 represents the weight of property 𝑔𝑔𝑘𝑘 in the calculation of the
overall creativity. The calculation of the above parameters for all 𝑔𝑔𝑘𝑘 properties lead to
the calculation of the complete creativity function (CCF), as the aggregation of the
1
partial creativity functions, as follows: 𝐶𝐶𝐶𝐶𝐶𝐶(𝑔𝑔 𝑆𝑆𝑛𝑛 ) = ∗ ∑4𝑘𝑘=1 𝑃𝑃𝑃𝑃𝑃𝑃𝑔𝑔𝑘𝑘 (𝑔𝑔𝑘𝑘 𝑆𝑆𝑛𝑛 )
4
If 𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆1 is the complete creativity of an artefact 𝑆𝑆1 and 𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆2 is the complete crea-
tivity of an artefact 𝑆𝑆2 , then the following properties generally hold for the complete
creativity function:
𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆1 > 𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆2 ⇔ (𝑆𝑆1 )𝑃𝑃(𝑆𝑆2 )
𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆1 = 𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆2 ⇔ (𝑆𝑆1 )𝐼𝐼(𝑆𝑆2 )
where P is a strict preference relation and I is an indifference relation, as perceived by
humans when evaluating the creativity of these artefacts.
Given a preference ranking of a reference set of artefacts, we define the creativity
differences 𝛥𝛥 = �𝛥𝛥1 , 𝛥𝛥2 , … , 𝛥𝛥𝑞𝑞−1 �, where q is the number of artefacts in the reference
set and 𝛥𝛥𝑖𝑖 = 𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆𝑖𝑖 − 𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆𝑖𝑖+1 ≥ 0 is the creativity difference between two subsequent
artefacts in the ranked set.
We then define an error parameter 𝐸𝐸 for each creativity difference:
𝛥𝛥𝑖𝑖 = 𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆𝑖𝑖 − 𝐶𝐶𝐶𝐶𝐶𝐶𝑆𝑆𝑖𝑖+1 + 𝐸𝐸𝑖𝑖 ≥ 0
We can then solve the following constrained optimization problem:
𝑞𝑞−1
𝛥𝛥 ≥ 0, 𝑖𝑖𝑖𝑖 (𝑆𝑆𝑖𝑖 )𝑃𝑃(𝑆𝑆𝑖𝑖+1 )
𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 �(𝐸𝐸𝑖𝑖 )2 𝑠𝑠. 𝑡𝑡. � 𝑖𝑖
𝛥𝛥𝑖𝑖 = 0, 𝑖𝑖𝑖𝑖 (𝑆𝑆𝑖𝑖 )𝐼𝐼(𝑆𝑆𝑖𝑖+1 )
𝑖𝑖=1
This optimization problem leads to the calculation of the partial creativity function
parameters for each property 𝑔𝑔𝑘𝑘 .Based on these values and the human assessment of
the story rankings, the results of the constrained optimization problem defined in the
previous section resolves in the calculation of the partial creativity parameters (a, b, c,
d and w). Regarding the impact of the various metrics in the computation of the over-
all creativity, we observed that Novelty is generally considered a particularly positive
attribute creativity-wise for the stories, its partial creativity (PC) increasing as its val-
ue increases (see Figure 1). In contrast, the remaining metrics reached their maximum
partial creativity at a certain value, after which their partial creativity started to de-
crease, indicating that e.g. recreational effort greater than a certain point is not per-
ceived as a direct indication of creativity (see Figure 1).
Group A: Group A: Group A: Group A: R.
Novelty Surprise Rarity Effort
1.00 1.00 1.00 1.00
0.50 0.50 0.50 0.50
0.00 0.00 0.00 0.00
0.00 1.00 2.00 0.00 1.00 2.00 0.00 1.00 2.00 0.00 1.00 2.00
Group B: Group B: Group B: Group B: R.
Novelty Surprise Rarity Effort
1.00 1.00 1.00 1.00
0.50 0.50 0.50 0.50
0.00 0.00 0.00 0.00
0.00 1.00 2.00 0.00 1.00 2.00 0.00 1.00 2.00 0.00 1.00 2.00
Fig. 1. PCs of Computational Creativity Metrics wrt their value (Group A & B respectively)
Hence, the obtained results indicate that, while the proposed computational creativity
metrics are correlated with the perception of humans for creativity, this correlation is
not direct for all metrics. The following section discusses on the implications of these
observations and details our approach for using the proposed metrics towards building
a dimensional plane that more accurately reflects the human perspective for creativity.
3 Transferring Computational Creativity Metrics to the
Human Perspective
As stated, each textual artefact can be described by 4 computational creativity met-
rics, namely, Novelty, Surprise, Rarity and Recreational Effort. Following the formu-
lation of the creativity metrics, therefore, the next hypothesis that was examined was
the reduction of the dimensional space for representing creativity as expressed
through creative artefacts, in an orthogonal space. In order to effectively conceptual-
ize human creativity, orthogonality is a particularly desirable attribute of the concep-
tualization space to be used, since it allows the examination of independent variables
when trying to analyse and influence / encourage certain creativity aspects. Hence, the
first step towards identifying the adherence of the computational creativity metrics
with the human perspective is to examine the orthogonality of the proposed metrics
formulation. To this end, we ran an experiment for calculating the four basic compu-
tational creativity metrics on two datasets derived from distinct and distant domains,
and determined whether the four metrics are orthogonal.
The first dataset comprised transcriptions of European Parliament Proceedings
[10]. Given the formulation of computational creativity metrics described in [9], we
consider as a “story” the proceedings of a distinct Parliament session and as a frag-
ment the speech of an individual MP within the examined session. The second dataset
was derived from a literary work, Stories from Northern Myths, by E.K. Baker, avail-
able via the Project Gutenberg collection. In this case, the story is a book chapter and
the story fragment is a paragraph within the chapter.
Table 1. Computational Metrics Correlation: Formal Verbal Transcriptions
Novelty Surprise Rarity R. Effort
Novelty 1.00000 0.13393 0.12329 -0.40681
Surprise 0.13393 1.00000 0.26453 -0.43151
Rarity 0.12329 0.26453 1.00000 -0.33499
R. Effort -0.40681 -0.43151 -0.33499 1.00000
Table 2. Computational Metrics Correlation: Literary Work
Novelty Surprise Rarity R. Effort
Novelty 1.00000 -0.64243 0.10392 -0.10762
Surprise -0.64243 1.00000 0.07376 -0.02538
Rarity 0.10392 0.07376 1.00000 -0.03882
R. Effort -0.10762 -0.02538 -0.03882 1.00000
In total, we examined 50 distinct parliament sessions from the Europarl dataset and
40 chapters from the storybook. Based on the obtained results, we calculated the cor-
relation between the four computational creativity metrics. Tables 1 and 2 provide the
correlation values between the four metrics. It is evident that the computational crea-
tivity metrics by themselves are not orthogonal. In order to better approximate the
human perception for creativity, we propose the following abstraction for modelling
the examined aspects of creativity to a space more closely resembling human think-
ing:
Novelty is the perspective to be held as the one dimension of the dimensional
space, as the conducted showed that it has a monotonic incremental relation with the
perception of humans on what is creative. Further more, it is a generally accepted
dimension of creativity. [11]
Atypicality, that is, the tendency to deviate from the norm without actually break-
ing through. In other words, to what extend (without necessarily being novel) the
artefact differs from the ordinary (thus being surprising, rare and difficult to con-
struct)
We consider Atypicality as a combination of the Surprise, Rarity and Recreational
Effort metrics, each bearing a different weight towards determining Atypicality.
These two axes also provide a rough conceptualization of the two major qualitative
aspects of creative work: whether the said work is visionary, i.e. it provides a ground-
breaking approach on a given field; and whether it is constructive, i.e. it uses in a
novel way established techniques and ideas in order to produce a high-quality artefact.
As stated, Novelty has an analogous and close to monotonic association with the hu-
man judgment for creativity. Therefore, and in order to satisfy our requirement of
orthogonality, we consider Novelty as the strictly defined dimension of our space and
seek for the formulation of Atypicality that results to a dimension orthogonal to Nov-
elty.
More specifically, let Atypicality of a text 𝑡𝑡 be the normalized weighted sum of its
𝑤𝑤 𝑆𝑆𝑆𝑆𝑆𝑆(𝑡𝑡)+𝑤𝑤𝑟𝑟 𝑅𝑅𝑅𝑅𝑅𝑅(𝑡𝑡)+𝑤𝑤𝑒𝑒 𝐸𝐸𝐸𝐸𝐸𝐸(𝑡𝑡)
Surprise, Rarity, and Recreational Effort: 𝐴𝐴(𝑡𝑡) = 𝑠𝑠 , with
𝑤𝑤𝑠𝑠 +𝑤𝑤𝑟𝑟 +𝑤𝑤𝑒𝑒
𝑤𝑤𝑠𝑠 , 𝑤𝑤𝑟𝑟 , 𝑤𝑤𝑒𝑒 ∈ [−1,1] . We aim to find the weight values that constitute Atypicality
orthogonal to Novelty, i.e. those weight values for which Correl(Novelty, Atypicali-
ty)= 0. We thus define the following optimization problem:
𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 ∑𝑛𝑛𝑖𝑖=1( 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶(𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑖𝑖 , 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝑖𝑖 ) )2 , s.t. 𝑤𝑤𝑠𝑠 , 𝑤𝑤𝑟𝑟 , 𝑤𝑤𝑒𝑒 ∈ [−1,1]
where n is the number of the combined datasets.
Although the search space of the optimization problem above is highly non-linear
solving this problem is straightforward. The resulting model defines two orthogonal
axes, Novelty and Atypicality, which define the space for measuring and characteriz-
ing the observed creativity, as an Euclidean vector, the length of which indicates the
quantitative aspect of the creativity exhibited by the artefact, while its direction indi-
cates the tendency for either Novelty (visionary creativity) or Atypicality (construc-
tive creativity). The following tables present the novelty and atypicality in the two
datasets, as well as, the correlation between these two dimensions for the found opti-
mum weight values.
Table 3. Correlation of Creativity Dimensions: Formal Verbal Transcription
Novelty Atypicality
Novelty 1.00000 2.986E-07
Atypicality 2.986E-07 1.00000
Table 4.Correlation of Creativity Dimensions: Literary Work
Novelty Atypicality
Novelty 1.00000 1.436E-07
Atypicality 1.436E-07 1.00000
4 The Creativity Profiling Server
The Creativity Profiling Server (CPS) allows the storage, maintenance and update of
creativity profiles of users using creativity exhibits that are produced from applica-
tions of the outside world. CPS provides a simple and straightforward API in order to
expose its functionalities and to facilitate the communication with the outside world.
Through the CPS API, the example application can submit creativity exhibits and
receive the corresponding creativity measurements, create group of users and finally
receive the creativity profile of a user. The aforementioned functionalities and the
internal structure of CPS are depicted in Figure 2.
Computational
Creativity Metrics
Calculator
Submit Creativity Exhibit
Computational Creativity Measurements
Creativity Exhibit Creativity Evidence &
Creativity
Model Controller Measurements
User Validation
Exhibits
Application
Validation
Creativity Exhibits
CPS API
Analysis
Create User
User Identifier Application
Validation Client Application Machine Learning
Group Users User Manager
Validator Components
Group Identifier
Application
Validation
User Profile
Update
User Validation
Get User Profile Creativity User Creativity User Profiles &
Creativity
Modelling Controller Properties
User Profile Properties User Profiles
C2Learn Creativity Profile Server (CPS)
Fig. 2. CPS Architecture
The distinct modules incorporated in the CPS Architecture are the following:
• Client Application Validator: The module is responsible for ensuring that a client
request is originated from an application registered to CPS.
• User Manager: This module is responsible for ensuring that client requests contain
a valid user profile ID. Also User Manager is responsible for creating and destroy-
ing groups by joining and disjoining user profile properties respectively
• Creativity Exhibit Model Controller: This module is responsible for storing, main-
taining and updating the creativity exhibits delivered by applications and also for-
ward the creativity exhibits to the Computational Creativity Metrics Calculator:
This module is responsible for calculating all the metrics of a creativity exhibit re-
garding of its type.
• Creativity User Modelling Controller: This module is responsible for storing,
maintaining and updating the Profile Properties of each User Profile in CPS. Also
this module delivers to client applications the properties of particular user profiles.
• Machine Learning Components: This module is responsible for calculating the
value of the Creativity Profile Properties of a given user.
In a typical situation an application creates a user through the CPS API. The CPS API
send the request to the User Management. Afterward User Management verifies
through the Application Validation module that the application is registered to CPS.
Since the application is validated User Management creates a unique user profile id
and sends it to the application. Since a user profile is created then the application can
submit creativity exhibits of this user. More specifically the application submits the
creativity exhibit to the CPS API along with type of the creativity exhibit and the
timestamp the creativity exhibit was created. After submission the API sends the crea-
tivity exhibit and its type to the Creativity Exhibit Model Controller module. After
validating the user and the application through the User Management and the Appli-
cation Validator respectively, the module sends the creativity exhibit to the Computa-
tional Creativity Metrics Calculator module. The Computational Creativity Metrics
Calculator returns back the measurements of the creativity exhibit. Afterwards, the
Creativity Exhibit Model Controller module stores the creativity exhibit along with
the measurements to the CPS database. Finally, the Creativity Exhibit Model Control-
ler invokes the Machine Learning Components. Machine Learning Components take
as input the creativity exhibit and calculate the values of the profile properties of the
user. Afterwards the newly calculated values are send to the Creativity User Model-
ling Controller module, which stores the values to the CPS database.
Once a user creativity profile is created, then the application can request through
the CPS API the User Profile Properties and also the Model which describes the pro-
file. After sending the request to the API, the request is redirected to the Creativity
User Modelling Controller module. This module, after validating the user and the
application through the User Management and the Application Validator respectively,
retrieves from the CPS database the properties for the corresponding user and send
them back to the application.
5 Incorporation of the model in CPS
Following the definition of the model, we combine within CPS the Surprise, Rarity
and Recreational Effort metrics in order to form another metric, which we call Atypi-
cality and is orthogonal to Novelty. Atypicality is calculated as a weighted average of
𝑤𝑤 ∗𝑆𝑆 +𝑤𝑤 ∗𝑅𝑅 +𝑤𝑤 ∗𝐸𝐸
Surprise, Rarity and Recreational Effort, as follows: 𝐴𝐴𝑇𝑇𝑖𝑖 = 𝑠𝑠 𝑖𝑖 𝑅𝑅 𝑖𝑖 𝐸𝐸 𝑖𝑖, where:
𝑤𝑤𝑆𝑆 +𝑤𝑤𝑅𝑅 +𝑤𝑤𝐸𝐸
i refers to an artifact, Si , Ri ,Ei and ATi to Surprise, Rarity, Recreational Effort and
Atypicality metrics respectively for the given artifact i, and wS,wR and wE are posi-
tive weights assigned to Surprise, Rarity and Effort respectively, in order to calculate
the Atypicality metric in a way as much uncorrelated (and thus, orthogonal) with
Novelty as possible. A user’s Creativity Profile, thus, consists of a two-dimensional
vector expressing two types of user’s creativity. The Visionary Creativity, which is
measured by the Novelty metric, and the Constructive Creativity, which is measured
by the Atypicality metric. CPS gathers all Creativity Exhibits (artefacts) that are pro-
duced by its users within external applications. In discrete time intervals, which we
call Time Window, CPS calculates and/or updates the Creativity Profile of each user.
The calculation of the creativity profiles for the users of the CPS is a repeated (once
per Time Window) two-phase process, and is explained below:
Phase A: Calculation of optimum Computational Creativity Metric Weights for the
Application Domain
We aim to find/ update the weight values [wS,wR,wE] of Surprise, Rarity and Rec-
reational Effort that constitute Atypicality orthogonal to Novelty, i.e. those weight
values for which Correl(N, AT) = 0. The optimum vector [wS,wR,wE] will be used in
Phase B for the calculation of the users’ Creativity Profiles for the new CPS Time
Window.
We thus define the following non-linear optimization problem:
𝑀𝑀𝑀𝑀𝑀𝑀. 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶(𝑁𝑁, 𝐴𝐴𝐴𝐴)2 , 𝑠𝑠𝑠𝑠. 𝑤𝑤𝑆𝑆 , 𝑤𝑤𝑅𝑅 , 𝑤𝑤𝐸𝐸 ≥ 0 , 𝑤𝑤𝑆𝑆 + 𝑤𝑤𝑅𝑅 + 𝑤𝑤𝐸𝐸 ≠ 0
Each time where a new CPS Time Window starts, we solve the above minimization
problem for all the artefacts of the application domain (all the creativity exhibits col-
lected for all CPS users and for all CPS Time Windows so far). It is evident that in
each execution of this process there is a strong probability of discovering a new vec-
tor [wS,wR,wE] that makes Atypicality (AT) more orthogonal to Novelty (N). In
order to reduce the sensitivity of the system to this continuous change, we update the
vector [wS,wR,wE] to be used in Phase B with the new vector retrieved by solving
the optimization problem defined in Eq. 1 only when the improvement (minimization)
in Correl(N, AT)2 exceeds 5%.
Phase B: Construct/update of Users’ Creativity Profiles
A user’s creativity profile is determined by the creativity exhibits produced by the
user alone or as a member of a group. Groups are treated by CPS as a user, meaning
that CPS will construct a creativity profile also for each group. In this case, the crea-
tivity profile is constructed/ updated based on the creativity exhibits of the group dur-
ing the last (just finished) time window. In the case of simple users (not groups) their
creativity profile is constructed/ updated based on all the creativity exhibits they con-
structed (either alone or as a group member). The first step for computing the creativi-
ty profiles is to transform the space (N,S,E,R) to the space (N, AT) and compute the
average of N and AT measures for the creativity exhibits for a given user and for the
time window that just finished, as follows:
B1. Calculate Average Novelty and Atypicality of Creativity Exhibits
In the general case, let a user U which participates in groups UG. In the case of
computing the creativity profile of a group, we have only the user U, which represents
the group. Such a user cannot be part of other groups. Let
����������� , ���������������
𝐸𝐸𝑇𝑇 ≡ [𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 ] of a user U, calculated for the creativity exhibits in the
time window T, after the transformation of the space (N,S,E,R) to the space (N, AT)
using the optimal weight vector [wS,wR,wE] (calculated in Phase A). Let also
�����������, ���������������
𝐺𝐺𝑇𝑇 ≡ [𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 ] of a user U, calculated for the creativity exhibits of UG
in the time window T, after the transformation of the space (N,S,E,R) to the space (N,
AT) using the optimal weight vector [wS,wR,wE] (calculated in Phase A).
The overall Average Novelty and Atypicality (PT) of all creativity exhibits for user
U is calculated as a fusion of ET and GT, relying on the analogy of the user’s and the
groups’ achievements. If the user’s creativity (ET) surpasses the creativity exhibited
within his/her participation in groups (GT), then only ET is considered. Otherwise, a
part of the difference between groups’ creativity and user’s creativity is also consid-
ered, as follows:
𝐸𝐸 𝐸𝐸𝑇𝑇 ≥ 𝐺𝐺𝑇𝑇 1 1
𝑃𝑃𝑇𝑇 = � 𝑇𝑇 , with 𝑘𝑘 = + ∗ 𝑡𝑡𝑡𝑡𝑡𝑡ℎ(2 ∗ [(𝐺𝐺𝑇𝑇 − 𝐸𝐸𝑇𝑇 ) − 1])
𝐸𝐸𝑇𝑇 + 𝑘𝑘 ∗ (𝐺𝐺𝑇𝑇 − 𝐸𝐸𝑇𝑇 ) 𝐸𝐸𝑇𝑇 < 𝐺𝐺𝑇𝑇 2 2
B2. Calculate Visionary and Constructive Creativity of User
Though all exhibits must be taken into account, the recent ones are considered
more important, as they depict the exact current status of the user’s creativity whereas
past exhibits play a less vital role. To give our model an essence of decay through
𝑃𝑃 𝐷𝐷−1
time, we use this formula: 𝐶𝐶𝑇𝑇 = 𝑇𝑇 + ∗ 𝐶𝐶𝑇𝑇−1 , where: 𝐶𝐶𝑇𝑇 is the vector describing
𝐷𝐷 𝐷𝐷
the Creativity of the user (or group) at the time window T, and 𝐶𝐶𝑇𝑇−1 at the time win-
dow T-1 respectively 𝐶𝐶𝑇𝑇 ≡ [𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶, 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖] and
D, a proportional constant of decaying analogous to the timespan.
6 Preliminary CPS Evaluation
In order to obtain a preliminary assessment for the effectiveness of the proposed ap-
proach, we conducted a two-phase experiment in order to determine (a) the degree to
which the selected computational creativity metrics conform to the opinion of experts
regarding the creativity exhibited in a textual artefact and (b) the degree to which the
proposed model for human creativity reflects the opinion of such experts.
For the purposes of the experiment, we employed twenty students who were asked
to produce five stories each under pre-defined topics. For the first stage of the exper-
iment, we sampled the stories produced during the aforementioned story writing ses-
sion, randomly selecting two stories by each student, and asked five experts to rank
them with respect to their creativity, as the latter is perceived by each of these experts.
We then compared the ranking results with the ranking derived from the results pro-
duced by the CPS. For the second stage of the experiment, we picked the complete set
of stories (i.e. five stories) for five of the users and asked from the same five experts
to rank these users with respect to their creativity, using as evidence the produced
stories. We then compared the expert ranking to the one produced by the CPS.
In order to evaluate the similarity between the rankings of the experts and the rank-
ings of the CPS, for the textual exhibits’ and the users’ ranks, we employed a metric
1
based on Kendall’s Tau, defined by the following equation: 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = +
2
𝑁𝑁𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 −𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑
, where 𝑁𝑁𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 stands for the concordant pairs of ranked
𝑛𝑛(𝑛𝑛−1)
exhibits or users, 𝑁𝑁𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 stands for the discordant pairs when comparing the or-
dering of the experts and the CPS and 𝑛𝑛 is the number of the examined exhibits or the
users. We calculated this metric for the series of textual exhibits rankings and the
series of participating users rankings. The following table presents the summary sta-
tistics of the two Success metric series we had as an outcome.
Table 5. Correlation Coefficient between Expert and CPS rankings
Textual Exhibits Users
Min Success 0.58 0.56
Average Success 0.74 0.71
Max Success 0.89 0.88
7 Conclusions & Future Work
The work described in the present paper showcases our findings towards transitioning
from computational creativity metrics associating specific attributes of text artefacts
with creativity aspects to a creativity calculation model that better reflects the human
perception of creativity. Furthermore, the present manuscript provides a summary of
the architectural design and functionality of the Creativity Profiling Server (CPS).
Towards the continuation of our research, we aim to examine the effectiveness of
the model in more complex experiments, examining textual exhibits from different
domains and modalities (prose, poetry, speech) in order to obtain a more general re-
flection of the human perception of creativity. Observation over more open-ended
experiments will likely lead to further refinements and extensions of the proposed
human creativity model.
8 References
1. Zhu, X., Xu , Z., Khot, T.: How creative is your writing? a linguistic creativity measure
from computer science and cognitive psychology perspectives. In Proceedings of the
Workshop on Computational Approaches to Linguistic Creativity, Association for
Computational Linguistics, pp. 87--93, (2009).
2. Lubart, T.:How can computers be partners in the creative process: classification and
commentary on the special issue. In International Journal of Human-Computer Studies,
vol. 63, pp. 365--369, (2005).
3. Boden, M.: The Creative Mind, 2nd Edition. London: Routledge, (2004).
4. Limb, C. J., Braun, A. R.:Neural substrates of spontaneous musical performance: An fMRI
study of jazz improvisation. In PLoS One, vol. 3, (2008).
5. Lehman, J., Stanley, K. O.:Exploiting Open-Endedness to Solve Problems Through the
Search for Novelty. In ALIFE, pp. 329--336, (2008).
6. Macedo, L., Reisenzein, R., & Cardoso, A.: Modeling forms of surprise in artificial agents:
Empirical and theoretical study of surprise functions. In Proceedings of the 26th annual
conference of the cognitive science society, NJ: Erlbaum, pp. 873--878, (2004) .
7. Maher, M. L., Brady, K., Fisher, D. H.:Computational models of surprise in evaluating
creative design. In Proceedings of the Fourth International Conference on Computational
Creativity, (2013).
8. Lehman, J., Stanley, K. O.:Beyond Open-endedness: Quantifying Impressiveness. In
Artificial Life, pp. 75--82, (2012).
9. Karampiperis, P., Koukourikos, A., Panagopoulos, G.:From Computational Creativity
Metrics to the Principal Components of Human Creativity. In Proc. of the 9th
International Conference on Knowledge, Information and Creativity Support Systems,
Limassol, Cyprus, (2014).
10. Koehn, P.:Europarl: A parallel corpus for statistical machine translation. In MT summit,
pp. 79--86, (2005).
11. G. Ritchie.:Some empirical criteria for attributing creativity to a computer program. In
Minds and Machines, vol. 17, pp. 67--99, (2007).
12. Kövecses, Z.:A new look at metaphorical creativity in cognitive linguistics. In Cognitive
Linguistics, vol. 21, pp. 663--697, (2010).
13. Veale, T.:An analogy-oriented type hierarchy for linguistic creativity. In Knowledge-Based
Systems, vol. 19, pp. 471--479, (2006).
14. Lehrer, A.:Brendalicious. Lexical creativity, texts and contexts. Amsterdam, John
Benjamin, , pp. 115-136, (2007).
15. Chiru, C. G.:Creativity Detection in Texts. In ICIW 2013, The Eighth International
Conference on Internet and Web Applications and Services, (2013).