=Paper=
{{Paper
|id=Vol-2395/paper2
|storemode=property
|title=Predicting Culture and Personality in Online Courses
|pdfUrl=https://ceur-ws.org/Vol-2395/paper2.pdf
|volume=Vol-2395
|authors=Sylvio Rüdian,Gergana Vladova,Jana Gundlach,Gunay Kazimzade,Niels Pinkwart
|dblpUrl=https://dblp.org/rec/conf/aied/RudianVGKP19
}}
==Predicting Culture and Personality in Online Courses==
https://ceur-ws.org/Vol-2395/paper2.pdf
Predicting culture and personality in online courses
Sylvio Rüdian Gergana Vladowa Gunay Kazimzade
Humboldt University of Berlin University of Potsdam Technical University of Berlin
Weizenbaum Institute for the Weizenbaum Institute for the Weizenbaum Institute for the
Networked Society Networked Society Networked Society
Berlin, Germany Potsdam, Germany Berlin, Germany
ruediasy@hu-berlin.de gvladova@lswi.de gunay.kazimzade@tu-berlin.de
Jana Gundlach Niels Pinkwart
University of Potsdam Humboldt University of Berlin
Weizenbaum Institute for the Weizenbaum Institute for the
Networked Society Networked Society
Potsdam, Germany Berlin, Germany
janagundlach@uni-potsdam.de pinkwart@hu-berlin.de
ABSTRACT still missing in learning environments. Researchers focus on drop-
Online courses support learners to engage in distance learning. One out rates or final outcomes. This information can be predicted based
emerging trend of the educational community is their personaliza- on clickstream data of participants that was previously collected
tion. Individual cultural characteristics and personality traits that during an online course [1]. Using the data, instructors have the
influence individuals’ behavior in online courses have not yet been ability to help students at risk.
examined in detail. It is often practically impossible to collect a lot By observing the industry, websites have the opportunity to collect
of personal information regarding personality or culture in online clickstream data as well. This can be used to predict demographic
courses. Therefore, it is necessary to fill in a comprehensive ques- data, which allows them to separate users into groups of customers
tionnaire. We show how accurately personality and cultural traits with similar attributes [2]. Marketers use this prediction to optimize
can be predicted by behavior in an online course. The paper reports the process for profit maximization. Online courses can be seen as
exploratory data-informed work. We use a neural network with be- a special category of websites with similar opportunities for opti-
havioral data as input. In case of successful prediction, instructors mization [3]. Instead of maximizing the profit, online courses fol-
can use these items to define targeting groups as a pre step for per- low the aim to teach, apply and test participants for knowledge
sonalization. Our results show, for example, that long-term orien- transfer. Alternatively, the motivation using a learning environment
tation can be predicted best by an individual’s behavior. It corre- could be optimized.
sponds to the ability and attitude of the individual to focus on the
future. Learners with high long-term orientation will spend longer Classical educational recommender systems support users in find-
periods of time in class preparing to successfully complete related ing learning material that could be beneficial for reaching their de-
exercises. We discuss our findings from an interdisciplinary per- sired goals [4]. This is a macro view of personalization as it tries to
spective and propose perspectives for further research on personal- find learning resources that the user potentially is looking for. By
ization. looking at the micro level of personalization we consider single
online courses which could be optimized for individuals. People
Keywords have different personalities, cultural background and learning
styles. Thus we aim to suggest adding new predictable items to a
Personalization, online courses, e-learning, Big Five, personality,
user model that can further be used to personalize online courses at
culture, machine learning
micro level [5].
1. INTRODUCTION Culture is a shared system of values [6] [7] [8]. The recent advance-
Personalization in online courses is a trending topic of the educa- ments in modernization have been identified as erasing cultural dif-
tional community. If we consider lifelong learning, online courses ferences [9]. Increased globalization is anticipated to cause hybrid-
have to provide support for a heterogeneous user base. The diver- ization [9] [10]. Additionally, collecting data on cultural, ethical
sity of learners requires methods for adaption as there is no one- and national belonging may not always be possible or is morally
size-fits-all learning environment. questionable. However, in line with our aim to find out how online
To be able to support learners with the knowledge and skills needed courses can be adjusted to fit the individual user's needs, infor-
to succeed in a rapidly changing world, personalized online learn- mation on national or cultural belonging is of great interest.
ing is one of the fast-growing research directions. Considering the Learning has a strong connection with the culture of individuals and
massiveness of the online learning resources it is essential to inves- groups. Therefore, the educational systems of one country are not
tigate on the impact of culture and personality of the users in their always applicable in another country which has different values,
experience of adapting online learning environments. Therefore, norms and standards [11] [12]. In order to predict culture in an
we need information about culture or personality traits, which are online learning context, we approach the model developed by G. H.
Copyright held by the author(s). Use permitted under the CC-BY
license CreativeCommons.org/licenses/by/4.0/
Hofstede [13] which is identifying "cultural dimensions" which between users’ cultural backgrounds and e-learning systems has
serve as measurement instruments of different cultures [14], and been a topic of research of several researchers.
validated to CVScale which is further applied in this research. The
cultural dimensions applied are Power Distance (PO), Uncertainty There is a strong connection between cultural dimensions and be-
Avoidance (UN), Collectivism (CO), Long- Term Orientation (LT) havior during online learning. For the last two decades, researchers
and Masculinity/Femininity (M). As cultural traits do not often investigated several qualitative and quantitative analysis on the im-
change in life, we have to consider these items for long-live learn- pact of cultural dimensions from G.H. Hofstede [13] to the learning
ing. The tendencies of collectivism, uncertainty avoidance and the and usability, behavior and outcomes of an online learning system.
high power distance of Eastern cultures have been found in online With respect to the impact of culture to offline learning, research of
learning environments [15] [16]. Bonk and Kim’s study [17] shows Liu [24] discusses the intersection of the Hofstede Dimensions and
the dominance of social interactions among Korean students at the the Cultural Dimensions within the context of the Learning Frame-
outset of their online collaboration, which demonstrates their cul- work. The paper refers to Bonk et al. [17] which proposes that the
tural inclination toward emphasizing relationships over tasks. Us- power distance dimension alongside collectivism and uncertainty
ing these items as relative stable factors in life, personalization avoidance leads to the dominance of social interactions and an em-
based on these characteristics can be an advantage because we can phasis of relationships over tasks for Korean students. Additionally,
learn them by using the system without the necessity of a compre- Hofstede [25] refers to a potential heavy reliance on instructors and
hensive questionnaire. textbooks for people with a high power distance dimension.
Instructional design community debates a lot about the impact of Individualism has a strong connection to activeness in class to ex-
personality and culture in the personalized learning construction press themselves, to appreciate diverse opinions in learning, and to
[18] [19] [20] [21]. Personality and culture of the learner has a be self-motivated. Further, the masculinity dimension connects
strong correlation with the different learning styles, efficiency of strongly with the high level of and desire for recognition. Further-
the learner and motivation during the learning process [22]. Thus, more, research that learners who avoid uncertainty are usually pre-
predicting personality type and cultural characteristics of the learn- ferred receiving answers from structured learning activities.
ing can benefit to the customization of online courses with respect
to the design and structure of the online learning materials. McLoughlin [26] states that the flexibility of learners from mixed
cultures in the e-learning systems is often limited. Most of those
The research objective of this paper is to explore the ability for pre- systems are adapted to the specific groups' need, learning style and
diction of culture / personality traits in online courses. Therefore, their learning requirements. Another study from Downey et al. [27]
we examine: Which considered traits can be predicted by behavior focuses on the relationship between national culture and the usabil-
in a linear online course? ity of an e-learning system. They integrate Hofstede’s cultural di-
mensions and Nielsen’s usability attributes into the usability study
The paper is structured as follows. The next section describes re-
lated work according to studies of learning and cultural / personal- of the e-learning materials and highlight the connection between
ity traits. Section 3 describes our methodology, followed by our re- each cultural dimension and its impact on usability.
sults. In section 5 we discuss our results and explain our decisions During the study of [28] with Arab students who were examined
made. Section 6 proposes some ideas for further investigations, fol- during online learning, participants expressed their fear and anxiety
lowed by our conclusion. of taking online courses because they equated online learning with
independent learning which is capturing Arab culture's high uncer-
2. RELATED WORK tainty avoidance [29]. The study of [30] examining Jamaican and
In an experimental study, Makhija et al. [23] explored the links be- Canadian women’s online learning experiences indicates the
tween demographic factors, personality, behavioral engagement groups’ cultural expectations regarding women’s roles in the home
and culture in relation to academic engagement. They used ques- and how it restricts their engagement and learning. Other studies
tionnaires to get dimensions of personality (Big Five Factor Model) [31] emphasized a strong uncertainty avoidance of Chinese stu-
and demographic factors. Cultural information was derived by ask- dents during online learning. They were constantly asking for
ing participants which culture they belong in and was limited to ask “rules and instructions” and if there are any rituals for them to fol-
for the country, where people currently live. Academic engagement low. With respect to usability and design, there have been a lot of
was measured by using variables like received grades and time they studies regarding the impact cultural background of the user to the
spent on completing the tasks. Behavioral engagement was repre- design preferences [32] and usability of the interfaces and online
sented by students’ attendance, participation in class and extracur- systems [33]. Research of Downey (2007) investigates how cultural
ricular activities. dimensions are interconnected with the usability of e-learning sys-
Kloft et al. [1] used clickstream of an online course to predict drop- tems. The study analyzes the cultural dimensions with respect to
outs. With considered scalar features they achieved an accuracy be- learnability, error rate, and user’s satisfaction and exploring the re-
tween 72% and 87%. This study shows that behavior can be used lationship linked to power distance, individualism and collectivism,
to predict dropouts. The resulting information about potential drop- femininity/masculinity and uncertainty avoidance.
outs can help instructors to detect students that aim to drop out of Cultural traits in online courses were investigated on its impact on
the course. communication difficulties [34]. Other studies focused on critical
Cultural background is an important concept with respect to the thinking, harmony, affection, compassion, emotionality, frustra-
way of thinking, performing and learning of a specific group of tion, participation, success and performance [35]. According to
people. Hence, investigating the cultural component in online Strang [36], culture is not cross-related to final grades. But grades
learning and its connection to design patterns of the learning envi- can be predicted based on students’ behavior [37]. Research on re-
ronment is crucial. Inclusivity of e-learning systems allows users lations between culture and the behavior limit culture to the country
across the world to access quality education. Thus, the relationship where participants live [23]. This is a very general view concerning
culture. Hofstede’s cultural dimensions have not been used yet in
online courses and have not been examined regarding personaliza- [23]. However, this short questionnaire does not meet the require-
tion. The is a gap in research. We want to bridge the gap by showing ments of our study for the following reasons: 1) validation or learn-
that cultural dimensions by Hofstede [14] can be predicted due to ing studies are still missing and 2) TIPI cannot provide a faceted
behavioral patterns in online courses. We also want to compare the picture of a single person, which allows the use of the longer ques-
accuracy of predictable items with personality traits and demo- tionnaire (Big Five) [48]. Specifically, Gosling et al. [49] point out
graphic data. that TIPI is "offered for situations where very short measures are
required, personality is not a priority, or researchers can tolerate the
In order to optimize individuals’ learning processes, a lot of infor- somewhat diminished psychometric characteristics of very short
mation about individual characteristics and their effects on learning measures", which is not consistent with our research objective. The
and behavior is needed. An online course can only be individual- Big Five personality model distinguishes five dimensions of per-
ized on the basis of certain realizable characteristics of the user. sonality (cf. Barrick and Mount [42]; Bidjerano and Dai [50]) and
The single learner with all the unique complexity of his individual- we consider the following:
ity cannot be captured. One way to describe and analyze a person
is by personality traits. In personality psychology, the most fre- - Openness to experience (O): Individuals are imaginative, curi-
quently used taxonomies of personality traits are the Big Five per- ous, flexible, creative, seeking novelty, original. With regard to
sonality dimensions. After decades of research, they were devel- learning contexts, it was found that openness is linked to a deep
oped by consensus with the aim of enabling the investigation of approach to learning, elaborative learning [51], [52], meaning-di-
specified areas of personality traits rather than examining many rected learning, and constructive learning [53].
specific attributes that make people unique. The dimensions of the
Big Five were developed based on natural language terms used by - Conscientiousness (C): The individual is systematic, efficient,
people describing themselves or others (for an extended overview organized, reliable, responsible, diligent, persistent, self-disci-
of the development of the taxonomy view [38]). In addition, it can plined. In the learning context it is associated with motivation, ef-
fort and perseverance [54] as well as with methodological and ana-
and will serve as a starting point for further research and theory
development, explanation and revision of the taxonomy according lytical learning [51].
to context [38]. The present framework of the Big Five is mainly - Neuroticism (N): Individuals are disturbing, anxious, insecure,
the result of the work of Goldberg [39], McCrae and Costa [40]. depressed, self-conscious, moody, emotional, unstable. Neuroti-
Komarraju et al. [41] investigate the influence of personality on cism is associated with poor critical thinking skills, analytical abil-
learning styles in the context of academic achievement. Conscien- ity and conceptual understanding. Individuals with high neuroti-
tiousness and agreeableness were found to be positively related to cism probably have a superficial approach to learning - to focus on
all four learning styles (synthesis analysis, methodological study, memorizing and superficial features of the material being studied,
rather than gaining a deeper, meaningful understanding of it [55].
factual fidelity and elaborative processing), while neuroticism was
found to be negatively related to all four learning styles. Extraver- We aim to use clickstream data and examine the prediction of per-
sion and openness are positively related to the elaborate processing. sonality or cultural traits. In comparison with the prediction of
Furthermore, the relationship between openness and average grade dropouts, we can detect behavioral features that cause them. Our
is mediated by reflective learning styles (synthesis analysis and experimental study shows that clickstream data can be used for pre-
elaborative processing). Relevant studies on education and work diction of our items and thus targeting groups can be detected by
performance support the five-factor model and its influence on sev- specific behavioral patterns. Exploring behavioral patterns can help
eral work-related constructs [42] [43] [44]. Extraversion, conscien- instructors to personalize different areas of an online course, based
tiousness and openness are positively related to training proficiency on targeting traits.
(defined as training performance, productivity data and time to
completion of training outcomes), whereby conscientiousness is 3. METHODOLOGY
explicitly associated with learning motivation and neuroticism - Our approach focuses on finding features that can be predicted by
negatively with learning motivation (e.g. Colquitt and Simmering behavioral data. We assume that features that have a known influ-
[45]; Colquitt et al. [46] [47]). ence on learning behavior can be predicted in an online course.
These features can be used for further studies to detect learners
We decided to concentrate on only three of the five major person-
needs according to personality or cultural traits in an online course,
ality traits, namely conscientiousness, openness and neuroticism.
which could be different for various targeting groups. Thus our con-
Most studies on the context of learning and personality show the
sidered items can be used for personalization under condition that
strong impact of the three traits. The reasons for this decision are,
the targeting groups’ learners’ needs are known. We use machine
on the one hand, the results of the studies presented - most of them
learning to predict our items based on the behavior. This shows
point precisely to these factors as the most important influencing
whether our items can be predicted and how well they perform in a
factors and as linked to learning behavior and output. On the other
real-world scenario. The resulting list contains each item and the
hand, we also take into account the particularities of our study - an
corresponding accuracy that could be achieved. Sorting by accu-
online course that aims to examine a participant's learning process,
racy gives us an idea which items are predictable due to behavior.
acting separately and without any interaction with other partici-
Items where the machine learning algorithm has a bad accuracy still
pants or a teacher. In this context, the two personality traits extra-
require completing a questionnaire if we need the traits.
version and agreeableness were removed as not being relevant for
our learning process. First, we give an overview of our online course. We used a com-
mercial online course to conduct our study. It consists of tree sub
However, there is another reason for abandoning the two dimen-
lectures (L1, L2 and L3) that include information pages [P] and in-
sions: Our pretest has shown that participants find the Big Five
teractive tasks (multiple-choice question [TMC], finding the right
questionnaire, consisting of 50 questions, too long, leading to
sequence [TSE], fill in blanks [TBL], open task [TOT]), followed by
breaks and useless results. Therefore, it was necessary to reduce the
a questionnaire [Q]. It has the following structure:
number of questions. We also considered using TIPI as a shorter
version of the Big Five questionnaire as proposed by Makhija et al.
L1 = {PBasics, PBasics, PExample, PExample, TMC, TSE, QCulture} questions, which are Hofstede’s original questions [29], the Values
L2 = {PBasics, PExample, PExample, PBasics, PBasics, Survey Module 1994 [14] and additional Hofstede works [8]. Some
{TMC, TMC, TBL, TMC}, QPersonality} additional items from other construct scales were used where appli-
L3 = {PBasics, PBasics, PBasics, PExample, TSE, TOT, QFeedback} cable and items were refined until the scale was valid and reliable
We used Moodle as technical learning platform and structured all [8].
contents and tasks. Interactive tasks were implemented with the Next, we designed a neural network for each item and optimized its
plugin H5P1. This plugin contains different methods of tasks with hyperparameters to achieve the best accuracy. Grid Search [59]
the ability to give interactive feedback. helped us to find the best hyperparameters automatically. We trans-
All questionnaires (for culture, personality and feedback) were ferred all considered traits to three classes, because the general idea
placed at the end of each lecture. We aimed to acquire some partic- of Big Five and CVScale is not to get exact values to describe per-
ipants that are interested in the online course’s topic itself and not sonality or culture. Instead, all values are used to classify people,
having financial interests. By having any of the questionnaires at e.g. the Big Five is used to understand social traits of employees
the beginning, the dropout rate would be much higher. Thus, we (range: 1-40). Thus, we defined three classes: low (1-13) – medium
decided to place them at the end. (13.1-26) – high (26.1-40) for personality traits and low (1-2.33) –
medium (2.331-3.66) – high (3.661-5) for cultural traits (range: 1-
The participant’s behavior was captured by their interactions with 5). Figure 1 shows an example of the derivation of openness to ex-
the online course. From every page view we logged the time until perience, based on three classes as defined before. All other distri-
the user clicks on another page or task. We also captured how often butions look the same and are nearly equal distributed, which
the user viewed pages to detect multiple views. For all tasks we avoids overfitting.
logged the time to finish and we logged the success rate of the an-
swers. The task {TMC, TMC, TBL, TMC} of L2 is a collection of tasks,
where we could extract the overall time only with the success rates
for each containing task individually. Texts entered in the open task
are captured additionally to extract their lengths. We also logged
the length of the feedback because we assumed that this infor-
mation could have an impact to our items. We define all this data
as our behavioral data B.
On the other hand, we used the answers of the questionnaires to
apply cultural and personality dimensions because we wanted to
identify influences of these dimensions to the behavior. Apart from
the culture and personality, we also collected demographic infor-
mation (age, gender). As we detected the time that all participants
need to view single pages, we also logged the browser header to Figure 1. Distribution of openness to experience.
split our data into two datasets (mobile device and desktop). This
split is necessary due to different screen sizes, which may lead to We used the 5-fold-cross-validation (5f-CV) that splits our data
different reading time because of the necessity to scroll down on into five parts, and we built the model with four of them and tested
small screens. We call this data D. The resulting dataset was with one part. Thus, our resulting accuracy is the result of predict-
mapped into the vector B, consisting of 13 items for page view du- ing on previously unseen data. We rotate the test part and average
rations, 13 items for repeated page views, 5 items for task durations the final accuracy to get an appropriate generalizable value.
and 8 items for the task success rates. D consists of 5 cultural di- Instructors can predict these items in order to define various target-
mensions, 3 personality dimensions, 2 demographic information. ing groups that share similar learning styles due to similar cultural
The 5 cultural dimensions were calculated according to CVScale backgrounds or personality traits. According to the learners needs
[8], our considered three personality traits were calculated with these groups can use different versions of an online course, which
given formulas of the Big Five test [38]. might help to achieve better learning goals. This adjustment can be
Various other scales such as Schwartz/Rockeach; GLOBE2 (Global providing different contents or usability changes in order to opti-
Leaderhip and Organizational Behavior Effectiveness), The World mize the online course for targeted personalization. What the con-
Value Survey 3and a scale by Minkov [56] as well as various adap- crete design decisions between multiple versions should be, has to
tations are in use to quantify cultural values. Among these variables be examined in further investigations.
such as GLOBE and the CVScale are built on the core of Hofstede's
dimensions. The CVScale comprises a “26-item five-dimensional
4. RESULTS
The online course itself was about a technical related topic (Search
scale of individual cultural values” [8] that estimates the Hofstede
Engine Optimization4). We distributed the free online course in dif-
cultural values at the individual level. Being regularly used [57]
ferent social media’s groups related to business, marketing and
[58], it shows reliability, validity and generalizability across sam-
startups. Additionally, we distributed the course via several univer-
ples and nations [8]. Also, it applies to a broader context beyond
sity’s mailing lists. Finally, 142 participants took part in our study.
management [8]. It has been mainly criticized for using the same
labels as within the Hofstede model, describing differing concepts We limited our study to desktop users to eliminate potential time
[15]. However, the pool of items used for the respective scale was biases due to different screen sizes. By examining data, less than
adapted and build upon modified items from the HERMES values 1% used the mobile version of our online course.
1 3
https://h5p.org/ http://www.worldvaluessurvey.org/wvs.jsp
2 4
https://globeproject.com https://course.seorld.com
Table 1. Accuracy for each item Depending on the participants’ acceptance of a maximum number
of questions, we can choose the best predictable subset of items
Configuration of 5f-CV
Class Item with corresponding questions. If we use the best three items in a
Neural Network in %
real-world scenario, this selection requires a questionnaire with 22
Nodes Activation questions (6 for long-term orientation index, 10 for openness to ex-
Culture PO 40 softmax 82.22 perience and 6 for individualism). In contrast, to get the character-
5 softmax istics of all 8 items, answers of 56 questions are required (26 for
3 hard_sigmoid culture and 30 for three personality traits) or 76 questions if we use
UN 40 hard_sigmoid 84.44 the complete Big Five questionnaire plus CVscale. Questionnaires
5 softplus with less questions might be used, but they have to be evaluated
3 hard_sigmoid first.
ID 40 hard_sigmoid 88.89 5. DISCUSSION
5 softplus Cultural dimensions in origin were identified with respect to their
3 hard_sigmoid influence on human interactions in established systems, social or-
LT 40 sigmoid 93.33 ganizations, and education. These were the factors that have an im-
5 relu pact on the usability of online learning systems, however, we have
3 hard_sigmoid to state that these cultural variables as defined by Hofstede [13]
were not designed specifically for studying usability and behavior
MA 40 sigmoid 86.67 in online learning. In the study of Zaharias et al. [60] researchers
5 relu analyzed a connection between collectivism and learnability of a
3 hard_sigmoid web-based testing system. Another study by Downey [27] investi-
Personality O 40 sigmoid 91.67 gates the usability attributes as learnability, efficiency, memorabil-
50 softmax ity, errors, and satisfaction and the results show that participants
3 softplus from collectivist cultures showed strong, statistically significant
levels of satisfaction with the system they used. These participants’
C 40 hard_sigmoid 83.33
results had strong correlations between their low uncertainty avoid-
50 softplus
ance score and their higher errant click rates.
3 hard_sigmoid
N 40 sigmoid 88.33 Also, individuals from cultures with high power distance indicator
50 softplus scores usually made more erratic mouse clicks while using the sys-
3 sigmoid tem. However, it is important to mention that not all indicated stud-
ies have a focus on online learning and each of them has different
Demographic Age 40 hard_sigmoid 85.33 research methodologies and findings as well as different partici-
data 5 softplus pants.
5 hard_sigmoid
The experimental study was limited by the number of participants
Gender 40 hard_sigmoid 83.33
that finished the online course and the questionnaires both. The re-
5 sigmoid sults become more accurate, the more participants take place. Thus,
3 sigmoid we aim to continue our study with more participants. Over 99% of
the 142 participants did not used a Smartphone to take part. From
the practical perspective, the experiments should be applied with
To predict every item, which is comparable to a real-world sce- mobile users, those results might be different from desktop users.
nario, we created a neural network for each, consisting of a three- This can help instructors to adjust contents by splitting the targeting
layer architecture (epochs: 500, batch_size: 250). The Grid Search groups by the used device as well.
approach optimized the hyperparameters for us, shown in Table 1.
Thus, we could find the optimal accuracy in 5-fold-cross-validation According to Hu et al. [61], gender and age can be predicted on
(5f-CV), that could be achieved if being applied in an application. general websites as well. At websites as an unstructured environ-
ment, they achieved an accuracy of 79.7% on gender and 60.3% on
We assumed that some features might not be predictable as there
age. Our online course has a linear structure with non-sparse data,
are no detectable behavior patterns. Our results in Table 1 show that
which makes it easier to predict gender and age. Thus, our accuracy
all accuracies for prediction are at least 82%. This is a surprising is better. Our result shows that personality and cultural traits can be
result which shows that all our items can be predicted by behavioral predicted even better, limited to our study by using behavioral data
data. The resulting accuracies are comparable to the accuracy to of the online course. In our study, we could benefit from the linear
predict dropouts in online courses. This topic has been investigated structure. If behavioral data becomes more unstructured due to ap-
a lot by the educational communities and authors are able to achieve plying educational recommender systems, our prediction rates will
accuracies between 72% and 87% [1]. Most of our accuracies are become worse.
even better for our considered items.
Although, the indulgence versus restraint measure by Hofstede was
The cultural long-term-orientation index and the openness to expe- not included in our questionnaire, one could assume that there
rience have the highest accuracy. These items’ classes can be accu- might be a strong link to online learning. Indulgence is concerned
rately predicted for most participants.
with any behavior that fosters fun and allows for the pursuit of de-
Online courses mostly have access to demographic data only. Ac- sires and enjoyment, whereas restraint indicates ones pulling one-
curacy is not bad but there are other items that can be predicted self together in order to comply with social norms [14]. This di-
better. Since our research question was to identify a subset of items, mension could have contributed to the click through patterns of the
we can conclude that all items can be predicted based on behavior.
online course. The course displays different aspects, such as gami- Additionally, given that the order of the CVScale was adapted to
fication concerning the tasks as well as survey submission and text avoid order response bias, additional scale validation could in-
reading which may potentially be rather tedious and hence requir- crease its reliability. One could even consider combining the vali-
ing some degree of discipline or restraint. The used questionnaire, dation of the two scales of personality and national cultural dimen-
the CVScale was not designed to get the indulgence index. Further sions in order to rule out any correlations which were previously
research should take this dimension into account by using further pointed out by Hofstede [14]. With respect to cultural dimensions,
questionnaires. further research could include other ways of data collection as well
as the combination of relevant cultural background specifications
Concerning personalization, we have to understand the relation be- and questionnaires. To further extend our research we also aim to
tween our considered traits and learning goals. According to Hof- analyze behavior within online learning from the perspective of the
stede, long-term orientation is a time based perspective and know- users with special needs and disabilities. Furthermore, the research
ing this dimension for every participant can help to understand how could be further expanded with the additional parameters as emo-
they perform in an online course. Being able to predict long-term tional engagement, and cultural specifications combining the sub-
orientation gives instructors information about culture, which is cultures and personality restrictions of the learners.
linked to different variants of learning. Students that have a high
openness to experience can use experience-based learning and To understand how many questions can be used in a real-world sce-
might perform better, while others need more structured knowledge nario, we need an additional study to examine the acceptance rate
to achieve the same level of knowledge. concerning the amount of questions we ask. If we have a concrete
number of accepted questions, we are able to give a recommenda-
If we cluster the behavioral data according to our considered traits, tion how many traits could be considered for personalization.
instructors can detect differences for various targeting groups. We
examined the trait openness to experience (O) and could see that 7. CONCLUSION
the average time spent on one specific page is the following: 111 In this paper we presented an experimental study to explore the pre-
sec. (low O), 156 sec. (medium O) and 142 sec. (high O). Partici- diction of culture and personality traits based on the behavior
pants with low OP spend less time on the page, thus the instructor within online courses. We used an online course with additional
could optimize the content for this specific targeting group. For per- questionnaires to get necessary data of our considered characteris-
sonalization it is important that claims like this will be triangulated tics. We trained neural networks to show how all dimensions can
with achievement data to optimize the online course concerning the be predicted in a real-world scenario. We followed the idea that, if
learning goal. How the optimization itself can take place is the in- our items could be predicted by the behavior. Unlike assumed, there
vestigation of further research. is no item that cannot be predicted and thus no item can be ignored
in general. Two items could be predicted best (long-term orienta-
Our experiment was limited to one specific online course. To pre-
tion and openness to experience). The cultural item “power dis-
dict our considered items in another online course, we still require
a training step. This is the general training problem that prediction tance” has the worst accuracy. This validates our assumption that
tasks like predicting dropouts or final outcomes have in common. this item can be predicted word by behavior in an online course.
We conclude that instructors could focus on the best two items for
To generalize our approach for a wider usage without the necessity
prediction and further usage in online courses.
of a training step, general behavior patterns must be found that have
an importance in prediction of our traits. Therefore, we have to re- Our study does not show how the online course should be adjusted.
peat our experiment with other online courses that have a different This has to be examined in further studies, but knowing which traits
structure to find behavioral similarities for prediction. are predictable can help instructors to split users into different tar-
geting groups, which are an important base to personalize online
6. FUTURE WORK courses. Thus, our approach helps to support lifelong learning with
Our future explorative research in the context of personality and personalized online courses for a wide range of people with differ-
learning includes the application and testing of the method with ent personalities and cultural backgrounds.
other personality tests. Two tests have already been identified as Previous research of predictions in online courses still ignored cul-
relevant: 1) The Myers-Briggs indicator [62]: This test consists of tural dimensions. Our experiment has shown that culture can also
94 items developed on the basis of the four bipolar discontinuous be considered at an individual level, instead of using the country
scales of the theory of Carl Jung [63]: Introversion-Extraversion, only, where participants currently live. We gave reasons for the de-
Sensations-Intuition, Thinking-Feeling and Judging-Perceiving. cisions we made for our experimental study and discuss the relation
The classification of respondents into one of the 16 personality of culture and personality with respect to learning in an online
types is based on the highest score obtained for each bipolar scale. course. Cultural and personality traits should be the focus of further
2) The Keirsey Temperament Sorter [64] developed 16 personality studies of personalized learning in online courses.
types based on works by Socrates and Plato (with their four tem-
perament models - Artisan (iconic), Guardian (pistish), Idealist (po- 8. ACKNOWLEDGMENTS
etic) and Rational (diatonic)). He has divided the four tempera- This work was supported by the German Federal Ministry of Edu-
ments into two categories (roles), each containing two types (role cation and Research (BMBF), grant number 16DII116 (Weizen-
variants). We could examine whether these traits can be predicted
baum-Institute). The responsibility for the content of this publica-
by behavior as well.
tion remains with the authors. We would like to thank the com-
We can use existing studies on the correlation between the Big Five pany seorld for providing access to their online courses.
and these two personality tests and on the correlation between these
personalities and learning styles. We also want to test the Big Five's
two characteristics - agreeableness and extraversion - in an appro-
priate collaborative learning environment.
REFERENCES International Journal of Artificial Intelligence in Education, Springer, 2014, pp.
249-250.
[23] A. Makhija, D. Richards, J. d. Haan, F. Dignum and M. J. Jacobson, "The
[1] M. Kloft, F. Stiehler, Z. Zheng and N. Pinkwart, "Predicting MOOC Dropout Influence of Gender, Personality, Cognitive and Affective Student Engagement
over Weeks Using Machine Learning Methods," in Proceedings of the 2014 on Academic Engagement in Educational Virtual Worlds," in AIED, London,
Conference on Empirical Methods in Natural Language Processing (EMNL, Springer International Publishing AG, 2018, pp. 297-310.
Qatar, Association for Computational Linguistic, 2014, pp. 60-65.
[24] X. Liu, S. Liu, S. Lee and R. Magjuka, "Cultural Differences in Online
[2] A. L. Montgomery, S. Li, K. Srinivasan and J. C. Liechty, "Modeling Online Learning: International Student Perceptions," in Educational Technology &
Browsing and Path Analysis Using Clickstream Data," in Marketing Science Society, 13, 2010, pp. 177-188.
23(4), 2004, pp. 469-631.
[25] G. Hofstede, "Cultural differences in teaching and learning," in International
[3] A. Carr‐Chellman and P. Duchastel, "The ideal online course," in British journal of Intercultural Relations 10, 1986, pp. 301-320.
Journal of Educational Technology 31(3), 2002.
[26] C. McLoughlin, "Culturally responsive technology use: developing an online
[4] O. C. Santos, "Educational Recommender Systems and Technologies: Practices community of learners".
and Challenges," 2011.
[27] S. Downey, R. Wentling, T. Wentling and A. Wadsworth, "The Relationship
[5] H. Drachsler, H. Hummel and R. Koper, "Personal recommender systems for Between National Culture and the Usability of an E-Learning System," 2004.
learners in lifelong learning: requirements, techniques and model," in
[28] A. Al-Harthi, "Distance Higher Education Experiences of Arab," in
International Journal of Learning Technology, 3(4), 2008, pp. 404-423.
International Review of Research in Open and Distance Learning Journal, 6
[6] R. Fischer, "Where is culture in cross-cultural research? An outline of a (3), 2005, pp. 1-14.
multilevel research process for measuring culture as a shared meaning system,"
[29] G. Hofstede, "Cultures and Organizations: Software of the Mind," London:
in International Journal of Cross Cultural Management, Vol. 9 No. 1, 2009, pp.
McGraw-Hill, 1991.
25-49.
[30] P. A. Gouthro, "Assessing Power Issues in Canadian and Jamaican Women's
[7] S. Schwartz, "A theory of cultural value orientations: explication and
Experiences in Learning via Distance in Higher Education," in Teaching in
applications," in Comparative Sociology, Vol. 5 Nos 2-3, 2006, pp. 138-182.
Higher Education 9(4), 2004, pp. 449-461.
[8] B. Yoo, N. Donthu and T. Lenartowicz, "Measuring Hofstede's five dimensions
[31] S. N. Smith and P. J. Smith, "Implications for Distance Education in the Study
of cultural values at the individual level: Development and validation of
Approaches of Different Chinese National Groups," in International Journal of
CVSCALE," in Journal of International Consumer Marketing 23(3-4), 2011,
E-Learning and Distance Education, 2000, pp. 71-84.
pp. 193-210.
[32] V. Evers, A. Kukulska-Hulme and A. Jones, "Cross-Cultural Understanding of
[9] J. N. Pieterse, "Globalisation and culture: Three paradigms," in Economic and
Interface Design: A Cross-Cultural Analysis of Icon Recognition".
political weekly, 1996, pp. 1389-1393.
[33] J.-S. Lee, "Interactivity: A New Approach," in Association for Education in
[10] H. J. Hermans and H. J. Kempen, "Moving cultures: The perilous problems of
Journalism and Mass Communication, Phoenix, AZ, 2000.
cultural dichotomies in a globalizing society," in American psychologist53(10),
1998, p. 1111. [34] C. N. Gunawardena, P. L. Wilson and A. C. Nolla, "Culture and Online
Education," in Handbook of Distance Education, 2003, pp. 753-775.
[11] Hall and Hall, "Understanding Cultural Differences," 1990.
[35] S. Uzuner, "Questions of Culture in Distance Learning: A Research Review,"
[12] B. Collis, "Designing for Differences: Cultural Issues in the Design of WWW-
in International Review of Research in Open and Distance Learning 10(3),
Based Course-Support Sites," 2002.
USA, 2009.
[13] G. Hofstede, "Culture's Consequences: International Differences in Work-
[36] K. D. Strang, "Beyond engagement analytics: which online mixed-data," in
Related Values," Beverly Hills, CA: Sage., 1980.
Educ Inf Technol, New York, Springer Science+Business Media, 2017, pp. 917-
[14] G. Hofstede, "Dimensionalizing cultures: The Hofstede model in context," in 937.
Online readings in psychology and culture2(1): 8, 2011, p. 9.
[37] B. Xu and D. Yang, "Motivation classification and grade prediction for MOOCs
[15] H. Ku and L. L. Lohr, "A case study of Chinese students’ attitude toward their learners," in Journal Computational Intelligence and Neuroscience, 2016.
first online learning experience," in Education Technology Research and
[38] O. P. John and S. Srivastava, "The Big Five trait taxonomy: History,
Development, 51(3), 2003, pp. 94-102.
measurement, and theoretical perspectives. Handbook of personality: Theory
[16] M. Wang, "Designing online courses that effectively engage learners from and research 2," 1999, pp. 102-138.
diverse cultural backgrounds," in British Journal of Educational Technology,
[39] L. R. Goldberg, "The development of markers for the Big-Five factor
38(2), 2007, p. 294–311.
structure.," in Psychological Assessment, 4, 1992, p. 26–42.
[17] K. Kim and J. B. C., "Cross cultural comparisons of online collaboration," in
[40] R. R. McCrae and P. T. J. Costa, "Updating Norman’s “adequacy taxonomy”:
Journal of Computer-Mediated Communication 8(1), 2002.
Intelligence and personality dimensions in natural language and in
[18] D. &. O. E. Song, "Learning styles based on the different cultural background questionnaires," in Journal of Personality and Social Psychology, 49, 1985, p.
of KFL learners in online learning," in Multimedia-Assisted Language 710 –721.
Learning, 14(3), 2011, pp. 133-154.
[41] M. Komarraju, S. J. Karau, R. R. Schmeck and A. Avdic, "The Big Five
[19] L. Mestre, "Accommodating Diverse Learning Styles in an Online personality traits, learning styles, and academic achievement," in Personality
Environment," in Reference & User Services Quarterly 64(2), JSTOR, 2006, and individual differences, 51(4), 2011, pp. 472-477.
pp. 27-32.
[42] M. Barrick and M. Mount, "The Big Five personality dimensions and job
[20] D. R. Annette Vincent, "Personalize training: determine learning styles, performance: A meta-analysis.," in Personnel Psychology, 44, 1991, p. 1–26.
personality types and multiple intelligences online," in The Learning
[43] M. Mount and M. Barrick, "Five reasons why the “big five” article has been
Organization, 8(1), MCB UP Ltd, 2001, pp. 36-43.
frequently cited," in Personnel Psychology, 51, 1998, p. 849–857.
[21] V. Eyharabide, I. Gasparini, S. Schiaffino, M. Pimenta and A. Amandi,
[44] A. Vinchur, S. J., F. Switzer and P. Roth, "A meta-analytic review of predictors
"Personalized e-Learning Environments: Considering Students’ Contexts," in
of job performance for salespeople," in Journal of Applied Psychology, 83,
Education and Technology for a Better World (WCCE), Springer, 2009.
1998, p. 586–597.
[22] P. Mohammed and P. Mohan, "Dynamic cultural contextualisation of
educational content in intelligent learning environments using ICON," in
[45] J. Colquitt and M. Simmering, "Conscientiousness, goal orientation, and [55] N. Entwistle, "Motivational factors in students' approaches to learning," in R.
motivation to learn during the learning process: A longitudinal study," in R. Schmeck (Ed.), Learning strategies and learning styles, New York, Plenum
Journal of Applied Psychology, 83, 1998, p. 654–665. Press, 1988, p. 21−49.
[46] J. Colquitt, J. LePine and R. Noe, "Toward an integrative theory of training [56] M. Minkov, "Cultural Differences in a globalizing world," Bingley, UK,
motivation: A meta-analytic path analysis of 20 years of research," in Journal Emerald, 2011.
of Applied Psychology, 85, 2000, p. 678–707.
[57] S. Kwok and M. Uncles, "Sales promotion effectiveness: the impact of
[47] D. A. Major, J. E. Turner and T. D. Fletcher, "Linking proactive personality and consumer differences at an ethnic-group level," in Journal of Product & Brand
the Big Five to motivation to learn and development activity.," in Journal of Management 14(3), 2005, pp. 170-186.
applied psychology 91(4), 2006, p. 927.
[58] K. J. Wilby, M. J. B. Z. Govaerts and D. H. J. M. Dolmans, "Exploring the
[48] J. J. Denissen, R. Geenen, M. Selfhout and M. A. Van Aken, "Single item Big influence of cultural orientations on assessment of communication behaviours
Five ratings in a social network design," in European Association of Personality during patient-practitioner interactions," in BMC medical education 17(1),
Psychology, 22(1), European Journal of Personality, 2008, pp. 37-54. 2017, p. 61.
[49] S. D. Gosling, P. J. Rentfrow and W. B. Swann Jr, "A very brief measure of the [59] "Tuning the hyper-parameters of an estimator," scikit-learn developers, 2017.
Big-Five personality domains," in Journal of Research in personality, 37(6), [Online]. Available: http://scikit-learn.org/stable/modules/grid_search.html.
2003, pp. 504-528. [Accessed 30 07 2018].
[50] T. Bidjerano and D. Y. Dai, "The relationship between the big-five model of [60] Z. P, V. K and P. K, "Designing online learning courses: implications for
personality and self-regulated learning strategies," in Learning and individual usability," 01 01 2001. [Online]. Available:
differences, 17(1), 2007, pp. 69-81. http://www.japit.org/zaharias_etal022.pdf. [Accessed 02 02 2019].
[51] E. Geisler-Brenstein, R. R. Schmeck and J. Hetherington, "An individual [61] J. Hu, H.-J. Zeng, H. Li, C. Niu and Z. Chen, "Demographic Prediction Based
difference perspective on student diversity," in Higher Education, 31, 1996, p. on User’s Browsing Behavior," in Predictive Modeling of Web Users, Canada,
73−96. 2007, pp. 151-160.
[52] A. Slaats, J. Van der Sanden and J. Lodewijks, "Relating personality [62] I. B. Myers, "The Myers-Briggs Type Indicator: Manual," Palo Alto, CA,
characteristics and learning style factors to grades in vocational education," in Consulting Psychologists Press, 1962.
Annual Meeting of the American Educational Research Association, Chicago,
1997. [63] C. G. Jung, "Psychological types (HG Baynes, trans., revised by RFC Hull)," in
The collected works of CG Jung, 6, 1971, pp. 510-523.
[53] V. V. Busato, F. J. Prins, J. J. Elshout and C. Hamaker, " The relations between
learning styles, the Big Five personality traits and achievement motivation in [64] D. Keirsey and M. M. Bates, "Please understand me: Character & temperament
higher education," in Personality and Individual Differences, 26, 1999, p. types," Del Mar, CA, Prometheus Nemesis Book Company, 1984, pp. 3-4.
129−140..
[54] T. Chamorro-Premuzic and A. Furnham, "Personality predicts academic
performance: Evidence from two longitudinal university samples," in Journal
of research in personality, 37(4), 2003, pp. 319-338.