<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Adapting Emotional Support to Personality for Carers Experiencing Stress</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Kirsten A Smith</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Judith Mastho</string-name>
          <email>j.masthoff@abdn.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nava Tintarev</string-name>
          <email>n.tintarev@abdn.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Wendy Moncur</string-name>
          <email>w.moncur@dundee.ac.uk</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University of Aberdeen</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Dundee</institution>
        </aff>
      </contrib-group>
      <abstract>
        <p>Carers - people who provide regular support for a friend or relative who could not manage without them - frequently report high levels of stress. Good emotional support (e.g. provided by an Intelligent Virtual Agent) could help relieve this stress. This study investigates whether adaptation to personality a ects the amount and type of emotional support a carer is given and possible interaction e ects with the stress experienced. We investigated the personality trait of Emotional Stability (ES) as it is interlinked with low tolerance for stress. Participants were presented with 7 stressful scenarios experienced by a ctitious carer and a description of their personality and asked to rank 6 emotional support messages. We predicted that people with low ES would be given more emotional support messages overall and that ES would a ect the type of emotional support messages given in each scenario. We found that participants gave more praise to the high ES carer with a trend towards other support types for the low ES carer.</p>
      </abstract>
      <kwd-group>
        <kwd>Ehealth</kwd>
        <kwd>personality</kwd>
        <kwd>emotional support</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Carers - people who provide regular support for another person, without payment
- save the UK economy £119 billion every year [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], but frequently report high
levels of stress [22]. Good quality emotional support can relieve this stress and
reduce negative a ect [20]. This work is motivated by the fact that Intelligent
Virtual agents that react to a ect can be e ective in delivering emotional support
[
        <xref ref-type="bibr" rid="ref12 ref18">12, 18, 20</xref>
        ]. Studies for First responders [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] and carers [20] have found that people
provide di erent types of emotional support to people experiencing di erence
types of stress. In this study we wish to expand on this to investigate whether
the personality of the person experiencing stress a ects the type of support they
are o ered and whether this interacts with the stressor experienced.
      </p>
      <p>
        Personality describes who we are and how we react in given situations. There
are many ways to measure personality. One of the most popular and reliably
validated is the Five-Factor Model (FFM) [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], which describes an individual's
personality on a set of scores on ve di erent factors or traits: Extraversion (I),
Agreeableness (II), Conscientiousness (III), Emotional Stability (or Neuroticism)
(IV) and Openness to Experience (V). We hypothesize that carers with di erent
personalities may require di erent types and amounts of Emotional Support. In
this paper, we focus on Emotional Stability. Highly emotionally stable
individuals are calm, non-neurotic and imperturbable [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ], while low ES individuals
(those with low Emotional Stability) are more likely to worry, feel negative
affective states and experience depressive symptoms [
        <xref ref-type="bibr" rid="ref13 ref14">23, 14, 13</xref>
        ], and as such may
require more support to deal with these emotions.
      </p>
      <p>
        There is evidence that people provide di erent types of emotional support to
low ES people. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] investigated the provision of Emotional Support for learners,
and found that Low ES learners received more emotional support than
emotionally stable learners. Additionally, the type of emotional support provided
di ered, with low ES learners receiving additional `emotional re ection'
(acknowledging how the learner is feeling) where they had performed poorly. We
want to investigate whether these ndings also apply to the carer domain.
      </p>
      <p>
        The eld of tailored health communication has long established the need to
personalise health messages in order to improve the cognition of the message
and incite behaviour change [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. While the aim of emotional support is not to
incite behaviour change per se, such personalisation is likely to also be bene cial
in creating more impactful emotional support.
      </p>
      <p>Conducting research with carers is di cult, owing to the fact that the
people who need support most (i.e. people who care over 50 hours a week and
experience social isolation) do not have the time or freedom to participate in
multiple experiments. As carers do not belong to a discrete cultural group and
are very common within society, we expect that the general public are capable
of empathising with carers. Therefore our approach is to present members of the
public with a scenario about a carer and ask what support they think the carer
would like. In this way we can generate a model of the types of support that
people think a carer would appreciate without taking up a carer's time. We of
course will validate this model by consulting carers at a later date.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Study</title>
      <p>In this study we examine the impact of high or low emotional stability on the
type and quantity of emotional support messages given to a ctional carer
experiencing di erent types of stress.
2.1</p>
      <sec id="sec-2-1">
        <title>Methods</title>
        <p>Design. We used a mixed design. As a between subject factor, each
participant saw only one personality level. As within subject factors, each participant
saw all 7 scenarios and 6 messages. Participants rated their empathy with the
scenario (here called `Sympathy' to disambiguate it from the message category
`Empathy'), to allow us to control for low empathy. They also ranked 6 support
messages. The Independent Variables were Scenario (7 levels), Message (6) and
Personality (2); the dependent variables were Sympathy (1-7) and Message Rank
(0-6, coded as First=6... Sixth=1 and unranked=0).</p>
        <p>Materials.</p>
        <p>
          { Stressful Scenarios of seven key stressors (adapted form the NASA-TLX [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]
by [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ]) depicting carers were taken from [20] (see Table 1).
{ Two validated descriptions of a high and low ES person (with neutral other
traits) were taken from [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ] (see Table 2).
{ Six validated emotional support messages depicting six di erent categories
of emotional support were taken from [20] (see Table 3).
        </p>
        <p>
          Participants. Participants were recruited from Mechanical Turk [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ] and were
paid $0.80. Participants had to complete an English comprehension test, have an
acceptance rate of at least 90% and reside in the US. There were 61 participants
(31 female). 11 were aged 18-25, 28 were 26-40 and 22 were 41-65.
Procedure. Participants were told what a carer was and that they would be
shown 7 scenarios involving a carer called James. They were then shown a short
        </p>
        <sec id="sec-2-1-1">
          <title>Read and follow the instructions below. Take your time - there are no right or wrong answers; we are interested in what you think.</title>
          <p>The following scenarios depict a carer in a stressful situation. A carer is a person who
provides regular support for another person (typically a friend or family member) without
formal payment.</p>
          <p>These scenarios are about a carer called James. He cares for Susan.</p>
          <p>James often feels sad, and dislikes the way he is. He is often down in the dumps and
suffers from frequent mood swings. He is often filled with doubts about things and is
easily threatened. He gets stressed out easily, fearing the worst. He panics easily and
worries about things. James is quite a nice person who tends to enjoy talking people and
tends to do his work.</p>
        </sec>
        <sec id="sec-2-1-2">
          <title>Scenario 1 of 7</title>
        </sec>
        <sec id="sec-2-1-3">
          <title>Today James wanted to drop Susan off at the day care center so he could have some free time, but the center was closed.</title>
          <p>Imagine you are James.</p>
          <p>How well do you think you can empathise with the stress he is experiencing in this
situation?
Very poorly= “I don’t understand this situation/would not find this stressful”
Very well=”l have experienced a similar situation and understand exactly how stressful it
is”
Please Select
Imagine you are James’s friend.</p>
          <p>Below is a selection of support messages.</p>
          <p>Rank as many messages as you think he would like to receive in this situation. Rank the
most important one as Best’. the next as ‘Second best’ etc.</p>
        </sec>
        <sec id="sec-2-1-4">
          <title>You don’t need to rank all of them if you don’t think James would like to receive them.</title>
        </sec>
        <sec id="sec-2-1-5">
          <title>Support Message</title>
        </sec>
        <sec id="sec-2-1-6">
          <title>Ranking</title>
          <p>You are an amazing person.</p>
          <p>Let me help you.</p>
          <p>Your work is very appreciated.</p>
          <p>You can do this.</p>
          <p>Just take it one step at a time.</p>
          <p>I understand how stressful it must be.</p>
          <p>Please explain why you have given these rankings.</p>
          <p>Please Select
Please Select
Please Select
Please Select
Please Select
Please Select
James often feels sad, and dislikes the James seldom feels sad and is comfortable
way he is. He is often down in the dumps with himself. He rarely gets irritated, is
and su ers from frequent mood swings. not easily bothered by things and he is
He is often lled with doubts about things relaxed most of the time. He is not
easand is easily threatened. He gets stressed ily frustrated and seldom gets angry with
out easily, fearing the worst. He panics himself. He remains calm under pressure
easily and worries about things. James is and rarely loses his composure.
quite a nice person who tends to enjoy
talking people and tends to do his work.
description of James' personality, either depicting high or low ES. This remained
at the top of the screen for all scenarios. They were then presented with each
scenario in turn, asked to rate their empathy with the carer's situation and were
asked to give as many of the 6 support messages as they wished and to rank the
messages they had chosen (see Figure 1).</p>
        </sec>
      </sec>
      <sec id="sec-2-2">
        <title>Hypotheses.</title>
        <p>H1 People will give di erent support messages to the low ES carer.
H2 People will give more support messages to the low ES carer.
2.2</p>
      </sec>
      <sec id="sec-2-3">
        <title>Results</title>
        <p>
          E ects of Scenario Personality on Message Rankings. Figure 2 shows
the mean ranks of each message for the 2 ES levels and the number of
messages overall. Previous research has found that empathy with a situation a ects
emotional support [
          <xref ref-type="bibr" rid="ref4">19, 4</xref>
          ]. Thus in order to ensure that the empathy level did
not impact our results, this was controlled for as part of our analysis. A 7 2
within-subjects ANCOVA was performed of Scenario Personality, controlling
for Sympathy, on Message rankings (6 levels). This was chosen as the most
appropriate test for this data (ANCOVA is a powerful test and can be used
for non-normal data) [21]. This was signi cant at F(1,419)=186.50, p&lt;0.01.
There were signi cant e ects for Scenario (F(6, 419)=3.54, p&lt;0.01) and
signi cant interaction e ects for Message Scenario (F(30, 2095)=11.67, p&lt;0.01)
and Message Personality (F(5, 2095)=2.44, p&lt;0.05).
        </p>
        <p>Post-hoc tests on Scenario revealed that the Mental Demand, Physical
Demand and Temporal Demand scenarios had signi cantly higher message
rankings than Isolation, indicating that more messages were given for these scenarios.
Post-hoc tests for the interaction of Message Scenario revealed the most
popular messages for each scenario, shown in Table 2.2. These results are similar to
the ndings in [20].</p>
        <p>Scenario</p>
        <p>Highest Ranked</p>
        <p>Messages
Mental Demand PRA, ENC, EMP
Temporal Demand SUP, APP, ENC, PRA
Physical Demand SUP, APP
Frustration SUP, EMP, APP
Interruption SUP, EMP, ENC, APP
Isolation APP, PRS</p>
        <p>Emotional Demand APP, PRS</p>
        <p>Post-hoc tests on Personality revealed a signi cant e ect of personality on the
Praise message. Participants ranked Praise signi cantly higher for the high-ES
carer (Mean=2.94, S.E.=0.15) than the low-ES carer (Mean=2.41, S.E.=0.15).
This supports hypothesis H1.</p>
        <p>E ects of Scenario Personality on Number of Messages ranked. A
7 2 within-subjects ANCOVA was performed of Scenario Personality,
controlling for Sympathy, on Number of Messages ranked. This was signi cant
at F(14,419)=3.59, p&lt;0.01. There were signi cant e ects for Scenario (F(6,
419)=2.64, p&lt;0.05). Post-hoc tests showed that fewer messages were given for
the Isolation Scenario (mean=3.75, S.E.=0.24) than for Mental Demand (mean=
4.82, S.E.=0.24) and Temporal Demand (mean=4.90, S.E.=0.24). This supports
ndings in [20]. There was no e ect of personality, contrary to H2.</p>
        <p>These results suggest that there is some variability in the type of emotional
support that people give to carers with high and low ES. The low ES carer
received less Praise; however, there was no signi cant di erence of the number
of messages ranked for each personality. This implies that the low ES carer must
have received more of another type of support. From Figure 2 we can see that
the low ES carer received more Empathy, Practical Advice and Encouragement
than the high ES carer. Although not signi cant, it may be that the low ES
carer received a mixture of these three support types instead of Praise and this
has diluted any e ect, as rankings were split between them.</p>
        <p>M
4 e
a
n
N
u
m
b
3 re
o
f
M
e
s
s
a
2 g
e
s
1
0
Supported</p>
        <p>Empathy</p>
        <p>Practical Advice Appreciated</p>
        <p>Encouragement</p>
        <p>Praise
Message Category Error Bars: +/- 1 SE
No. Messages
Personality</p>
        <p>Low ES
High ES</p>
        <p>Fig. 2. Mean rank of messages and mean no. messages for ES High and Low</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Discussion</title>
      <p>
        We found that the High emotionally stable carer was given more Praise than the
low ES carer. The data also suggests that low ES carers receive a wider range
of emotional support. This might be because neurotic individuals are more
worried about failing a task [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] and so are not praised but reassured with empathy,
encouraged and given advice. The High ES carer isn't given as wide a variety of
support as they aren't perceived as needing it. Encouragement has an advantage
over Praise in that it can be delivered when things are going badly, while Praise
is appropriate only when someone has performed well in a task - thus
encouragement could be seen as a better motivator [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] and so was provided to the low
ES carer.
      </p>
      <p>It is of course hard to tell why participants picked certain messages over
others for the carer as we did not obtain useful qualitative data about their
choices. It is possible that the changing scenarios became more salient to the
participants than the personality description and that they neglected to consider
personality when they were picking messages. Identifying participants with high
and low ES and investigating which messages they pick for the carer would
perhaps yield clearer results.</p>
      <p>
        This study uses Mechanical Turk. This is a useful tool for crowd-sourcing
a large number of diverse participants (vs typical university student samples).
Furthermore, data obtained from Mechanical Turk has been found to be high
quality and reliable [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. 36% of our participants were aged 41-65; in England
and Wales, people aged 50-64 are most likely to be carers [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. By examining our
demographic data about our participants, 20 out of 61 reported to be informal
carers, while a further 25 claimed to be professional carers, from forced choice
of `professional carer', `informal carer' and `other'. While it is not clear whether
all these responses are honest, it is at least an indication that many Mechanical
Turk users are familiar with caring.
      </p>
      <p>While this study distinguishes between di erent types of stressor that a carer
might experience, we do not distinguish between carers of people with di erent
health conditions. There might be a considerable di erence between o ering
emotional support for palliative care and long-term mental health care for
instance.</p>
      <p>
        This study investigates which type of support to use if the stressor is known
- we do not investigate how the stressor can be detected. We anticipate that
this emotional support could implemented in a system that makes use of
sentiment analysis [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] to detect the stressor from social network status posts or by
prompting the carer to write something about their day.
4
      </p>
    </sec>
    <sec id="sec-4">
      <title>Conclusion</title>
      <p>We have found evidence that the emotional support to provide to carers in
stressful situations may need to be adapted to carer personality. We have also
found support for [20], that emotional support should adapt to stressor. In future
work we plan to investigate other personality traits, expand on the number
and type of messages provided and consider the e ects of gender and culture.
Additionally, whilst this paper has investigated the emotional support people
would provide, this may not be the same as what people would like to receive.
We will therefore also investigate the e ectiveness of emotional support messages
and adaptations on carers with di ering personality traits.
19. Reynolds, W.J., Scott, B.: Empathy: a crucial component of the helping
relationship. Journal of psychiatric and mental health nursing 6(5), 363{370 (1999)
20. Smith, K.A., Mastho , J., Tintarev, N., Moncur, W.: The development and
evaluation of an emotional support algorithm for carers. Intelligenza Arti ciale 8(2),
181{196 (2014)
21. Vickers, A.J.: Parametric versus non-parametric statistics in the analysis of
randomized trials with non-normally distributed data. BMC medical research
methodology 5(1), 35 (2005)
22. Vitaliano, P.P., Zhang, J., Scanlan, J.M.: Is caregiving hazardous to one's physical
health? a meta-analysis. Psychological Bulletin 129(6), 946{72 (2003)
23. Watson, D.: Mood and temperament. Guilford Press (2000)</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1. 2011 census
          <article-title>- unpaid care snapshot</article-title>
          .
          <source>O ce for National Statistics</source>
          (
          <year>2011</year>
          ), http: //www.ons.gov.uk/ons/guide-method/census/2011/carers-week/index.html
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Buckner</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Yeandle</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          : Valuing carers
          <year>2011</year>
          .
          <string-name>
            <surname>Carers</surname>
            <given-names>UK</given-names>
          </string-name>
          , London (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Buhrmester</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kwang</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gosling</surname>
          </string-name>
          , S.D.:
          <article-title>Amazon's mechanical turk a new source of inexpensive, yet high-quality, data? Perspectives on psychological science 6(1), 3{5 (</article-title>
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>M.H.</given-names>
          </string-name>
          :
          <article-title>The e ects of dispositional empathy on emotional reactions and helping: A multidimensional approach</article-title>
          .
          <source>Journal of personality 51(2)</source>
          ,
          <volume>167</volume>
          {
          <fpage>184</fpage>
          (
          <year>1983</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Dennis</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          : Adapting Feedback to Learner Personality to Increase Motivation.
          <source>Ph.D. thesis</source>
          , University of Aberdeen (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Dennis</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kindness</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mastho</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mellish</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>Towards e ective emotional support for community rst responders experiencing stress</article-title>
          .
          <source>Humaine Association Conference on A ective Computing and Intelligent Interaction</source>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Eysenck</surname>
            ,
            <given-names>H.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eysenck</surname>
            ,
            <given-names>S.B.G.</given-names>
          </string-name>
          :
          <article-title>Manual of the Eysenck Personality Questionnaire (junior and adult)</article-title>
          .
          <source>Hodder and Stoughton</source>
          (
          <year>1975</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Goldberg</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>The structure of phenotypic personality traits</article-title>
          .
          <source>American Psychologist</source>
          <volume>48</volume>
          ,
          <issue>26</issue>
          {
          <fpage>34</fpage>
          (
          <year>1993</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Hart</surname>
            ,
            <given-names>S.G.</given-names>
          </string-name>
          :
          <article-title>Nasa-task load index (nasa-tlx); 20 years later</article-title>
          .
          <source>In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting</source>
          . vol.
          <volume>50</volume>
          , pp.
          <volume>904</volume>
          {
          <fpage>908</fpage>
          .
          <string-name>
            <surname>Sage Publications</surname>
          </string-name>
          (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Hawkins</surname>
            ,
            <given-names>R.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kreuter</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Resnicow</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fishbein</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dijkstra</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Understanding tailoring in communicating about health</article-title>
          .
          <source>Health education research 23(3)</source>
          ,
          <volume>454</volume>
          {
          <fpage>466</fpage>
          (
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>John</surname>
            ,
            <given-names>O.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Srivastava</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>The big ve trait taxonomy: History, measurement, and theoretical perspectives</article-title>
          .
          <source>Handbook of personality: Theory and research 2</source>
          (
          <year>1999</year>
          ),
          <volume>102</volume>
          {
          <fpage>138</fpage>
          (
          <year>1999</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , Moon,
          <string-name>
            <given-names>Y.</given-names>
            ,
            <surname>Picard</surname>
          </string-name>
          , R.W.:
          <article-title>This computer responds to user frustration:: Theory, design, and results</article-title>
          .
          <source>Interacting with computers 14(2)</source>
          ,
          <volume>119</volume>
          {
          <fpage>140</fpage>
          (
          <year>2002</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Lahey</surname>
            ,
            <given-names>B.B.</given-names>
          </string-name>
          :
          <article-title>Public health signi cance of neuroticism</article-title>
          .
          <source>American Psychologist</source>
          <volume>64</volume>
          (
          <issue>4</issue>
          ),
          <volume>241</volume>
          (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Larsen</surname>
            ,
            <given-names>R.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ketelaar</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Personality and susceptibility to positive and negative emotional states</article-title>
          .
          <source>Journal of personality and social psychology 61(1)</source>
          ,
          <volume>132</volume>
          (
          <year>1991</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15. MT:
          <article-title>Amazon mechanical turk</article-title>
          . http://www.mturk.com
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Paltoglou</surname>
          </string-name>
          , G.:
          <article-title>Sentiment analysis in social media</article-title>
          .
          <source>In: Online Collective Action</source>
          , pp.
          <volume>3</volume>
          {
          <fpage>17</fpage>
          . Springer (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Pitsounis</surname>
            ,
            <given-names>N.D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dixon</surname>
            ,
            <given-names>P.N.</given-names>
          </string-name>
          :
          <article-title>Encouragement versus praise: Improving productivity of the mentally retarded</article-title>
          .
          <source>Individual Psychology: Journal of Adlerian Theory, Research &amp; Practice</source>
          (
          <year>1988</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Prendinger</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ishizuka</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>The empathic companion: A character-based interface that addresses users'a ective states</article-title>
          .
          <source>App Arti cial Intell</source>
          <volume>19</volume>
          (
          <issue>3-4</issue>
          ),
          <volume>267</volume>
          {
          <fpage>285</fpage>
          (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>