<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Using aspect-based sentiment analysis as a tool for socio-technical system design</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Parisa Saadati</string-name>
          <email>parisa.saadati@uwl.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Laden Husamaldin</string-name>
          <email>laden.husamaldin@uwl.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jose Abdelnour-Nocera</string-name>
          <email>nocera@uwl.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University of West London, School of Computing</institution>
          ,
          <country country="UK">UK</country>
        </aff>
      </contrib-group>
      <fpage>232</fpage>
      <lpage>255</lpage>
      <abstract>
        <p>Automation can enhance performance and reduce human errors, but poorly designed interactive technologies can harm the user experience, leading to suboptimal outcomes for both systems and organisations. Social and technical subsystems are essential and interdependent and must be considered when designing organisational structures. Neglecting these aspects limits the potential for effective change. This paper explores the role of Aspect-Based Sentiment Analysis (ABSA) in improving user experience (UX) by addressing social factors in automated environments. As a subfield of Natural Language Processing (NLP), ABSA extracts insights tied to specific aspects from user feedback, allowing for targeted improvements that enhance both UX and system performance. With the growing reliance on unstructured data from sources like social media, we ask: Can ABSA play a key role in enhancing system design? Can it facilitate data collection and interpretation for the development of automated systems in large organisations? To assess ABSA's effectiveness, we compiled a dataset of 8,060 tweets (2016-2023) related to automation and smart workplaces, using it as a case study. Six pre-trained or fine-tuned ABSA models were applied to analyse the data, focusing on dimensions such as usability, effectiveness, and innovation. Our findings highlight ABSA's potential to guide improvements in socio-technical automated systems, and we propose recommendations for decision-makers, developers and researchers to identify strengths and weaknesses in automated systems using any type of user-generated comments. Finally, we present a lifecycle model using sentiment analysis to refine UX strategies, though further exploration of factors like trust and accessibility is needed. While the study offers valuable insights, further exploration of factors such as trust, accessibility, and ethical considerations is needed.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Aspect-Based Sentiment Analysis</kwd>
        <kwd>Socio-Technical Systems</kwd>
        <kwd>User Experience</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>In today's fast-paced business environment, large organizations are embracing digital
transformation to remain competitive. This involves overhauling back-end systems,
customer interactions, and internal processes with emerging technologies such as AI, ML, and
IoT. Automation is crucial for refining workflows, decision-making, and user satisfaction.
However, integrating new technologies within complex organizations poses significant
challenges. A user-centric approach and automation of routine tasks can streamline
communication and foster success. Human-machine collaboration will be key, necessitating job redesign
for effective synergy. Organizations must bridge the gap between potential and realized value
from technology investments, requiring a managerial framework that emphasizes user
experience, value proposition, digital evolution, skills, and improvisation. Designing effective social
and technical systems requires user insights, but traditional methods like cognitive task
analysis and workshops, climate surveys, and participatory design may be impractical in large,
diverse organizations. More efficient methods, such as social network analysis and
user-generated comment analysis, can provide iterative design insights.</p>
      <p>
        Understanding public sentiment towards products and services is essential in the digital
age. Aspect-based Sentiment Analysis (ABSA) is a critical technique within sentiment
analysis, offering a detailed examination of opinions expressed in user-generated content.
Unlike general sentiment analysis, which assesses overall sentiment [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], ABSA focuses on
specific aspects or features of a product or service [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] as shown in figure 1 below. ABSA's
effectiveness can be influenced by domain-specific language and short, informal text [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. This
detailed analysis provides actionable insights for targeted improvements [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] and helps
companies tailor their offerings and communications to align with user preferences. It also identifies
trends and issues, allowing for proactive enhancements in the user experience [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        ABSA is particularly useful in sociotechnical computing, where the interaction between
social and technical systems is complex. By incorporating ABSA, researchers can gain
nuanced insights into user experiences and expectations, aiding informed design decisions
and improving user satisfaction. Recent advancements in NLP and machine learning, such as
BERT and Graph Neural Networks, have enhanced ABSA's accuracy and contextual
understanding [
        <xref ref-type="bibr" rid="ref4 ref6 ref7">4, 6, 7</xref>
        ]. These advancements enable comprehensive analysis of large text datasets
from various sources, enriching sociotechnical studies.
      </p>
      <p>This paper explores ABSA's application as a sociotechnical tool for interpreting data in
large organizations and enhancing the user-centered design of autonomous technologies. It
discusses ABSA model results, and applications, and concludes with challenges and future
directions for this interdisciplinary approach.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Literature Review</title>
      <p>
        Aspect-Based Sentiment Analysis (ABSA) and socio-technical approaches both focus on
the interaction between technology and human systems, though in different contexts.
Sociotechnical approaches emphasise the co-evolution of social and technical systems,
recognising that technology does not operate in isolation but is deeply embedded within
human contexts [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. This approach highlights the importance of designing systems that
consider not only technical performance but also human factors such as culture, workflow,
and communication. It is often used in fields such as system design, organisational change,
and technology implementation to ensure a harmonious interaction between people and
technology [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Integrating ABSA within socio-technical frameworks can indeed offer valuable
insights into how users perceive specific aspects of technological systems, influencing their
broader adoption and integration into society. By breaking down sentiments expressed by
users into specific aspects of the system (e.g., usability, reliability, security), ABSA helps in
understanding what elements of the user experience contribute positively or negatively to
system adoption [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] or if the result is neutral what type of strategies can be taken for
improvement?
      </p>
      <p>
        For example, identifying aspects such as "ease of use" or "accessibility" in user feedback
allows system designers to address critical pain points, thus improving the seamlessness of the
user journey [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. The socio-technical approaches, which emphasise the interplay between
human users and technological systems, benefit from this insight, as it ensures that both
technical and human factors are aligned to enhance user satisfaction and system efficiency[
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>
        By focusing on specific user experience factors as aspects, ABSA, ABSA provides
actionable insights to system developers, ensuring that technology evolves not only with
technical innovation but also in alignment with user needs and expectations, leading to
broader and more successful adoption of technological solutions [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
      </p>
      <sec id="sec-2-1">
        <title>2.1. Sociotechnical Approaches for Competitive Advantage in Large</title>
      </sec>
      <sec id="sec-2-2">
        <title>Organisations</title>
        <p>
          Research emphasises user-centred design as pivotal in developing Sociotechnical systems.
Findings by [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ] underscore that leading companies embracing digital transformation
(which has some element of automation) experience significant revenue growth compared to
laggards. Failure to adapt may result in substantial revenue loss. Therefore, achieving digital
value creation through such systems is crucial for market competitiveness [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ].
        </p>
        <p>
          In designing future systems, the initial step involves comprehensive problem diagnosis
using tools like systems thinking, future inquiries, and human-centred research [
          <xref ref-type="bibr" rid="ref12 ref14 ref15 ref16 ref17">12, 14–17</xref>
          ].
Effective diagnosis, however, does not guarantee successful strategy formulation. Organisations
must grasp system dynamics to establish design principles and intervention models. This
iterative process demands experimentation and evaluation to drive impactful change. Engaging
the end user’s insights and experience in the design systems is a must to increase the
acceptance of the systems and increase user satisfaction. Following a sociotechnical systems theory
which underscores the interdependence of social and technical components within complex
organisational systems is a key to success. Neglecting either aspect impedes effective
organisational change and improvement, necessitating input from all stakeholders.
        </p>
        <p>
          Future-oriented design requires creativity and collaboration, leveraging tools such as
design thinking, design fiction, prototyping, work domain analysis, collaborative modelling
[
          <xref ref-type="bibr" rid="ref18 ref19">18, 19</xref>
          ], and computer-based simulation.
        </p>
        <p>
          In addition, it is proven that human-centric approaches are gaining traction for their
efficiency and user engagement benefits [
          <xref ref-type="bibr" rid="ref20">20, 21</xref>
          ]. Participatory design, involving
stakeholders as co-designers, is increasingly employed in organisational development and
system design, though its effectiveness varies across different contexts [22, 23]. However,
trials and pilots are common strategies for testing future automated systems in large
organisations [24]. Despite being resource-intensive, these methods provide realistic insights,
particularly in integrating AI technologies [25, 26].
        </p>
        <p>In conclusion, adopting sociotechnical approaches for the selection or design of new
automated systems offers a competitive edge in the digital era by ensuring systems are
technologically advanced, socially relevant, and user-centric. Organisations that embrace
these principles effectively navigate digital Ergonomic and human factors, which is critical for
digital transformation in complex organisations. Various studies present models addressing
these factors, enhancing task performance and employee engagement [27–30]. These models
are often standardised [31] transformations, driving innovation and sustainable growth.</p>
      </sec>
      <sec id="sec-2-3">
        <title>2.2. Sociotechnical Approach in Complex Organisations</title>
        <p>
          Bostrom and Heinen (1977) described organisational work systems as composed of two
interconnected subsystems: social (involving people and structure of the company) and technical
(comprising technology and tasks of the organisation) [32]. The authors believed that
organisational behaviour issues involving social subsystem components directly impact the
technical subsystem's failure and success (Figure 2). It is proposed that an STSD approach will
provide a more realistic view of organisations [
          <xref ref-type="bibr" rid="ref21">33</xref>
          ].
        </p>
        <p>
          In this approach, organisational variables will also be considered to build vast, complex
infrastructures that contain cohesive systems of human relations, technical objects, and
cybernetic processes. This application has extended beyond the workplace setting, such as smart
homes [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ].
        </p>
        <p>
          Based on STSD, the human and machine interaction should not be considered in isolated
scenarios (Figure 3 on the left). Designing effective systems for complex organisations need a
pragmatic approach to the engineering of socio-technical systems similar to Figure 3 (on the
right) [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ].
        </p>
        <p>According to the STSD approach, the ergonomic and human factors must be studied for
digital transformation in complex organisations. These factors can assist the employees in
their tasks and increase their engagement [27–30]. Some of these models have been
transformed into a list of standards by the International Organisation for Standardisation
(ISO) [31].</p>
        <p>
          The environmental factors and their impact on organisations should also be considered for
digital transformations. For example, [30] study on automated gates on borders listed several
human factors (such as motivation, trust, skills, situation awareness and workload) as well as
environmental factors in the socio-cultural, legal, organisational and operational contexts.
The reality is replacing AI methods such as ABSA can provide fine-grained feedback analysis
and help socio-technical systems become more responsive and adaptive. This is particularly
valuable in socio-technical contexts, where both human and technical elements need to
evolve together to meet user expectations efficiently. For example, studies
have shown that combining ABSA with deep learning and multi-task models can automate
the detection of user issues in real-time, enabling faster adjustments to systems (such as in
customer service or incident response systems) based on identified trends in feedback
about usability, performance, or reliability [
          <xref ref-type="bibr" rid="ref22">34</xref>
          ]. In another study, ABSA has been proposed as
a key tool for cybersecurity incident response systems, where understanding sentiments
about security tools can help adjust processes and communications between users and
technical teams [
          <xref ref-type="bibr" rid="ref23">35</xref>
          ].
        </p>
      </sec>
      <sec id="sec-2-4">
        <title>2.3. User Engagement and User Experience Goals</title>
        <p>
          User engagement with technology is a crucial aspect of user experience design, as it
determines the success and adoption of interactive systems. Engagement is a complex
construct characterised by cognitive, affective, and behavioural components, such as
attention, motivation, and emotional investment [
          <xref ref-type="bibr" rid="ref24">36</xref>
          ]. One key definition of user
engagement is "a quality of user experience characterized by attributes of challenge,
positive affect, endurability, aesthetic and sensory appeal, attention, feedback,
variety/novelty, interactivity, and perceived user control" [
          <xref ref-type="bibr" rid="ref25">37</xref>
          ]. Although user engagement
can be assessed before full-scale implementation of any new system (via usability testing or
pilot implementation), it should be performed in a real environment for all types of users (e.g.,
customers and employees). This is particularly important for automating systems that will
replace daily human tasks, as they can positively and negatively affect human performance
[
          <xref ref-type="bibr" rid="ref26">38</xref>
          ]. Human reactions depend on the impact of new systems on the work process and
environment [30]. Early studies have shown how employees would be influenced by the feedback
they receive from machines, and the perceived reliability and accuracy of automated systems
[
          <xref ref-type="bibr" rid="ref27">39</xref>
          ]. Automation influences many human factors including performance, situational
awareness, workload, motivation, stress and trust [
          <xref ref-type="bibr" rid="ref28">28, 30, 40</xref>
          ].
        </p>
        <p>Research shows that human-centred design approaches (e.g., HWID) that engage the
end-users before going to the implementation phase result in the development of more
effective solutions for problems that need new technologies. Involving the human
perspective and emotion in all steps of the new system development helps understand the
machine and human interaction on a bigger scale. All these can impact the user experience to
have a seamless journey when using the implemented system. In addition, a better
understanding of the work domain, customers (users) and employees’ needs can be shaped into a
shared vocabulary of technical and social requirements between the developers and
decisionmakers. Involving users in the design process can also increase user empowerment (e.g., via
employee engagement).</p>
        <p>
          UX is beyond the material, good industrial design, multi-touch, or fancy interfaces [
          <xref ref-type="bibr" rid="ref29">41</xref>
          ]
and has a direct relation to user engagement. Positive user experience can be evoked if the
technology, system, or product satisfies a generalised set of users experience goals and can
happen or improve over time. These goals are targeted experiences that end users should
have when collaborating or interacting with a particular technology [
          <xref ref-type="bibr" rid="ref30">42</xref>
          ]. They are two main
concepts in understanding UX and its constituting factors. The first concept [
          <xref ref-type="bibr" rid="ref31">43</xref>
          ] believes
that experience is evoked from the interplay of humans and technology. Thus, experiences
are dynamic, unique, situated, and never the same. They also believed in objectifying
experience into distinct fields of study (i.e., a holistic approach to experience). The next
concept is by Hassenzahl and his colleagues. [
          <xref ref-type="bibr" rid="ref32">44</xref>
          ] This study proposes categorising
experiences based on psychological needs and linking them to affect and product
perception. It presents a need to generalise user experience to use human-computer
interaction (HCI) applications effectively.
        </p>
        <p>
          When users use the system to carry out their tasks in both concepts, the UX designer
wants them to have a specific experience. Designers collaborate with users to understand the
technology's needs, such as emotional and system qualities. These needs determine the UX
goals and are crucial in directing system design. This can ensure that the planned attributes
are met and result in a better experience [
          <xref ref-type="bibr" rid="ref33">45</xref>
          ].
        </p>
        <p>Context relevance is crucial in user experience (UX) research, as it helps in
understanding the user's environment, motivations, and behaviours. Providing detailed
descriptions is essential for uncovering different facets of user experience and constructing
meaningful interactions. While it may be challenging to outline a definitive list of UX goals
due to the complexity and uniqueness of each project, several common objectives guide UX
design. These include enhancing usability, improving accessibility, ensuring user satisfaction,
fostering engagement, and creating seamless, intuitive interactions. Achieving these goals
requires a deep understanding of the user's context, making thorough research and attention to
detail indispensable.</p>
      </sec>
      <sec id="sec-2-5">
        <title>2.4. Exploring Sentiment Analysis in Social Media Approaches</title>
        <p>ABSA allows for extracting sentiments related to specific product or service aspects from
user-generated content. By focusing on particular features, ABSA provides a nuanced
understanding of user opinions.</p>
        <p>
          In the last decade, opinion mining and sentiment analysis have been the subject of
fascinating interdisciplinary research. With the ever-increasing number of social media
messages posted daily, millions of users express opinions on various subjects, including the
characteristics of products and services they have already bought or intend to buy in the
near future. In this regard, [
          <xref ref-type="bibr" rid="ref34">46</xref>
          ] introduces a method using Semantic Web technologies,
natural language processing, and machine learning for in-depth sentiment analysis of
Twitter product opinions. However, scalability challenges may arise due to the reliance on
Semantic Web technologies, and the suggested exploration of advanced deep learning
algorithms could introduce complexities and increased computational resource needs. [47]
states and presents a CNN-based method for aspect term extraction and sentiment analysis in
user reviews, offering high accuracy and efficiency. However, the reliance on pre-defined
aspect categories may limit accuracy, suggesting potential improvements with Word2Vec and
other techniques. The application to restaurant reviews via web scraping could pose
challenges in maintaining accuracy for large-scale analysis. On the other hand, [48] introduces
ABSA-PER, achieving notable accuracy in sentiment analysis of customer reviews, yet
acknowledges limitations in handling diverse information. The study suggests
potential enhancements, including multi-word clustering and objectivity feature
exploration, to improve performance and accuracy in aspect identification [49].
        </p>
        <p>ABSA is a powerful tool for clear consumer sentiment recommendations, overcoming
subjective judgment limitations. The proposed method employs a BERT-based supervised
classifier for fine-grained tweets, enhancing ABSA accuracy. Despite its benefits, limitations
include the ongoing challenge of improving overall sentiment analysis robustness and
accuracy.</p>
        <p>In conclusion, integrating ABSA within socio-technical frameworks offers valuable
insights into how users perceive specific technological aspects, influencing adoption and
engagement. By addressing both technical and human factors, ABSA enhances system design by
identifying key user experience elements, such as usability, accessibility, and effectiveness.
This alignment ensures that technology evolves in harmony with user needs and
expectations, fostering broader and more successful adoption. Additionally, sociotechnical
approaches, which consider the co-evolution of social and technical systems, provide a
competitive edge, especially in large organizations undergoing digital transformation.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Methodology</title>
      <p>We utilized Twitter Scraper to collect tweets, a powerful tool that enables efficient data
extraction from the Twitter platform. Twitter Scraper is a Python library that simplifies
retrieving tweets based on specific search criteria, allowing researchers to gather relevant
data for analysis. Web scraping has become a source of data collection for aspect-based
sentiment analysis and has recently been used for different purposes. From young people's
attitudes in the climate change debate [50] to predict the trend of cryptocurrencies [51],
identifying user interest from social data [52] and even child tracking and predicting the violence
on children in social media [53]. Figure 4 summarises the steps taken for the methodology.</p>
      <sec id="sec-3-1">
        <title>3.1. Data collection and Pre-Processing</title>
        <p>The study collected relevant tweets using keywords related to the assessment,
effectiveness, performance, innovativeness, usefulness, usability, and value of automated
systems and smart workplaces in large organisations. These keywords were carefully derived
from the literature review and case studies to ensure the relevance and depth of the data. The
tweets were downloaded, extracted, and processed from the Twitter platform (now known as
X), adhering to the ethical and legal guidelines of both the platform and associated
universities. Data collection spanned from 2016 to the first quarter of 2023, a period marked by
significant advancements in automation and Industry 4.0 technologies, including the widespread
adoption of IoT and AI [54–56]. The collection ended in early 2023 when X implemented a
paywall restricting automated access to tweets, impacting thirdparty tools' ability to access
the platform’s data [57].</p>
        <p>We used 8060 Tweets for our experiment. Pre-processing is essential for sentiment
classification, especially with unstructured data, to address inaccuracies, redundancies,
missing values, and inconsistencies. The pre-processing stage, illustrated in Figure 5, involves:
 Data cleaning
 Handling missing values
 Removing redundancies
 Transforming data formats
 Extracting relevant features</p>
        <p>This process ensures the dataset is clean and consistent, leading to more accurate and
reliable classification results and ultimately enhancing the models' performance and the
analysis's validity.</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. ABSA Models Utilised for Analysis</title>
        <p>For this study, all selected models are primarily focused on and pre-trained for English
language tasks. They are either based on RoBERTa or its distilled version, DistilRoBERTa,
which was initially trained on large English-centric datasets. Both Models do not inherently
include multilingual capabilities as part of their original design or pre-training; their
architecture is language-agnostic, but the choice of training data focuses them on English.
Theoretically, the models can be adapted to other languages by fine-tuning non-English
data. However, for further use in non-English data, it is better to use models that have been
explicitly designed or fine-tuned for those languages or multilingual models such as mBERT
(Multilingual BERT) and XLM-RoBERTa, which are trained on text from multiple languages
and are capable of understanding and processing text in those languages. Table 1 lists the
models used for this paper and explains their architecture, application and labels.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Result</title>
      <sec id="sec-4-1">
        <title>4.1. Identified Aspects and Sentiments</title>
        <p>ABSA is a more detailed form of sentiment analysis that identifies whether a text
expresses a positive, neutral, or negative sentiment and associates these sentiments
with specific aspects or features mentioned in the text. This can provide deeper insights into
user opinions on various products, services, or topic attributes.</p>
        <p>Our results offer valuable insights into user sentiment on specific product, service, or
topic aspects. This can inform targeted improvements, marketing strategies, and user
experience design. Related to our main aim, it can provide a better understanding to the
development team for improving their UI design and for decision-makers to decide on the
selection of the trials quickly.</p>
        <p>We used general UX goals from other papers [58] and our previous case studies[59],
which we found easy to recognise by the models with no major changes or interpretations.
The aspects we used were:</p>
        <p>Aspects: ["user friendly", "easy", "valuable", "accessible", "useful", "effective",
"performance", "innovative"]</p>
        <p>The analysis in model 1 shows that most aspects are predominantly viewed neutrally in
both models, indicating either a balanced view or a lack of strong opinions. Positive
sentiments tend to outweigh negative aspects, suggesting overall positive reception in areas
like effectiveness, innovation, performance, user-friendliness, and value (in model 1).</p>
        <p>The aspect with the most positive feedback is "User Friendly/Easy," highlighting this as a
strength. The "Useful" aspect has the highest number of negative sentiments, suggesting an
area that may need improvement or further investigation to understand users' concerns.</p>
        <p>When selecting model 2, we acknowledged that the model is trained within the context of
financial news or services. Still, we decided to try it on our dataset (which targets
automationrelated topics) and explore the results. This can produce valuable insights, particularly in
areas where financial and economic considerations play a significant role in discussions about
automation, such as system performance or economic impacts. It is interesting how changing
the model can show different results on specific aspects such as effectiveness and innovation.
The rest of the aspects also show different results but with no significance.</p>
        <p>Overall, Model 3's results indicate a general neutrality trend in the perception of various
product or service aspects. This suggests that while there are no major areas of failure, there
is a significant opportunity to improve the product/service's features, benefits, and overall
value proposition to shift these neutral perceptions towards more positive ones.
Enhancements, clearer communication, and focus on innovation could be key strategies to
improve user perception and sentiment.</p>
        <p>Model 4's analysis is more polarised, with only positive and negative sentiment
categories, unlike the other three models, which included a neutral category. This approach
provides a clearer contrast in user perceptions, highlighting strengths and areas for
improvement more distinctly. As this model has no neutral label, it is not a reserved
interpretation of sentiments and has a more definitive stance on each aspect.</p>
        <p>The absence of a neutral category in this model could imply a more decisive perception
from respondents, either favourably or unfavourably, towards the product or service's
various aspects. This model offers clearer insights into what aspects are highly valued and
which might be problematic or less satisfactory to users. The significant number of negative
sentiments in aspects like "user-friendly/easy" underscores specific areas where
improvements could substantially impact user satisfaction.</p>
        <p>Overall, the sentiment analysis in model 5 indicates a predominantly positive reception of
various aspects (accessibility, effectiveness, innovation, and user-friendliness) related to the
main context of tweets. This positive reception could reflect well on the public's perception of
the efforts, information, and innovations shared on Twitter concerning the topic (e.g.,
Automation). For example, in the context of automation, it suggests that it is seen as a key driver
of progress and efficiency. It highlights areas where the response or information
dissemination is particularly strong, such as accessibility, effectiveness, and performance. However, the
presence of negative sentiments, even in small numbers, suggests room for improvement in
making every aspect as effective, innovative, and valuable as possible. The
"Juliensimon/reviews-sentiment-analysis" (model 6) analysis shows a generally positive outlook on the
evaluated product or service, with strengths in accessibility, user-friendliness, and effectiveness.
However, the notable presence of neutral sentiments, especially in effectiveness, performance,
and usefulness, signals a critical insight: a considerable group of users or customers is not
fully swayed. This neutrality might indicate uncertainty, lack of information, or mismatched
expectations, which could be addressed through improved communication, enhancements to
the product/service, or more targeted user education.</p>
        <p>The significant number of neutral sentiments regarding usefulness poses a particularly
interesting challenge, suggesting a need for the product/service to more clearly demonstrate its
benefits or adapt to better meet user needs. In contrast, the high positivity in
user-friendliness/ease of use highlights a clear advantage, suggesting that efforts to make the
product/service accessible and easy to use are paying off.</p>
        <p>This nuanced picture painted by the sentiment analysis points towards opportunities to
convert neutral perceptions into positive ones, potentially by addressing specific areas of
concern highlighted by the study or bolstering communication around the product/service's
strengths.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Factors Influencing ABSA's Performance</title>
        <p>During this study, we noticed that several factors influence the role of ABSA in a large
organisation's understanding of its users. These factors will influence the ABSA
performance and should be strategically planned. For example, the right model and
selecting the right aspects adjusted to the organisational environment should be
considered. One of this study's first findings is the significance of model selection, training,
and fine-tuning in Aspect-Based Sentiment Analysis (ABSA); it is crucial to consider the
underlying principles of natural language processing (NLP) and machine learning. The
choice of model directly impacts the accuracy, relevance, and depth of insights derived from
sentiment analysis. These factors are central to ensuring the analysis is precise and
actionable for improving system and product design or enhancing user interaction.</p>
        <p>Model Selection: The choice of a sentiment analysis model should align with the specific
nature of the data (e.g., tweets, financial news, product reviews) and the analysis goals. For
example, models trained on specific data types, such as
"Sonny4Sonnix/twitter-roberta-basesentimental-analysis-of-covid-tweets," may perform better on similar datasets due to their
specialised vocabulary and context understanding. This concept is echoed in academic
literature. Zhang et al. (2018) discussed how model performance in sentiment analysis could
significantly vary depending on the domain-specific data it was trained on, emphasising the
importance of model-domain alignment [60].</p>
        <p>Training and Fine-tuning: The process of training and fine-tuning a model on
domainspecific datasets enhances its ability to interpret the nuances of sentiment within that domain
accurately. Ruder et al. (2019) highlight the effectiveness of fine-tuning pre-trained language
models on task-specific datasets, noting substantial improvements in model performance
across various NLP tasks, including sentiment analysis [61].</p>
        <p>In summary, the choice of sentiment analysis model, coupled with strategic training and
fine-tuning, plays a pivotal role in extracting meaningful and actionable insights from data.
These insights are invaluable for refining product designs and enhancing user interactions,
underscoring the intertwined relationship between advanced NLP techniques and practical
application in product development and user experience optimisation.</p>
        <p>The foundation of strategic approaches can be illustrated in Figure 10.</p>
        <p>In addition to the choice of model, training, and fine-tuning, there are several other
crucial factors and strategic approaches that companies should consider based on ABSA
results to improve product design, user experience (UX), and customer interactions:</p>
        <sec id="sec-4-2-1">
          <title>1. Combining Multiple Models for a Holistic View</title>
          <p>Complementarity: No single model may capture all nuances of sentiment across diverse
datasets. Combining insights from multiple models, such as those trained on general social
media content and others fine-tuned on domain-specific data, can offer a more
comprehensive view of user sentiment. This approach is supported by the ensemble
methods in machine learning, where multiple models are used together to improve
predictions [62].</p>
        </sec>
        <sec id="sec-4-2-2">
          <title>2. Actionable Insights for Product and Service Enhancement Prioritisation: Analyse ABSA results to identify and prioritise areas for improvement. For example, if "user-friendly/easy" receives mixed sentiments, focus on enhancing the UX to be more intuitive.</title>
          <p>User-Centric Design: Incorporate user feedback loops, leveraging ABSA to guide the
iterative design process. By focusing on aspects that directly impact user satisfaction and
engagement, companies can tailor their products more effectively to meet user needs.</p>
        </sec>
        <sec id="sec-4-2-3">
          <title>3. Strategic Marketing and Communication Targeted Messaging: Use ABSA insights to craft marketing messages highlighting the most positively received features and addressing common misconceptions or negative perceptions.</title>
          <p>Feature Development: Align product development goals with user sentiment trends,
investing in features most valued by users, as identified through sentiment analysis.</p>
        </sec>
        <sec id="sec-4-2-4">
          <title>4. Enhancing User Interaction</title>
          <p>Personalisation: Tailor user experiences based on individual or segment-specific
feedback. Personalisation can significantly enhance user engagement and satisfaction, as
indicated by the positive reception of accessibility and effectiveness.</p>
          <p>Feedback Integration: Implement mechanisms for continuous user feedback collection and
analysis. This ongoing engagement helps refine the user experience, aligning products with
user expectations.</p>
        </sec>
        <sec id="sec-4-2-5">
          <title>5. Continuous Monitoring and Adaptation</title>
          <p>Sentiment Tracking: Regularly monitor user sentiment to capture changing trends,
preferences, and pain points. This proactive approach allows for timely adjustments to
product features or UX strategies.</p>
          <p>Cultural and Contextual Sensitivity: Be aware of the cultural and contextual factors
influencing sentiment analysis results. Tailoring products and communications to respect
these factors can enhance global user satisfaction and market reach.</p>
          <p>In conclusion, while selecting, training, and fine-tuning the right models are foundational
for effective ABSA, leveraging these insights requires a strategic, multifaceted approach. By
combining insights from</p>
          <p>multiple models, focusing on actionable improvements, and
maintaining a cycle of feedback and adaptation, companies can significantly enhance their
products, services, and user experiences.</p>
          <p>Life cycle chart illustrating the key stages involved in leveraging insights for
effective</p>
          <p>Aspect-Based Sentiment Analysis (ABSA) in a strategic, multifaceted
approach illustrated in Table 2.
6- Continuous</p>
          <p>Improvement Loop</p>
          <p>Stages
Identify relevant data sources and models for ABSA.</p>
          <p>Choose appropriate machine learning algorithms and techniques.</p>
          <p>Select relevant aspects and sentiment categories for analysis.</p>
          <p>Train selected models using annotated data.</p>
          <p>Fine-tune models to improve accuracy and performance.</p>
          <p>Validate models using cross-validation techniques.</p>
          <p>Integrate multiple models to harness diverse insights.</p>
          <p>Develop a unified framework for analysis.</p>
          <p>Ensure compatibility and interoperability between different models.</p>
          <p>Extract actionable insights from the integrated model outputs.</p>
          <p>Identify areas for improvement based on sentiment analysis results.</p>
          <p>Translate insights into actionable strategies and decisions.</p>
          <p>Gather feedback from users and stakeholders.</p>
          <p>Incorporate feedback to refine models and analysis techniques.</p>
          <p>Continuously adapt the ABSA system to evolving requirements and
contexts.</p>
          <p>Iterate through the life cycle to refine models and processes.</p>
          <p>Monitor performance metrics and adjust strategies accordingly.</p>
          <p>Strive for ongoing enhancement of products, services, and user
experiences.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Discussion</title>
      <p>We believe the insights gained from ABSA can inform improvements in sociotechnical
systems. This can be as small as understanding users' needs, pains, or any gains they want
from a system or as a strategic insight for improving the design and functionality of S.T.
systems. This section discusses the number of headings, such as model labels, model
interpretation, and practical implications of our tweet analysis influencing S.T. systems. The
Tweets dataset is used as a use case but can be expanded to any other type of user-generated
comments.</p>
      <p>Models label positive, negative, neutral and irrelevant.</p>
      <p>In the context of sentiment analysis, sentiments are typically categorised into several
types, most commonly positive, negative, and neutral. Some models might only distinguish
between positive and negative sentiments, effectively forcing every piece of analysed text into
one of these two categories. This approach can simplify the analysis but might miss out on
capturing sentiments that are not positive or negative but neutral or mixed.</p>
      <p>Including a neutral category allows for a more detailed and nuanced understanding of
sentiment. Neutral sentiment captures a range of neither explicitly positive nor negative
responses. For instance, a review might state, "The product works as expected," without
expressing delight (positive) or dissatisfaction (negative). This statement would likely be
classified as neutral. Recognising these neutral sentiments is important because they can indicate
areas where customers or users are unsure. This can be particularly valuable for
understanding aspects of a product, service, or topic where opinions are not strongly polarised but might
indicate satisfaction without enthusiasm or a lack of significant complaints.</p>
      <p>By acknowledging neutral sentiments, the model provides a more complete picture of how
people feel about a subject. It allows analysts to identify what aspects are liked or disliked and
what factors might be overlooked, taken for granted, or seen as just satisfactory. This
nuanced view is crucial for making informed decisions about improvements or where efforts
should be focused to enhance satisfaction, engagement, or perception. Some of the common
strategies for addressing aspects with highly neutral sentiments are,</p>
      <p>Enhancing communication: as users may not appreciate or fully recognise certain
features, better communication about the unique selling points can turn Neutral comments
towards more positive ones. For example, they may not understand how effective or useful an
automated system is for the organisation. This directly impacts advocating the social side of
sociotechnical computing.</p>
      <p>User Education: It is worth educating the users while communicating with them and
providing more information, tutorials, or demonstrations on effectiveness, ease of use, and
performance to help users appreciate these aspects more, potentially converting neutral
sentiments into positive ones.</p>
      <p>Continuous Improvement: Neutral sentiment often indicates room for improvement.
Constant updates, addressing user feedback, and introducing innovative features can
gradually enhance user perceptions. It has been proven that using systems helps users
understand the system better and improves interaction. Collecting user reviews and
feedback and comparing the ASBA in different stages of product/service development can be
valuable and help developers and decision-makers.</p>
      <p>Segmentation Analysis: If possible, investigate the segments of users who rated these
aspects neutrally, which can bring new insights for design or system improvements.
Understanding their expectations or comparing their feedback with those who expressed
strong positive or negative sentiments might offer actionable insights. For example, in large
organisations, segmentation is not possible before the design, but it may be easier to look into
it while going through the trials/Pilots.</p>
      <p>In summary, while high neutral sentiments suggest satisfactory perceptions, they also
highlight opportunities for making the aspects of effectiveness, user-friendliness, and
performance stand out more distinctly to users, potentially turning neutrality into
advocacy.
6. Conclusion
Our research underscores the pivotal role of Aspect-Based Sentiment Analysis (ABSA) as a
tool within sociotechnical approaches in large organisations. By leveraging ABSA, we
extracted nuanced insights from user-generated content, particularly tweets, to better
understand user sentiments regarding various aspects of automated systems and smart
workplaces. These aspects were selected from the UX goals of other studies. We proposed
integrating ABSA into the design process combines data-driven insights with humancentred design
principles, which is especially valuable in organisations where collecting user feedback is
challenging.</p>
      <p>Our key findings reveal that ABSA offers a fine-grained understanding of user opinions
by linking sentiments to specific aspects or features. This enables researchers, developers, and
organisations to more precisely identify system strengths and areas for improvement. We
evaluated the efficacy of six different ABSA models based on architectures like RoBERTa and
DistilRoBERTa, each demonstrating varying capabilities in capturing and analysing
sentiments. This emphasises the need for careful model selection and fine-tuning. The analysis
also highlighted the importance of considering contextual factors, such as cultural and
situational influences, which affect user sentiments. This consideration is crucial for designing
technically sound and socially relevant systems.</p>
      <p>These analyses have several implications for sociotechnical systems. Firstly, integrating
ABSA into the UX design process allows organisations to develop more user-centric systems
that meet specific user needs and preferences, improving overall user satisfaction and
engagement. Secondly, ABSA enables decision-makers to base their strategies on concrete
data-driven insights, facilitating better-informed decisions regarding system design and
functionality. Lastly, continuous monitoring and adaptation based on ABSA findings help
organisations keep their systems aligned with user expectations and technological
advancements.</p>
      <p>Future research should explore additional sociotechnical factors such as trust,
accessibility, and ethical considerations to understand user experience in automated
environments comprehensively. Further studies are needed to address the challenges of
crosslinguistic sentiment analysis, ensuring that models accurately capture sentiments across
different languages and cultural contexts. Additionally, combining multiple ABSA models could
provide a holistic view of user sentiments, enhancing the accuracy and reliability of
sentiment analysis.</p>
      <p>In conclusion, integrating ABSA into sociotechnical computing frameworks presents a
powerful approach to improving user experience and system effectiveness. By addressing
social and technical dimensions, organisations can foster better user interactions and achieve
more sustainable digital transformation.
[21] Jenkins, T., Boer, L., Brigitta Busboom, J., Simonsen, I. &amp;Oslash;stby: The Future
Supermarket: A Case Study of Ethnographic Experiential Futures. In: Proceedings of the
11th Nordic Conference on Human-Computer Interaction: Shaping Experiences,
Shaping Society. pp. 1–13. ACM, Tallinn Estonia (2020).
https://doi.org/10.1145/3419249.3420130.
[22] Revenäs, Å., Opava, C.H., Martin, C., Demmelmaier, I., Keller, C., Åsenlöf, P.: Development
of a Web-Based and Mobile App to Support Physical Activity in Individuals With
Rheumatoid Arthritis: Results From the Second Step of a Co-Design Process. JMIR Res
Protoc. 4, e22 (2015). https://doi.org/10.2196/resprot.3795.
[23] Behagel, J., Turnhout, E.: Democratic Legitimacy in the Implementation of the Water
Framework Directive in the Netherlands: Towards Participatory and Deliberative
Norms? Journal of Environmental Policy &amp; Planning. 13, 297–316 (2011).
https://doi.org/10.1080/1523908X.2011.607002.
[24] Hertzum, M., Bansler, J.P., Havn, E.C., Simonsen, J.: Pilot Implementation: Learning from
Field Tests in IS Development. CAIS. 30, (2012).
https://doi.org/10.17705/1CAIS.03020.
[25] Jack, D., Boian, R., Merians, A.S., Tremaine, M., Burdea, G.C., Adamovich, S.V., Recce, M.,
Poizner, H.: Virtual reality-enhanced stroke rehabilitation. IEEE Trans. Neural Syst.</p>
      <p>Rehabil. Eng. 9, 308–318 (2001). https://doi.org/10.1109/7333.948460.
[26] Brooks, J., McCluskey, S., Turley, E., King, N.: The Utility of Template Analysis in
Qualitative Psychology Research. Qualitative Research in Psychology. 12, 202–222
(2015). https://doi.org/10.1080/14780887.2014.955224.
[27] Kraemer, S., Carayon, P., Sanquist, T.F.: Human and organizational factors in security
screening and inspection systems: conceptual framework and key research needs.</p>
      <p>Cogn Tech Work. 11, 29–41 (2009). https://doi.org/10.1007/s10111-008-0121-4.
[28] Langan-Fox, J., Sankey, M.J., Canty, J.M.: Human Factors Measurement for Future Air
Traffic Control Systems. Hum Factors. 51, 595–637 (2009).
https://doi.org/10.1177/0018720809355278.
[29] Ylikauppila, M., Toivonen, S., Kulju, M., Jokela, M.: Understanding the Factors Affecting
UX and Technology Acceptance in the Context of Automated Border Controls. In: 2014
IEEE Joint Intelligence and Security Informatics Conference. pp. 168–175. IEEE, The
Hague, Netherlands (2014). https://doi.org/10.1109/JISIC.2014.33.
[30] Kulju, M., Ylikauppila, M., Toivonen, S., Salmela, L.: A Framework for Understanding
Human Factors Issues in Border Control Automation. In: Barricelli, B.R., Roto, V.,
Clemmensen, T., Campos, P., Lopes, A., Gonçalves, F., and Abdelnour-Nocera, J. (eds.)
Human Work Interaction Design. Designing Engaging Automation. pp. 215–228.
Springer International Publishing, Cham (2019).
https://doi.org/10.1007/978-3-030-052973_15.
[31] ISO 26800: ISO 26800:2011(en), Ergonomics — General approach, principles and concepts,
https://www.iso.org/obp/ui/#iso:std:iso:26800:ed-1:v1:en, last accessed 2022/12/30.
[32] Bostrom, R.P., Heinen, J.S.: MIS Problems and Failures: A Socio-Technical Perspective.</p>
      <p>Part I: The Causes. MIS Quarterly. 1, 17 (1977). https://doi.org/10.2307/248710.
U. (eds.) Eurasian Business Perspectives. pp. 129–138. Springer International
Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-48505-4_9.
[47] Gupta, M., Mishra, A., Manral, G., Ansari, G.: Aspect-category based Sentiment Analysis
on Dynamic Reviews. In: 2020 IEEE 5th International Conference on Computing
Communication and Automation (ICCCA). pp. 492–496. IEEE, Greater Noida, India
(2020). https://doi.org/10.1109/ICCCA49541.2020.9250914.
[48] Banjar, A., Ahmed, Z., Daud, A., Ayaz Abbasi, R., Dawood, H.: Aspect-Based Sentiment
Analysis for Polarity Estimation of Customer Reviews on Twitter. Computers, Materials
&amp; Continua. 67, 2203–2225 (2021). https://doi.org/10.32604/cmc.2021.014226.
[49] Zhang, Y., Du, J., Ma, X., Wen, H., Fortino, G.: Aspect-Based Sentiment Analysis for User</p>
      <p>Reviews. Cogn Comput. 13, 1114–1127 (2021). https://doi.org/10.1007/s12559-021-09855-4.
[50] Navarro, A., Tapiador, F.J.: Twitch as a privileged locus to analyze young people’s
attitudes in the climate change debate: a quantitative analysis. Humanit Soc Sci
Commun. 10, 844 (2023). https://doi.org/10.1057/s41599-023-02377-4.
[51] Jahanbin, K., Chahooki, M.A.Z.: Aspect-Based Sentiment Analysis of Twitter Influencers
to Predict the Trend of Cryptocurrencies Based on Hybrid Deep Transfer Learning
Models. IEEE Access. 11, 121656–121670 (2023).
https://doi.org/10.1109/ACCESS.2023.3327060.
[52] Tahir, R., Naeem, M.A.: A Machine Learning based Approach to Identify User Interests
from Social Data. In: 2022 24th International Multitopic Conference (INMIC). pp. 1–6.
IEEE, Islamabad, Pakistan (2022).
https://doi.org/10.1109/INMIC56986.2022.9972956.
[53] Nallakaruppan, M.K., Srivastava, G., Gadekallu, T.R., Reddy, P.K., Krishnan, S., Polap, D.:
Child Tracking and Prediction of Violence on Children In Social Media Using Natural
Language Processing and Machine Learning. In: Rutkowski, L., Scherer, R.,
Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., and Zurada, J.M. (eds.) Artificial
Intelligence and Soft Computing. pp. 560–569. Springer Nature Switzerland, Cham
(2023). https://doi.org/10.1007/978-3-031-42505-9_47.
[54] Lee, I., Lee, K.: The Internet of Things (IoT): Applications, investments, and challenges
for enterprises. Business Horizons. 58, 431–440 (2015).
https://doi.org/10.1016/j.bushor.2015.03.008.
[55] Schrauf, S., Berttram, P.: Industry 4.0 - How digitization makes the supply chain more
efficient, agile, and customer-focused,
https://www.pwc.ch/en/publications/2017/how-digitization-makes-the-supplychain-more-efficient-pwc-2016.pdf, (2016).
[56] Jordan, M.I., Mitchell, T.M.: Machine learning: Trends, perspectives, and prospects.</p>
      <p>Science. 349, 255–260 (2015). https://doi.org/10.1126/science.aaa8415.
[57] EDPS Supervisory: EDPS SUPERVISORY OPINION ON THE USE OF SOCIAL MEDIA
MONITORING FOR EPIDEMIC INTELLIGENCE PURPOSES BY THE EUROPEAN
CENTRE FOR DISEASE PREVENTION AND CONTROL (‘ECDC’), https://
www.edps.europa.eu/system/files/2023-11/23-1109_EDPS_Opinion_2021_0830_EN.pdf, (2023).
[58] Anwar, A., Rehman, I.U., Kostadinova Dicheva, N., Haq, I.U.: Analysing User Experience
of Dynamic Group Formation (DGF) in Intelligent Tutor Collaborative Learning (ITSCL)
using Aspect-Based Sentiment Analysis. In: XXIII International Conference on Human
Computer Interaction. pp. 1–5. ACM, Lleida Spain (2023).
https://doi.org/10.1145/3612783.3612788.
[59] Saadati, P., Abdelnour-Nocera, J., Clemmensen, T.: Co-designing Prototypes for User
Experience and Engagement in Automation: Case Study of London-based Airport
Future Workplace. In: Bhutkar, G., Barricelli, B.R., Xiangang, Q., Clemmensen, T.,
Gonçalves, F., Abdelnour-Nocera, J., Lopes, A., Lyu, F., Zhou, R., and Hou, W. (eds.)
Human Work Interaction Design. Artificial Intelligence and Designing for a Positive
Work Experience in a Low Desire Society. pp. 158–177. Springer International
Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-02904-2_8.
[60] Zhang, Z., Robinson, D., Tepper, J.: Detecting Hate Speech on Twitter Using a
Convolution-GRU Based Deep Neural Network. In: Gangemi, A., Navigli, R., Vidal, M.-E.,
Hitzler, P., Troncy, R., Hollink, L., Tordai, A., and Alam, M. (eds.) The Semantic Web. pp.
745–760. Springer International Publishing, Cham (2018).
https://doi.org/10.1007/978-3-319-93417-4_48.
[61] Ruder, S., Peters, M.E., Swayamdipta, S., Wolf, T.: Transfer Learning in Natural Language
Processing. In: Proceedings of the 2019 Conference of the North. pp. 15–18. Association
for Computational Linguistics, Minneapolis, Minnesota (2019).
https://doi.org/10.18653/v1/N19-5004.
[62] Polikar, R.: Ensemble based systems in decision making. IEEE Circuits Syst. Mag. 6, 21–
45 (2006). https://doi.org/10.1109/MCAS.2006.1688199.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Nguyen</surname>
            ,
            <given-names>T.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shirai</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>PhraseRNN: Phrase Recursive Neural Network for Aspectbased Sentiment Analysis</article-title>
          .
          <source>In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing</source>
          . pp.
          <fpage>2509</fpage>
          -
          <lpage>2514</lpage>
          . Association for Computational Linguistics, Lisbon, Portugal (
          <year>2015</year>
          ). https://doi.org/10.18653/v1/
          <fpage>D15</fpage>
          -1298.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Pang</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Opinion Mining and Sentiment Analysis</article-title>
          . now Publishers Inc (
          <year>2008</year>
          ). https://doi.org/10.1561/9781601981516.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Jiang</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yu</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhou</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhao</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Target-dependent Twitter Sentiment Classification</article-title>
          . In: Lin,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Matsumoto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            , and
            <surname>Mihalcea</surname>
          </string-name>
          , R. (eds.)
          <article-title>Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies</article-title>
          . pp.
          <fpage>151</fpage>
          -
          <lpage>160</lpage>
          . Association for Computational Linguistics, Portland, Oregon, USA (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shen</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mi</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>Attention-based Sentiment Reasoner for aspect-based sentiment analysis</article-title>
          .
          <source>Hum. Cent. Comput. Inf. Sci. 9</source>
          ,
          <issue>35</issue>
          (
          <year>2019</year>
          ). https://doi.org/10.1186/s13673-019-0196-3.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Schouten</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Frasincar</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          :
          <article-title>Survey on Aspect-Level Sentiment Analysis</article-title>
          .
          <source>IEEE Trans. Knowl. Data Eng</source>
          .
          <volume>28</volume>
          ,
          <fpage>813</fpage>
          -
          <lpage>830</lpage>
          (
          <year>2016</year>
          ). https://doi.org/10.1109/TKDE.
          <year>2015</year>
          .
          <volume>2485209</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chu</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saidani</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhou</surname>
          </string-name>
          , W.:
          <article-title>IoT-based location and quality decisionmaking in emerging shared parking facilities with competition</article-title>
          .
          <source>Decision Support Systems</source>
          .
          <volume>134</volume>
          ,
          <issue>113301</issue>
          (
          <year>2020</year>
          ). https://doi.org/10.1016/j.dss.
          <year>2020</year>
          .
          <volume>113301</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Zarandi</surname>
            ,
            <given-names>A.K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mirzaei</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>A survey of aspect-based sentiment analysis classification with a focus on graph neural network methods</article-title>
          .
          <source>Multimed Tools Appl</source>
          .
          <volume>83</volume>
          ,
          <fpage>56619</fpage>
          -
          <lpage>56695</lpage>
          (
          <year>2023</year>
          ). https://doi.org/10.1007/s11042-023-17701-y.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Baxter</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sommerville</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>Socio-technical systems: From design methods to systems engineering</article-title>
          .
          <source>Interacting with Computers</source>
          .
          <volume>23</volume>
          ,
          <fpage>4</fpage>
          -
          <lpage>17</lpage>
          (
          <year>2011</year>
          ). https://doi.org/10.1016/j.intcom.
          <year>2010</year>
          .
          <volume>07</volume>
          .003.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Mumford</surname>
            ,
            <given-names>E.: A</given-names>
          </string-name>
          <string-name>
            <surname>Socio-Technical Approach</surname>
          </string-name>
          to Systems Design.
          <source>Requirements Eng</source>
          .
          <volume>5</volume>
          ,
          <fpage>125</fpage>
          -
          <lpage>133</lpage>
          (
          <year>2000</year>
          ). https://doi.org/10.1007/PL00010345.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Pontiki</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Galanis</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Papageorgiou</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Manandhar</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Androutsopoulos</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>SemEval2015 Task 12: Aspect Based Sentiment Analysis</article-title>
          .
          <source>In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval</source>
          <year>2015</year>
          ). pp.
          <fpage>486</fpage>
          -
          <lpage>495</lpage>
          . Association for Computational Linguistics, Denver, Colorado (
          <year>2015</year>
          ). https://doi.org/10.18653/v1/
          <fpage>S15</fpage>
          -2082.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Teng</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          , Zhang, Y.:
          <article-title>Inducing Target-Specific Latent Structures for Aspect Sentiment Classification</article-title>
          .
          <source>In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)</source>
          . pp.
          <fpage>5596</fpage>
          -
          <lpage>5607</lpage>
          . Association for Computational Linguistics,
          <string-name>
            <surname>Online</surname>
          </string-name>
          (
          <year>2020</year>
          ). https://doi.org/10.18653/v1/
          <year>2020</year>
          .emnlpmain.
          <volume>451</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Ghosh</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Byrden</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , Wilson, J.:
          <source>Future Ready Enterprise Systems</source>
          , https://www.accenture.com/gb-en/insights/future-systems/future-readyenterprise-systems,
          <source>last accessed</source>
          <year>2022</year>
          /07/29.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Trabert</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Beiner</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lehmann</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kinkel</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>Digital Value Creation in sociotechnical Systems: Identification of challenges and recommendations for human work in manufacturing SMEs</article-title>
          .
          <source>Procedia Computer Science</source>
          .
          <volume>200</volume>
          ,
          <fpage>471</fpage>
          -
          <lpage>481</lpage>
          (
          <year>2022</year>
          ). https://doi.org/10.1016/j.procs.
          <year>2022</year>
          .
          <volume>01</volume>
          .245.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Abdelnour-Nocera</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oussena</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Burns</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Human Work Interaction Design of the Smart University</article-title>
          . In: Abdelnour Nocera,
          <string-name>
            <given-names>J.</given-names>
            ,
            <surname>Barricelli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.R.</given-names>
            ,
            <surname>Lopes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Campos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            , and
            <surname>Clemmensen</surname>
          </string-name>
          , T. (eds.)
          <article-title>Human Work Interaction Design</article-title>
          .
          <article-title>Work Analysis and Interaction Design Methods for Pervasive and Smart Workplaces</article-title>
          . pp.
          <fpage>127</fpage>
          -
          <lpage>140</lpage>
          . Springer International Publishing,
          <string-name>
            <surname>Cham</surname>
          </string-name>
          (
          <year>2015</year>
          ). https://doi.org/10.1007/978-3-
          <fpage>319</fpage>
          -27048-
          <issue>7</issue>
          _
          <fpage>9</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Ross</surname>
            ,
            <given-names>J.W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sebastian</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Beath</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mocker</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Moloney</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fonstad</surname>
          </string-name>
          , N.:
          <source>Designing and Executing Digital Strategies</source>
          .
          <volume>17</volume>
          (
          <year>2016</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Kadir</surname>
            ,
            <given-names>B.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Broberg</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Conceição</surname>
            ,
            <given-names>C.S.</given-names>
          </string-name>
          <article-title>da: Current research and future perspectives on human factors and ergonomics in Industry 4.0</article-title>
          .
          <string-name>
            <surname>Computers</surname>
          </string-name>
          &amp; Industrial Engineering.
          <volume>137</volume>
          ,
          <issue>106004</issue>
          (
          <year>2019</year>
          ). https://doi.org/10.1016/j.cie.
          <year>2019</year>
          .
          <volume>106004</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>O</given-names>
            <surname>'Conor</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Dyson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Payne</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            ,
            <surname>Franklin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Lemarchand</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            ,
            <surname>Tulquois</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            ,
            <surname>Allen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            ,
            <surname>Forster</surname>
          </string-name>
          ,
          <string-name>
            <surname>C.</surname>
          </string-name>
          :
          <article-title>Digital transformation in the aviation sector | Insights | DLA Piper Global Law Firm</article-title>
          , https://www.dlapiper.com/en/global/insights/publications/2017/04/digitaltransformation
          <article-title>-in-the-aviation-sector</article-title>
          ,
          <source>last accessed</source>
          <year>2022</year>
          /08/05.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Kymalainen</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kaasinen</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Aikala</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hakulinen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Heimonen</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Paunonen</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ruotsalainen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lehtikunnas</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mannonen</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Evaluating Future Automation Work in Process Plants with an Experience-Driven Science Fiction Prototype</article-title>
          .
          <source>In: 2016 12th International Conference on Intelligent Environments (IE)</source>
          . pp.
          <fpage>54</fpage>
          -
          <lpage>61</lpage>
          . IEEE, London, United
          <string-name>
            <surname>Kingdom</surname>
          </string-name>
          (
          <year>2016</year>
          ). https://doi.org/10.1109/IE.
          <year>2016</year>
          .
          <volume>17</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Evers</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jonoski</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Almoradie</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lange</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Collaborative decision making in sustainable flood risk management: A socio-technical approach and tools for participatory governance</article-title>
          .
          <source>Environmental Science &amp; Policy. 55</source>
          ,
          <fpage>335</fpage>
          -
          <lpage>344</lpage>
          (
          <year>2016</year>
          ). https://doi.org/10.1016/j.envsci.
          <year>2015</year>
          .
          <volume>09</volume>
          .009.
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Simonsen</surname>
          </string-name>
          , Jesper and Robertson, Toni:
          <article-title>Routledge international handbook of participatory design</article-title>
          .
          <source>Routledge</source>
          (
          <year>2012</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>J.</given-names>
            <surname>Hester</surname>
          </string-name>
          ,
          <string-name>
            <surname>A.</surname>
          </string-name>
          :
          <article-title>Socio-technical systems theory as a diagnostic tool for examining underutilization of wiki technology</article-title>
          .
          <source>The Learning Organization</source>
          .
          <volume>21</volume>
          ,
          <fpage>48</fpage>
          -
          <lpage>68</lpage>
          (
          <year>2014</year>
          ). https://doi.org/10.1108/TLO-10-2012-0065.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [34]
          <string-name>
            <surname>He</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>W.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ng</surname>
            ,
            <given-names>H.T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dahlmeier</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Exploiting Document Knowledge for Aspectlevel Sentiment Classification</article-title>
          .
          <source>In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)</source>
          . pp.
          <fpage>579</fpage>
          -
          <lpage>585</lpage>
          . Association for Computational Linguistics, Melbourne, Australia (
          <year>2018</year>
          ). https://doi.org/10.18653/v1/
          <fpage>P18</fpage>
          -2092.
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [35]
          <string-name>
            <given-names>Al</given-names>
            <surname>Sabbagh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.: Cybersecurity</given-names>
            <surname>Incident Response: A Socio-Technical Approach</surname>
          </string-name>
          . Department of Computer and Systems Sciences, Stockholm University, Stockholm (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Alsheail</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Alexandrovskz</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gerling</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>Designing for Disengagement: Challenges and Opportunities for Game Design to Support Children's Exit From Play</article-title>
          , https://arxiv.org/abs/2303.15400, (
          <year>2023</year>
          ). https://doi.org/10.48550/ARXIV.2303.15400.
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [37]
          <string-name>
            <given-names>O</given-names>
            <surname>'Brien</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.L.</given-names>
            ,
            <surname>Toms</surname>
          </string-name>
          ,
          <string-name>
            <surname>E.G.</surname>
          </string-name>
          :
          <article-title>What is user engagement? A conceptual framework for defining user engagement with technology</article-title>
          .
          <source>J. Am. Soc. Inf. Sci</source>
          .
          <volume>59</volume>
          ,
          <fpage>938</fpage>
          -
          <lpage>955</lpage>
          (
          <year>2008</year>
          ). https://doi.org/10.1002/asi.20801.
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [38]
          <string-name>
            <surname>Endsley</surname>
            ,
            <given-names>M.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kaber</surname>
            ,
            <given-names>D.B.</given-names>
          </string-name>
          :
          <article-title>Level of automation effects on performance, situation awareness and workload in a dynamic control task</article-title>
          .
          <source>Ergonomics</source>
          .
          <volume>42</volume>
          ,
          <fpage>462</fpage>
          -
          <lpage>492</lpage>
          (
          <year>1999</year>
          ). https://doi.org/10.1080/001401399185595.
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [39]
          <string-name>
            <surname>Parasuraman</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Riley</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          : Humans and Automation: Use, Misuse, Disuse, Abuse.
          <source>Hum Factors</source>
          .
          <volume>39</volume>
          ,
          <fpage>230</fpage>
          -
          <lpage>253</lpage>
          (
          <year>1997</year>
          ). https://doi.org/10.1518/001872097778543886.
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [40]
          <string-name>
            <surname>Parasuraman</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sheridan</surname>
            ,
            <given-names>T.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wickens</surname>
          </string-name>
          , C.D.:
          <article-title>A model for types and levels of human interaction with automation</article-title>
          .
          <source>IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans</source>
          .
          <volume>30</volume>
          ,
          <fpage>286</fpage>
          -
          <lpage>297</lpage>
          (
          <year>2000</year>
          ). https://doi.org/10.1109/3468.844354.
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [41]
          <string-name>
            <surname>Hassenzahl</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>User experience and experience design</article-title>
          .
          <source>The encyclopedia of humancomputer interaction. 2</source>
          , (
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [42]
          <string-name>
            <surname>Hekkert</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mostert</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stompff</surname>
          </string-name>
          , G.:
          <article-title>Dancing with a machine: a case of experiencedriven design</article-title>
          .
          <source>In: Proceedings of the 2003 international conference on Designing pleasurable products and interfaces - DPPI '03</source>
          . p.
          <fpage>114</fpage>
          . ACM Press, Pittsburgh, PA, USA (
          <year>2003</year>
          ). https://doi.org/10.1145/782896.782925.
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [43]
          <string-name>
            <surname>McCarthy</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wright</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Technology as experience</article-title>
          .
          <source>interactions. 11</source>
          ,
          <fpage>42</fpage>
          -
          <lpage>43</lpage>
          (
          <year>2004</year>
          ). https://doi.org/10.1145/1015530.1015549.
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [44]
          <string-name>
            <surname>Hassenzahl</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Diefenbach</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Göritz</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Needs, affect, and interactive products - Facets of user experience</article-title>
          .
          <source>Interacting with Computers</source>
          .
          <volume>22</volume>
          ,
          <fpage>353</fpage>
          -
          <lpage>362</lpage>
          (
          <year>2010</year>
          ). https://doi.org/10.1016/j.intcom.
          <year>2010</year>
          .
          <volume>04</volume>
          .002.
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [45]
          <string-name>
            <surname>Väätäjä</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Savioja</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Roto</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Olsson</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Varsaluoma</surname>
            ,
            <given-names>J.:</given-names>
          </string-name>
          <article-title>User experience goals as a guiding light in design and development - Early findings</article-title>
          . In:
          <article-title>INTERACT 2015 Adjunct proceedings</article-title>
          . pp.
          <fpage>521</fpage>
          -
          <lpage>527</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [46]
          <string-name>
            <surname>Cotfas</surname>
            ,
            <given-names>L.-A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Delcea</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nica</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>Analysing Customers' Opinions Towards Product Characteristics Using Social Media</article-title>
          . In: Bilgin,
          <string-name>
            <given-names>M.H.</given-names>
            ,
            <surname>Danis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            ,
            <surname>Demir</surname>
          </string-name>
          ,
          <string-name>
            <surname>E.</surname>
          </string-name>
          , and Tony-Okeke,
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>