=Paper=
{{Paper
|id=Vol-3265/paper_4234
|storemode=property
|title=Using semantics to interpret learning needs by organizational climate surveys
|pdfUrl=https://ceur-ws.org/Vol-3265/paper_4234.pdf
|volume=Vol-3265
|authors=Eliana Brunetti,Sara Zuzzi,Daniela Pellegrini
|dblpUrl=https://dblp.org/rec/conf/telexbe/BrunettiZP22
}}
==Using semantics to interpret learning needs by organizational climate surveys ==
https://ceur-ws.org/Vol-3265/paper_4234.pdf
Using semantics to interpret learning needs by organizational
climate surveys
Eliana Brunetti1, Sara Zuzzi2 and Daniela Pellegrini2
1
University of Naples Federico II, 80133 Napoli, Italy
2
Piazza Copernico s.r.l. via Francesco Gentile 135/a, Rome, 00173, Italy
Abstract
Organizational climate survey is a typical tool used to understand and monitor individual, team
and organizational features factors that distinguish the company climate.
Performing this activity in the final phase of a pandemic means collecting information about
changes that remote working brought to the work processes and to the new each organizational
dimension, highlighting present or future transformation paths. It is also an implicit survey of
potential training needs. The construction of a new dimension of working spaces, which we
can describe as an organizational metaverse, happens thanks to the real and deep understanding
of changing needs and dynamics in the working processes. Organizational health in terms of
identity, relationships, processes, and work results, also represents a starting point for the
exploration of new learning needs of the organization. The transformation makes the
organization be more and more dynamic, making it to function as a multiform organism, in
which the multiple relational dimensions in real and the virtual world means an identity re-
construction and new forms of sociality: it may increase sharing and comparison and rise new
needs that the company must know. Combining the climate survey with NLP analysis tools to
understand the expressed meanings in written open questions and a collaborative activity,
represents a significant experimentation of the multiple dimensions connected between the
individual and the organization that becomes not only a physical place, processes, and results,
but an increasingly social dimension that must be fully understood. From the analysis of the
post-pandemic climate, the organization must be able to understand new useful meanings and
need to support its progressive improvement and the well-being and individual satisfaction.
Keywords 1
organizational climate analysis, semantics, collaborative learning, organizational learning
needs
1. Introduction
Organizational climate became over the years an interesting subject of research, turning into a lens
with which is possible to understand business and organizational environments from both structural and
psychosocial points of view. The focus on these studies has been supported by the interest in discovering
how to involve the human and social dimensions in organizational development. The climate survey,
in these cases, becomes a fundamental tool to identify the areas of improvement and the strengths to
work on and to start an effective changing process in the organization. Providing an unambiguous
definition of “organizational climate” is not immediate, since research on this topic has seen this subject
from different points of view. This led to the definition of several operational dimensions and as many
theoretical approaches. In literature, from the first structural vision [1], linking the organizational
climate exclusively to business data, there has been a gradual integration of perceptive [2], interactive
[3,4] and cultural [5] dimensions, considering first the individual and then the group as active builders
of the organizational climate. The currently shared theoretical models are part of the cultural approach,
which assigns to the organizational culture a fundamental role in the construction of the organizational
Proccedings of the Third Workshop on Technology Enhanced Learning Environments for Blended Education, June 10–11, 2022, Foggia, Italy
EMAIL: eliana.brunetti@unina.it (A. 1); szuzzi@pcopernico.it (A. 2); dpellegrini@pcopernico.it (A. 3)
©️ 2020 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
climate, together with the interaction between individuals. The chance of a constructive exchange,
source of shared perceptions, becomes real thanks to the existence of the organizational culture, a set
of meanings co-constructed and shared by the members, also guiding their thinking and behaviour [5].
The organizational climate is now intended as a set of perceptions and meanings shared by the members
of a company in relation to policies, ideologies, practices, and processes characterizing the daily work
experience of each. [3, 4, 6].
The project presented in this work is aimed to respond to the specific needs expressed by the
company Piazza Copernico, a rapidly growing reality which is now facing the pandemic COVID-19
emergency through a restructuring plan of working practices and methods of communication. Its
transformation was mainly linked to the "virtualization", more than a digitalization, of social
relationships, mainly as a change in the communication forms and logics, but also as a progressive
process of autonomy and responsibility at work for employees. Investigating these aspects is strategic
for a major transformation of work processes, which has resulted first in a forced change to full
smartworking (i.e. work from home), then a progressive rebalancing of the work experience in a hybrid
model, first regulated and then spontaneous [7]. While this process undoubtedly has positive advantages
for the company and for people, some aspects are highly critical in individualization vs isolation, in the
creation of subcultures, and in the implications of zoom fatigue [8], stress and burn out.
The theme of the research was the desire to investigate the individual and organizational impacts of
this transformation into what could be defined as the construction of a new dimension of working
spaces, that we can define as organizational metaverse. A single individual represents a virtual working
world connected to others, the work teams represent other dimensional space level, until to the company
as a universe itself. Thus, the individual is simultaneously immersed and interconnected in multiple
dimensions with different rules, logics, symbolism/meanings, and patterns of intervention, having to
regulate his action in parallel in all these worlds.
From the organizational point of view, therefore, there is the important need to understand
individuals’ experiences, their opinions, but also the aspects of concern, to be able to act on the available
leverages to improve working well-being, promoting a conscious use of available tools, and working
on soft strategic skills. The climate survey represents therefore a fundamental moment also to
investigate the latent formative needs, also in relation to the adaptive and transformative ability of the
people and to the current changes in companies. The climate survey is currently recognized as one of
the action strategies, described [9] as participatory and shared actions implemented within the company.
Specifically, action research [10], action science [11], action Inquiry [12] and action learning [13, 14]
are recognized as processes that involve intervention within organizations with the dual purpose of
bringing practical transformations in response to customer needs and allowing an advancement of
knowledge in the theoretical field. In this work is referred to action research (AR), in which the subject
and the object of investigation coincide, and the search process becomes the engine of change itself.
The origins of the AR are attributed to Lewin’s work [10], according to which social research had
to address both the study of general laws, and the diagnosis of specific situations. Currently, different
approaches of AR have been developed and used in the organizational field, but all are based on the
purpose of combining action and reflection, theory, and practice, through interaction and exchange with
others, with the aim of identifying practical solutions to real problems and seeing people, communities
and organizations grow and improve [15].
The analysis of the organizational climate, therefore, constitutes a first trigger for corporate change:
from the moment in which each member of the organization is invited to focus and explain their
experiences, a process of change that cannot be stopped [16] which also represents a signal of great
importance on the company’s willingness to pay due attention and to give due importance to all the
components of the organization. The AR is therefore a moment of understanding, improvement, and
review [17]. In line with these theoretical references, the internal R&D team started a research project
that involved the design of a multi-tool survey system for the detection of organizational climate, a data
collection session, and its analysis. One of the innovative aspects proposed by this project is the use of
semantic analysis to enrich interpretation of the collected data.
The opportunity to integrate this aspect was guaranteed using the internal software named
SemantiCase, developed by the team and using semantic analysis and sentiment analysis models. In the
case of the organizational climate survey conducted, the use of SemantiCase has made immediately
usable the information contained in the open written texts, which have enriched the climate definition
of the company, with greater detail the areas of improvement and its strengths.
2. Climate survey phases: the research in Piazza Copernico
The organizational climate survey project rose from a need born from Piazza Copernico, a
consulting company operating in the field of Digital Learning.
The main aim of the company was to observe the direct consequences of the changes due to the
pandemic emergency from COVID-19, which has affected our country in recent years. Since year 2020,
like many other Italian and worldwide companies, Piazza Copernico has started the construction of a
new balance on the remote working to maintain productivity and ensuring a high level of safety for all
employees. During the last two years, the gravity of the world situation, and the consequent Italian
regulations on health & safety, required the partial maintenance of the remote work, pushing the
company towards a rebuilding of working practices and communication methods. On a structural level,
this change seems to have been positively absorbed by the company, which has integrated the tools
needed by each member of staff to carry out their work in the best possible way and has seen a
significant increase in productivity. The future management intent, in fact, is oriented towards the
opportunity of a mix of remote working and work in the office. The company demand was, therefore,
to investigate the fundamental dimensions underlying the organizational climate, to obtain a descriptive
picture of the employees’ new global perceptions; the required objective is to highlight both the
strengths and the areas of improvement on which to operate for the future development of the
organization, with a particular focus on perceptions related to the new hybrid working mode.
In response to this need, the internal Research and Development team started a research project that
involved the design of an investigation tool built ad hoc to adapt to the specific requests of the client,
subsequent data collection and analysis. As will be discussed in detail in the following paragraphs, the
innovative aspect of the tool conceived and proposed with this work is represented by the integration
of structural data with a collection of short written texts. The subsequent textual analysis was realized
with SemantiCase software, a tool developed by the company’s research team that integrates semantic
analysis and sentiment analysis models.
2.1. Preliminary Demand analysis
First, it was necessary to analyse in detail the client’s request. The goal was to find a meeting point
between the interest, needs and expectations of the company and the point of view of researchers, based
on theoretical models and previous studies, basing on the rich literature on the subject. In this step of
the project the company management was actively involved, as client of the investigation, both to share
a common knowledge on organizational climate investigation and to identify the most interesting
dimensions to be included in the survey tool. First meetings brought the client’s attention to the
dimensions of the organizational climate that literature indicates as fundamental; this mediation work
also aimed at identifying a compromise between business expectations and the needs of the research
work for the construction of a valid tool. The first confrontation with the company’s managing director
revealed the need to investigate different domains of the business climate, from the three main
organizational perspectives: the company, the team, and the individual. The identification of research
dimensions has been realized through a work of active exchange with the management. In this way, it
was possible to clarify the goals of the investigation and to plan its procedure and timeframes. In the
first meeting it was then possible to identify the survey domains for the survey, which were briefly
summarized in the following table (Table 1) and grouped according to the specific collective unit to be
considered (Organization, Team, or Person).
Table 1
Selected survey domains
Organization Team Person
•Transparecy •Comunication • Autonomy and responsability
•Values system and goals •Collaboration • Job role
•Innovation •Efficacy • Training and personal
•Flexibility and adaptation •Autonomy development
•Processes and tools •Support • Work satisfaction
• Self-efficacy
•Environmental comfort
In summary, the domains related to the research areas concern the structural elements, as well as
factors related to the psychological experience of the people who are part of it. One of the latent goals
is also to explore the level of knowledge and understanding of employees about the company, the inner
processes and the value system and the objectives that represent the basis for the pursuit of the
company’s mission. From a knowledge management perspective, the climate survey also aims at
enhancing the perceptions and evaluations expressed by the staff, who thus begins to assume – and is
recognized in – an active role in the development of the organization. The chosen topics are the ones
considered as the most relevant by the client: perceived corporate image by employees, perception of
the remote working, individual contribution assessment to potential business change and company
values perception.
2.2. Preliminary context analysis
It is necessary to rely on descriptive data to have a clear picture of the current company scenario and
to set the organizational climate data collection and interpretation of. For this reason, researchers
collected information from: the company’s structural data; a semi-structured interview about the
perception of the company by the management point of view. Collected data are synthetized in the table
below (Table 2):
Table 2
Data about Piazza Copernico
Date of establishment 2007
Revenue in constant growth, with significant investments
People 45 between partners, employees, and collaborators
Areas (or teams) 9
People number per Direction and Strategy: 1
area Staff:
• Administration: 2 + 1 team leader
• Marketing: 1 + 1 team leader
• IT: 1
• Public tenders and financed projects: 2
• Sales: 4
Operations
• Learning Design and production: 9 + 1 team leader
• LMS e Services: 12 + 1 team leader
• Conformity: 3 + 1 team leader
• Research and development: 4 + 1 team leader
Work setting Hybrid (remote work/smart working to be integrated with 6 days per
month in office). Deployed since June 2020.
It was also considered essential to gather the point of view of the management, involved in the first
phase of the investigation definition. The proposed goal is to collect client’s perceptions and
expectations, to guide both the tool design and the interpretation of the results. For this purpose, a semi-
structured interview was conducted: the structural data and the perception of the domains selected in
the previous phase where integrated and organized in relation to the corporate organizational level
(Table 3).
Table 3
Semi-structured interview answers
Values system and Organizational
Communication Training Rewarding
goals goals
• An improvement • Perceived values • The growth of • The company is • The rewarding
has been and goals are company revenue responsible of mechanism
perceived in mainly focused on and the providing training chosen by the
recent years. It is the worker harmonious courses and management, has
perceived as wellbeing that is development records a high been shared
linked to the new guaranteed between the level of exclusively with
needs arising from through the acquisition of new partecipation. In the Managers
the reorganization worktime flexibility customers and the addition, a high area, to not
of working and a functional creation of new demand for influence the
methods. In this life-work balance. jobs are identified specific external working methods
regard, it is as the main goals. training is of the members to
specifically The management reported, in whom it is
reported the assesses a particular from the directed.
introduction of a medium-high level technical area and
bi-monthly of the staff's the SML area.
newsletter, to effectiveness in
share information achieving these
and updates of all goals, with a
operational areas, significant margin
and WeShare of improvement.
remote meeting,
that involve all
people in
organisation.
Finally, from a free description of the working environment, as perceived by the management, emerged
a strong perception of autonomy, a good component of mediation that provides for or resolves the rare
internal conflicts, and in general a strong collaborative and supportive bond within all working teams.
2.3. Research inquiry system
The detection of the organizational is a particularly complex activity, due to the multi-dimensional
nature of the survey process. Analysis models available in literature have many dimensions under
observation because they are based on theoretical approaches that focus attention on as many aspects
of business life or address them from different points of view. From the theories and practices given by
studies reported in literature, it is possible to recognize the questionnaire as a primary tool for an
analysis about organizational climate. It measures attitudes opinions and individual perceptions about
the organizational fundamental dimensions.
To satisfy the need to explore and investigate the areas of improvement and strengths, the survey
designer intended to integrate data collection based on usual Likert-scale items with the collection of
written texts on specific stimuli related to the areas of greatest interest to the company. The survey
design was set up to have a multi-level verification and control model of the information and to
progressively facilitate the respondent’s spontaneous response, to highlight the real meanings of work
experience. The questionnaire design has, therefore, provided a scheme in which structured questions
and open questions were alternated, and was concluded with an anonymous collaborative activity. In
fact, a second way for collecting open texts has been prepared, providing for the implementation of a
short focus group to be conducted in digital mode, using an online platform that can ensure the
anonymity of participants.
2.3.1.Survey
The tool proposed below has been designed to respond to the specific needs expressed by the client
and to overcome the limits that emerged from the tools in the literature.
In compliance with the current safety regulations for the COVID-19 pandemic in progress during
data collection, and to adapt to the working mode currently adopted by the company, the questionnaire
has been implemented using the Microsoft Form tool, allowing in this way the remote participation of
all the members of the company at the same time.
Moreover, it provides an innovative aspect linked to the integration of the classic evaluation items
with open text items. In the AR perspective, the production of short texts takes up the principle of action
and reflection: while it enhances the process of reflexivity in the individual participants, it also allows
a more detailed study of the issues investigated and also the potential collection of issues spontaneously
emerging. Three types of items have been integrated into the questionnaire (Image 1):
Closed answer item
Likert item Narrative Item
mainly used for the collection of
consist of a series of statements provided for the collection of
structural and socio-
about which the participant can textual data. The participant has
demographic data. The
express his or her degree of to produce short written text in
participant must select one or
agreement on a 5-point Likert response to a given stimulus or
more answers among those
scale theme
proposed
Image 1: Types of items in the questionnaire
The tool items are organized in 10 thematic sections (Table 4); each section presents a brief
introduction to guide the participant in understanding the specific reference area and how to answer the
questions. In total, the tool has 28 items, but the third section, composed of 4 items, is addressed
exclusively to collaborators who play the role of Area Manager or Team Leader.
Table 4
Sections topics
Items number Topic
Section 1 - include privacy consent to the study with a brief All partecipants
description of the research project. There are information
about expected objectives, timing and the anonymity of
answers in the analysis.
Section 2 9 closed answers socio-demographic data (gender, age, educational All partecipants
items qualification, working age); job role of the participant
(type of contract, working area and responsibilities); work
mode preference (from home, in the office or hybrid).
Section 3 3 closed answers structural composition of the team (number of members, Team managers
items new members of the last year and processes managed);
1 Likert item areas of communication, perceived effectiveness,
autonomy, collaboration and transparency.
Section 4 1 narrative item the participant is asked to explain the work experience of All partecipants
the last year.
Section 5 2 closed answers company value system: values and objectives; the All partecipants
items individual adherence to the value system and the
1 Likert item commitment of the company members to its
dissemination.
Section 6 3 Likert items organizational domains: flexibility and adaptation of the All partecipants
company during pandemic emergency; available tools and
spaces and the commitment to ensure a good work-life
balance; functionality of the communication tools used;
transparency and communication processes.
Section 7 1 closed answer collaboration, perceived effectiveness, autonomy, All partecipants
item communication, group and team leader support and inter-
1 Likert item team collaboration with other areas of the company.
Section 8 1 closed answer smartworking characteristics and relevant aspects. All partecipants
item
1 narrative item
Section 9 2 Likert items individual point of view: job role and work experience All partecipants
related to the last year; clarity and consistency of the job
role, processes and working strategies, autonomy,
involvement and expectations; individual work
experience; self-efficacy; perceived growth; training
opportunities; job satisfaction.
Section 10 1 narrative item what members perceive to be their actual or potential All partecipants
contribution in the changing and development processes
of the company.
All the items were the subject of preliminary analysis by the researchers and the management
involved in the first phase of the project; the aim was to evaluate the usability of the tool as well as the
understanding and adequacy of the proposed items. Although the feedback was generally positive,
several changes were made to improve the tool and make it more functional for its purpose.
2.3.2. Focus group
The climate survey was planned with a second data collection activity, also involving the production
of textual data and supporting the creation of exchange dynamics within the staff group. The choice of
the used tool (https:///textdoc.co/channels) is based on the focus group activity, defined as "a detection
technique for social research based on a discussion within a small group of people, in the presence of
one or more moderators, focused on a topic that you want to investigate in depth" [18]. The activity
therefore includes a kind of group interview, led by one or more moderators who propose incentives to
start and guide the discussion. The group’s response to these stimuli should give shape to the discussion:
the strength of this technique is the interaction that is created between the participants, whose exchange
has the potential to lead to ideas qualitatively and quantitatively greater than a single interview. The
focus group can provide oral or written answers; while in the first case, the stimuli correspond to the
oral answers of the participants, with the second method participants are asked to write their own
answers, which will then be read and discussed with the group. The advantage of the written method is
to overcome the individual and group dynamics that could influence the discussion by excluding some
participants. In this study a digitalized version of the written focus group was conducted, using as a
support tool an online platform that would ensure the partecipation in the discussion anonymously,
without requiring registration by personal or business email [19]. The aim of this activity is to collect
data in textual form, which are not polluted by desirability, and to collect the results of an active
exchange between participants. The task involves first the choice of a nickname that ensures everyone
to remain anonymous; after a first brief familiarization with the tool will then start the discussion. The
moderator of the chat, belonging to the lead team of the survey, aims to guide the discussion on the
central theme: company values, specifically proactivity and inclusion. Usually, the first focus group
conducted in a research activity aims to collect an assessment of the validity of the interview grid and,
in this specific case, also of the online platform used. It is essential to study the participants' point of
view on the topics considered to be salient, as they will not necessarily coincide with those identified
by the research team or the client [20].
3. Data analysis
3.1. Method and sample
The whole company’s staff (44 members) was involved in the organisational climate survey,
but there were only 41 participants in the questionnaire. The study was conducted on
14th February 2022 with a self-administrated questionnaire in synchronous online mode, using the
Microsoft Form tool, to ensure compliance with the safety standards required by the pandemic and to
remain consistent with the hybrid working mode carried out by the company. All the participants were
informed about the methods of data collection and the guarantee of the right to anonymity, information
that was then communicated through the first section of the questionnaire about the informed consent.
Based on the socio-personal and structural data collected under the second section of the questionnaire,
the sample is defined by the following characteristics (Table 5).
Table 5
Sample data
Gender - 39% identifies with female gender
- 61% identifies with male gender
Age - 15% below 30 years
- 27% between 31 and 40 years
- 32% between 41 and 50 years
- 17% between 51 and 60 years
- 9% above 60 years
Degree - 37% has a high school degree
- 20% has a bachelor’s degree (first level)
- 37% has a bachelor’s degree (second level)
- il 6% has a PhD – Philosophy Doctor
Seniority - only 5% of people is employed since less of 1 year
- 32% between 1 and 3 years
- 29% between 4 and 7 years
- 34% above 7 anni
Team Leader 8 participants (20%) declare to be Team Leader
Team Direction and strategy: 1 LMS and services :13
Administration: 1 Conformity: 3
R&D: 4 Learning design and production: 10
Marketing: 2 Public tenders and financed projects: 2
IT:1 Sales: 3
Once the questionnaire had been administered, the second activity planned for the survey, the focus
group in digital form, was introduced. An online platform was selected as a support tool to ensure that
everyone could participate remotely and anonymously. The activity recorded a great participation (40
participants of 44) and expected a duration of 20 minutes. Due to the large number of participants, badly
managed by the tool, technical problems arose; nevertheless, the discussion recorded 155 exchanges.
The emerged data will be analysed in the following paragraphs. It is important to note that, in
confirmation of the theories mentioned above, the start of the investigation process immediately
triggered the process of reflexivity and the awareness of all members of the organization, that at the end
of the two activities have defined themselves enthusiastic of the experience, also expressing the strong
need to create a space of periodic exchange about the company.
3.2. Preliminary analysis of structural data
This paragraph aims to illustrate the results of a preliminary analysis of the data collected through
closed items and items on a scale, within the domains on which greater attention had been required by
the client: smart working, communication, team work, and individual work experience.
• Smart working. The first area of interest, central to the client’s request, is the perception of
smart working (i.e. work from home) and its characteristics. The first relevant information comes
from the item related to the general evaluation of the smart working experience; on a scale from 1
to 10, 93% expressed votes considered medium-high (from 7 to 10), while the remaining
percentage expressed its opinion through average values (5 and 6). As for Likert-scale items, the
company’s level of adaptation and its flexibility was evaluated by 100% of participants with high
degree of agreement (4 and 5); for positive statements in relation to tools, environmental comfort
and life-work balancing were recorded percentages between 73% and 88% for high agreement
values. Fundamental, in response to the specific company need, has been the analysis of the items
relative to the preference in the workplace, to be able to evaluate the possibility of maintaining the
currently active hybrid organization. In this case, as reported in the Diagram 3.1, it is possible to
notice as a good percentage of the 78% prefers to maintain the current hybrid modality, with slight
differences on the number of days in mandatory presence. It is pointed out that the participants
who responded to the item indicating the preference "Always in the office", did not register critical
issues in the items related to tools and environment; The variables potentially related to this choice
should therefore be investigated in greater depth in future analyses.
Preference of workplace
17
15
6
3
Always at home Less than 6 days in office More than 6 days in office Always in in office
Figure 3.1: Frequencies related to the item "Where do you prefer to do your job?"
• Communication. Results about the communication area, under business and team point of view,
generally have a high degree of agreement (points 4 and 5 of the scale), with a percentage from
80% to 85%; we consider this result as freedom of expression towards the company and the team,
the level of listening by the team and the functional exchanges to achieve the achieved objectives.
The two items that need more attention, concern the ability to obtain information about the changes
in the company and the teamwork: in this regard of the positive statements a high percentage of
agreement is respectively recorded (4 and 5) only 66% and 61%.
• Teamwork. To confirm the company expressed expectations, the results show the high levels in
perceived effectiveness and the team autonomy. In addition, positive perceptions in relation to
intra-group collaboration and support are confirmed. The item that reveals a potential growth area
is inter-group collaboration; its positive statement has a lower degree of agreement (points 1 and
2 of the scale) for 20% and average (point 3) for 29%.
• Individual working experience. Under the individual point of view, there are good results about
the clarity and consistency of job role, with high agreement percentages (on points 4 and 5 of the
scale) for 80-88%. On the other hand, one of the potential areas of concern seems to be the choice
of the most appropriate strategies and work processes, with a medium-low agreement rate (from
points 1 to 3) of 34% of the sample. Turning to the psychological aspects investigated under the
individual point of view, a high job satisfaction was found, with 93% of the responses related to
high agreement levels. Similarly, the items corresponding to self-efficacy and problem-solving
report about 80% on high levels for these two dimensions. The dimension of the work pressure,
although well balanced by the personal skills self-evaluation, requires attention as 66% of the
participants indicated that they had worked in the last year at least one particularly stressful project.
Training opportunities and job growth, while presenting a distribution oriented to the highest values
of agreement, are given as to be paid attention, because about 30% of participants do not recognize
such opportunities. Partly confirming this are the results of the item relating to individual skills not
recognized by the company: 72% of the members feel they still have a lot to give to the company.
3.3. Semantic analysis of narrative questions
3.3.1. Analysis tools
The innovative aspect of the survey on organizational climate described in this work is represented
by the possibility of collecting textual data and conducting a semantic analysis on them. This allows to
enrich the traditional methodology, making it possible to use qualitative information that generally risk
to be excluded from the survey tools present in the literature. The opportunity to integrate written
answers in the questionnaire allows participants to express themselves more freely and in a wider way,
transmitting further meanings through an authentic communication. Semantic analysis was carried out
in an immediate and accessible way by the tool SemantiCase, a software developed in Piazza
Copernico’s Research and Development team. The mission of the software is to offer a user-friendly
system to the operators, although it integrates in its back-end complex semantic representation models.
SemantiCase has made it possible to analyse the information contained in the open texts, which may
contribute to the definition of the company’s climate, defining in a more accurate way the areas of
improvement and the strengths.
Specifically, in the course of this work, the analysis used:
- the semantic algorithm based on the Structural Topic Model.
- the Sentiment analysis algorithm.
Using the Structural Topic Model (STM) [21], a dataset of documents can be synthesized into a
predetermined number of topics. The topic is a distribution on a vocabulary, as well as a document is a
distribution on different topics. The main task of this model is to infer the latent variables of the topic
distribution within a document and the terms of distribution of each of them. The use of this model
allowed to identify the structure of the most representative contents on a probabilistic basis. The
subsequent interpretation of the emerged data, conducted by the research team, has made possible the
identification of the most shared themes that have been further explained through the labelling process.
Subsequently, thanks to the analysis of the Sentiment made for each single text, it was possible to obtain
the positive or negative polarity of the topics emerged from the previous analyses.
The results of the 3 narrative items in the questionnaire and the focus group activity will be reported
in more detail below, examining in particular:
- Topic proportion table: data relating to the distribution of the probability of the topics and the
number of representative records of each.
- Topic Sentiment: average sentiment value associated with the single topic (with its minimum and
maximum that describe the confidence interval).
In the data interpretation and labelling, reference was also made to:
- FREX: the list of the most significant and exclusive words of the topic.
- Wordcloud: display the most significant topic words.
Prior to the analysis, a verification of the correlations between the co-variates of the structured part was
carried out. The aim was to identify which covariates were significant to effectively segment the results.
A correlation analysis was performed with a Pearson’s Chi-squared test. The results have led to the
exclusion of related variables in the analysis of the results. Moreover, the small number of the sample
led us to choose only some co-variates for the semantic analysis of narrative questions, such as the
Seniority and the Area Manager Role.
3.3.2.Results
Narrative item 1: perceived company image. The first narrative item included in the questionnaire
was intended to collect short texts describing in general terms the work experience in the last year.
Through semantic analysis, 12 different topics were detected; the subsequent interpretation of the data,
which took place through the analysis of the FREX lists and in some cases of the individual texts
collected, led to the need to create macro-topic, semantically combining topics under a single reference
label. In the following table (Table 6) it is possible to observe in detail the characteristics emerged from
the analysis. It is important to underline that the topic with the highest proportion refers to smart
working, although the item in question did not explicitly refer to this mode of work. From the more in-
depth analysis of the macro-topic in question emerged the consideration of smart working as an
effective response of the company to the pandemic situation, which all members have adapted flexibly
in a short time. The two following macro-topics, in order of proportion, describe the work experience
itself, leaving several aspects to emerge. As described by the average Sentiment, the comments related
to the work communicate very positive opinions, confirmed by the description with terms about
satisfaction and gratification. These aspects seem to be in line with the findings of the scale items in the
questionnaire in relation to the job satisfaction of the participants.
Table 6
Topic proportion and sentiment analysis of narrative item 1*
Label (macro- Total Sentiment Sentiment
Topic Proportion
topic) Proportion (mean) (min)
11 22% 0.22 -0.15
Smart working 31%
2 9% 0.33 -0.26
6 9% 0.91 0.34
1 5% Intense but 0.71 -0.09
satisfactory work 26%
3 3% on various levels 0.08 -0.88
7 9% 0.1 -0.49
8 3% 0.93 -0.04
Dynamic, flexible,
4 12% and rewarding 18% 0.49 -0.01
work
12 3% 1.35 0.36
5 12% 12% 0.24 -0.26
Colleagues support
10 3% Team coordination 3% 0.22 -0.76
Individual
9 3% 3% 0.32 -0.65
responsibility
*Some topics weren’t significant and they are not included in the table.
Narrative item 2: smart working evaluation. Second item focus is on remote working. Specifically,
the item required participants to produce short texts referring to the relevant aspects, both positive and
negative, related to the work experience through this specific modality. As for the previous narrative
item, through the interpretation and labelling, the creation of macro-topics (composing several
semantically related topics) has been deployed. In total, the semantic analysis identified 12 topics than
synthesized in 5 macro-topics. In the table (Table 7) the salient elements are reported in detail. As
shown in the table below, the macro-topic with a greater global proportion, formed by the union of 4
topics, refers to the aspects of optimization of time and effectiveness of work experienced by
participants during the period of smart working. This topic seems to confirm the data emerged from the
questionnaire that report an average positive evaluation of the experience under examination. Special
attention should also be paid to the second macro-topic, which instead highlights the achievement of a
functional work-life balance suited to personal needs.
Table 7
Topic proportion and sentiment analysis of narrative item 2
Label Total Sentiment Sentiment
Topic Proportion
(macro-topic) Proportion (mean) (min)
11 31% 0.26 -0.03
7 13% Optimized times and 0.1 -0.35
55%
2 3% effective work 0.2 -0.75
8 8% 0.26 -0.32
3 5% 0.48 -0.24
10 10% Improved work-life 0.29 -0.21
26%
9 3% balance
4 8% 0.25 -0.32
12 3% Difficult 0.22 -0.78
8%
6 5% communication 0.39 -0.32
Intrusive
1 7% 7% 0.09 -0.5
communication
Autonomy and
5 4% 4% 0.22 -0.59
flexibility
Narrative item 3: effective or potential personal contribution to company change. Semantic
analysis carried out on the third narrative item led to the identification of 14 different topics. The item
required participants to present in a short text the contribution, potential or actual, that each felt able to
offer to actively participate in the company growth process and improvement. Also, in this case an
interpretation been carried out that has led to the identification of 8 macro-topics, whose details are
presented in the table below (Table 8). Three macro-topics with greater proportion bring attention to
the same improvement areas that emerged from the preliminary analysis on structured items, explained
in the previous paragraph. The first macro-topic highlights the high level of involvement of individuals
towards the company and their own team, and the emerging need for professional development through
training paths. The second macro-topic reflects the ideal of innovation that is strongly present in the
company; then there are all contributions that refer to the innovation of projects (in processes,
approaches, and technologies) as a driver of change and business growth. The third macro-topic
emphasizes the perceived need for an improvement in both internal and external communication.
Table 8
Topic proportion and sentiment analysis of narrative item 3
Label Total Sentiment Sentiment
Topic Proportion
(macro-topic) Proportion (mean) (min)
13 17% Personal development 0.29 -0.11
24%
9 6% in and for team -0.02 -0.7
10 9% 0.21 -0.36
Process, project,
12 6% approach, and 17% 0.57 -0.13
technology innovation
4 2% 0.98 -0.03
1 8% Improvement of internal 0.5 -0.09
and external 14%
2 6% communication 0.4 -0.3
3 11% Personal contribution 11% 0.33 -0.17
7 6% Increase market 1 0.3
knowledge and improve 12%
11 6% product targeting 0.15 -0.55
5 9% Keep actual conditions 9% 0.14 -0.43
6 6% Develop new areas and 0.31 -0.47
8%
14 2% company business 0.42 -0.64
Increase company
8 5% 5% 0.31 -0.47
branding on web
Focus group activity. The last semantic analysis was carried out in reference to the textual data
collected through the experimental activity of online focus groups. The activity provided an exchange
of messages between all participants, in a completely anonymous form, on an online chat platform. The
discussion was constantly moderated through stimulus-questions regarding the area of corporate values.
After the first phase of familiarization with the tool and the choice of a username that would guarantee
anonymity to all, the first stimulus question was asked. With this first intervention, participants were
asked to focus on three specific organizational values (proactivity, inclusion, and innovation) with the
request to indicate which of those proposed was the least clear. In this first phase the focus was mainly
on two of the three values listed: proactivity and inclusion. The second question-stimulus required
confident participants to master a good understanding of the two concepts, to clearly explain their
definitions. The focus of most of the participants in the exchange has been on the concept of proactivity;
for this reason, the two subsequent stimulus-questions have required to contextualize the value within
the company life and in the contact with customers. Although the tool used to conduct the focus group
presented technical problems that made it difficult to use, the group showed a very strong collaborative
response that led to the identification of creative solutions to overcome difficulties and proceed with
the activity. Semantic analysis conducted has extrapolated, from the entire history of the messages
exchanged during the activity, a total of 25 topics. In this case, during the interpretation of the data, it
was considered appropriate to group the topics mainly according to the topics suggested by the stimulus-
questions.
In the table (Table 9) are shown the labels of the single topics that suggest the aspects highlighted
in reference to the topics suggested by the moderator. The greater proportion was recorded on the
macro-topic related to the theme of proactivity, declined to the specific corporate context of Piazza
Copernico, both in the performance of internal activities and in contact with customers. The labels
selected for the 10 topics lead back to the aspects highlighted in the messages written on the theme.
An interesting observation is to highlight on the last two macro-topics reported in the table. The label
"Out of theme" groups together all the exchanges that have taken place to report the difficulties
encountered in using the tool used to carry out the focus group. The next macro-topic, "Acting
proactivity", includes all the messages aimed at finding a solution to solve the difficulties emerged and
reported. In this way, the group seems to have carried out a brief and effective practical demonstration
of its ability to take an active, collaborative, and proactive role in solving an unexpected problem.
Table 9
Topic proportion e labelling of focus group activity
Label
Total
Topic Proportion Label (topic) (macro-
Proportion
topic)
21 5% Engagement
5 4% Growth
6 2% Efficiency
18 4% Efficiency#2
8 1% Everyone's commitment Proactivity in
Piazza 45%
14 21% Context grounding Copernico
17 3% proactivity
19 1% Positivity
20 3% Communication
25 1% Mentor question
1 6% Capability of intervention
4 4% Proactivity
11 6% Predict Proactivity
general 23%
16 1% Predict#2 meaning
24 4% Predict#3
12 2% Autonomy and responsibility
3 10% Comparison and inclusion
9 3% Mutual understanding Inclusion 15%
10 2% Diversity variety
adherence to
2 7% awareness 7%
the values
13 3% Tool difficulties
Off-topic 6%
23 3% Tool difficulties#2
15 1% “To be proactive” fear
Acting
7 2% unity 4%
proactivity
22 1% “To be proactive” invitation
4. Acknowledgements
We would like to sincerely thank Oliviero Vittori (Director) and Marco Luzzatto (Administrator) for
allowing this initiative and all Piazza Copernico’ staff who enthusiastically joined the research and for
their rich contribution.
5. Conclusions
This project regarded the internal climate survey for Piazza Copernico, using an innovative tool
(created ad hoc) thanks to the collaboration of the internal team of Research and Development. The
main purpose of the investigation, as requested by the client, was to explore people perceptions about
organizational structural changes that the company faced due to the pandemic emergency from COVID-
19 since 2020. Collected data on this theme through the classic structured items (e.g., multiple choice
or assessments scales) were supplemented by the semantic analysis of short written texts. Using the
SemantiCase internal software it was possible to extrapolate, in a functional and fast way, qualitative
information that generally risk being excluded from the survey tools because of their processing
complexity.
On a preliminary analysis, the perception of the hybrid work (remote work combined with at least 6
days in office) seems to be generally preferred. This survey was supplemented by more detailed
observations taken from the written texts; the most frequently mentioned positive aspects of smart
working are connected to concentration, autonomy, and cost savings; otherwise, the negative aspects
refer mainly to the loss of social interaction and less defined working times.
Another aspect that was particularly significant was individuals: if the structured items revealed the
perception that not all the skills are fully recognized or exploited by the company, semantic analysis on
written texts has made it possible to identify a strong desire to put oneself at the forefront of the
company’s evolutionary process, with the intention of get involved for individual and team professional
development, through specific training paths or the integration of the unknows skills in the organization.
The proposed interpretation has allowed the company not only to recognize strengths and areas
improvement issues, but also to gather suggestions on possible future solutions that will guide the
evolution of business and the organization of working methods.
The next step for organizational climate survey is to use the collected information to establish the
useful actions to define a change process. In the organizations, besides the changes that regard processes
and working modalities, it is very important to consider upskilling needs. If the pandemic has
accelerated the development of digital skills, providing the organization with an observatory on the real
needs in connection to the real work processes means a great and immediate understanding and
identification of the areas of development.
The climate survey therefore represents a first analysis of the training needs, in reference to digital
skills, soft, organizational and development paths to be proposed in the future. For example, in Piazza
Copernico the survey contributed to different organizational decisions such as:
• Accelerated adoption of sharing tools (teams).
• Support for individual development through training courses.
• Promotion of autonomy and collaboration between different groups (activation of some
integrated projects between groups).
• Annual scheduling of internal climate analysis.
For future developments, it will also be important to validate the data collection process and tool,
including by expanding the database of collected responses, to increase the statistical analysis quality
to be cross-referenced with the analysis of meanings. Moreover, during the validation phase it is also
important to identify possible correlations between the data collected through the different types of
items, to confirm the validity of the tool and the actual role played by the textual data in defining the
dimensions of the organizational climate. In addition, after this first pilot study, it may be necessary to
support the conduct of a wider data collection, involving different companies to collect comparable and
usable data to validate the survey tool proposed in this work.
Further considerations can be made about:
• Evaluation of the impact of structured items on open items, to reduce any effects of mutual
influence.
• Managing the items integration and rethinking survey administration to promote the
maximum expression of perceptions.
Conducting periodic follow-up studies to identify new training needs.
Integration with other company variables, such as turnover rate, productivity per area,
company results, risk-stress prevention, salary levels, to provide a more detailed picture of
working in the company.
Semantic analysis models are a promising opportunity to foster the collection of perceptions in the
organizations, that is less conditioned by the response styles involved in classic evaluation items.
Continuing studies in this field offers the possibility of collecting more data with a qualitative depth
and the absence of any bias that allow to bring out the deep meaning expressed by individuals.
6. References
1. Forehand, Garlie A., and Gilmer Von Haller. "Environmental variation in studies of organizational
behavior." Psychological bulletin 62.6 (1964): 361.
2. James, Lawrence R., and Allan P. Jones. "Organizational climate: A review of theory and
research." Psychological bulletin 81.12 (1974): 1096.
3. Schneider, Benjamin, and Arnon E. Reichers. "On the etiology of climates." Personnel psychology
36.1 (1983): 19-39.
4. Schneider, Benjamin, Mark G. Ehrhart, and William H. Macey. "Organizational climate research."
The handbook of organizational culture and climate 29 (2011): 12169-012.
5. Schein, Edgar H. "Culture as an environmental context for careers." Journal of Organizational
Behavior 5.1 (1984): 71-81.
6. Ostroff, Cheri, and Leanne E. Atwater. "Does whom you work with matter? Effects of referent
group gender and age composition on managers' compensation." Journal of applied psychology
88.4 (2003): 725.
7. Toto, G. A., Limone, P. (2020). Hybrid Digital Learning Environments for College Student
Education. In 2nd Symposium of Psychology-Based Technologies, PSYCHOBIT 2020.
8. Lovink, G. W. "The anatomy of Zoom fatigue." Eurozine (2020).
9. Raelin, J. Preface., Management Learning 30(2) (1999): 115–125.
10. Lewin, Kurt. "Action research and minority problems." Journal of social issues 2.4 (1946): 34-46.
11. Argyris, Chris, Robert Putnam, and Diana McLain Smith. "Action Science San Francisco." CA,
Jossey-B (1985).
12. Torbert, William R. "Torbert, William R., Creating a Community of Inquiry: Conflict,
Collaboration, Transformation. New York: John Wiley and Sons, 1976." (1976).
13. Mwaluko, G. S., and Tom B. Ryan. "The systemic nature of action learning programmes." Systems
Research and Behavioral Science: The Official Journal of the International Federation for Systems
Research 17.4 (2000): 393-401.
14. Revans, Reginald W. "What is action learning?." Journal of management development (1982).
15. Reason, Peter, and Hilary Bradbury, eds. Handbook of action research: Participative inquiry and
practice. Sage (2001).
16. Kaneklin, Cesare. "La ricerca azione e il suo sviluppo in Italia." La ricerca azione e il suo sviluppo
in Italia (2006): 1000-1010.
17. Dickens, Linda, and Karen Watkins. "Action research: rethinking Lewin." Management learning
30.2 (1999): 127-140.
18. Corrao, S. "Il focus group, Milano, Franco Angeli." Cerca con Google (2000).
19. Toto, G. A., Limone, P. (2020). New Perspectives for Using the Model of the Use and Acceptance
of Technology in Smart Teaching. In International Workshop on Higher Education Learning
Methodologies and Technologies Online (pp. 115-125). Springer, Cham.
20. Bovina, Livia. "I focus group. Storia, applicabilità, tecnica." Bezzi C. (edited by), Valutazione
(1998): 37-45.
21. Roberts, M. E., Stewart, B. M., Tingley, D., Lucas, C., Leder‐Luis, J., Gadarian, S. K., ... & Rand,
D. G. (2014). Structural topic models for open‐ended survey responses. American journal of
political science, 58(4), 1064-1082.