English. Human Resources are one of the most important assets in modern organizations. Their capability of facing employees' needs is critical in order to have an effective and efficient company, where people are the center of all business processes. This work is focused on developing new techniques that, leveraging a data driven approach, can help Human Resources to find a more precise employee satisfaction categorization, to easily identify possible issues and to act in a proactive fashion.
Italiano. Le Risorse Umane sono una delle funzioni piu´ importanti nelle aziende moderne. La loro capacita` di affrontare le necessita` dei dipendenti e` fondamentale per avere un’azienda efficiente, dove le persone sono al centro di tutti i processi di business. Il presente lavoro e` focalizzato sullo sviluppo di nuove tecniche che, facendo leva su un approccio data driven, possano aiutare le Risorse Umane a dare una categorizzazione della soddisfazione dei dipendenti piu´ precisa, ad identificare piu´ facilmente possibili problemi condivisi e ad agire in maniera proattiva.
Every modern organization has a dedicated function which takes care of its employees, commonly called Human Resources (HR). HR duties are related to the capability of creating value through people, ensuring that everyone can express his own potential and has a productive and comfortable office environment.
Nowadays, HR can rely on data to create a new paradigm based on a data driven approach, where analysts can leverage data in order to get more complete, detailed and data-supported decisions.
Being able to monitor employees’ engagement
and satisfaction is critical in order to maintain a
positive and constructive office environment. The
benefit for the company is in the capability of
retaining the best employees and keeping the overall
workforce strong and motivated. Furthermore,
recent surveys
This paper is organized as follows. Section 2 presents a literature review on both themes of HR Management and text mining, Section 3 summarizes the motivations that drove the present study, Sections 4 and 5 discuss data and methodology, respectively, and Section 6 presents the results. Finally, Section 7 discusses the implications of the findings and further possible developments. 2
Despite the great interest that is arising around the application of Data Science methods and Natural Language Processing (NLP) to HR problems, very few studies exist on the topic.
The entire field of corporate HR Management has been revolutionized by the pioneering work done by People Operations at Google (well described in Bock (2015)), that first put a spotlight on the benefits of having a more scientific and rigorous approach to these areas which have been traditionally more reluctant to adopt change.
Employee satisfaction has been linked to
longrun stock returns
Text clustering, and more generally text
classification, is a well established topic in the NLP
research area
Flourished in the last decade, sentiment
analysis aims to classify the polarity of a given text –
whether the expressed opinion in a document or
a sentence is positive, negative, or neutral
Enel HR Business Partners’ (HR-BPs) job consists in monitoring employees’ well-being, acting when necessary to solve issues. In doing so, they periodically interview employees and register information about their satisfaction, motivation, work-life balance and other personal issues in textual notes.
Currently, employees are manually classified by HR-BPs in three main categories: Demotivated, Neutral and Motivated. Unfortunately, employee motivation is not a very reliable indicator of employee well-being, since it may mask an underlying dissatisfaction, or more generally the presence of issues that HR department should act on. Indeed, one can face several problems in the office everyday life but still be motivated. We therefore chose to consider the sentiment, as it shows through interviews, as a proxy of employee satisfaction.
With the present study, we aim to categorize employee satisfaction in a more detailed and automatic way, identifying common trends among employees and clustering them into groups that share similar problems. The goal is to help HR-BPs in having an overall view of their resources’ mood and make effective adjustments in critical situations. It will also help in such situations when new HR-BPs take over a group of already interviewed resources, allowing them to have a clearer understanding of the employees and their criticalities without having to read all interviews.
For all the aforementioned reasons, we performed a classification of the interviews based on their sentiment (Section 5.1) prior to send them into the text clustering algorithm (Section 5.2). In the present study, we chose to focus only on negative moods, since they include the biggest issues HR should monitor. Nevertheless, the practical usage of this system involves the whole set of sentiment classes, since HR is interested in monitoring the entire workforce well-being and in following its evolution over time.
In choosing methods, we had to tackle the challenge to balance the scientific rigor and the need of ease of interpretation and communication to all actors involved in the process. We therefore chose to use well understood and controllable techniques, like sentiment analysis and k-means clustering. 4 4.1
HR System Integration provided interviews data, a file containing 53k textual notes in more than 5 languages taken by HR-BPs during interviews. Interviews spanned approximately 1 year, from June 2015 to July 2016, and they were performed by 142 different HR-BPs.
For the present study, we focused only on Italian interviews (25k interviews) and selected a single interview for each employee (23k interviews), since in the few cases of repeated interviews texts were not relevant (e.g., “See previous interview”).
Notes shorter than 5 words (the 5th percentile of the distribution of the number of words in each note) were considered irrelevant. As a result, in the present study we considered a dataset of 22k 4.2
Data preparation includes removing punctuation,
numbers and stop words (we removed 300
common Italian stop words, including some
peculiar words that are not relevant in this context,
like “Enel”, “colloquio”, etc.), changing letters
to lower case and lemmatization
We performed sentiment classification of texts by customizing and improving a publicly available lexicon2. In total, we used 3428 Italian labeled unigrams and 10451 bigrams, categorized as positive (4736), neutral (4367) or negative (4776) based on their polarity.
The sentiment classification model proposed in this paper is based on a score 'sent that weights differently unigrams and bigrams with a factor : 'sent = (1 ) 'uni + 'bi where 0 1, 'uni is the difference between the number of positive and negative unigrams, normalized by the number of words in the text and 'bi is the difference between the number of positive and negative bigrams, normalized by the number of bigrams in the text. Final sentiment was then calculated according to the formula 8+1 if 'sent > > Sent = < 1 if 'sent < >:0
otherwise.
Model calibration (i.e. the choice of parameters and ) was performed by comparing model re1https://github.com/napolux/ paroleitaliane
2https://github.com/opener-project/ public-sentiment-lexicons sults with the ones produced by manually annotating a subset of 200 (randomly chosen) texts (training set): two judges classified texts independently and a third one solved the cases where there wasn’t agreement. Agreement between the two independent judges was measured by calculating Cohen’s Kappa ( = 0:6).
We chose = 0:7 and = 0:0004 so that accuracy, recall and precision of the sentiment model were maximized. Although we may have chosen to optimize parameters in order to maximize negative texts recognition, we chose to consider the overall accuracy on the three classes, because from a business perspective it is more valuable to monitor the entire workforce satisfaction and to follow its evolution over time. While for we tried manually different settings, weighting more bigrams than unigrams, for we used the ROC curve and the area under it, picking the one with maximal sum of true-positive and false-negative values. 5.2
For notes’ clustering, we focused only on those classified as negative from the sentiment model (Section 5.1).
Since we didn’t have a target variable to
model (unsupervised classification), we chose to
adopt the k-means clustering algorithm, using
kmeans++ technique to seed the initial cluster
centers
The clustering model was applied on the TFIDF matrix, built with bigrams appearing in at least 2 documents. In this way, we reduced our dimensionality from the initial 37k bigrams to 5k. To calculate proximity among documents, we used cosine similarity.
Additionally, Silhouette distance has been chosen to select the best number of clusters: different models were computed by varying the number of clusters between 2 and 30 and the respective Silhouette scores were compared, fixing the number of clusters at 12 (corresponding to the highest score). 6
The application of this sentiment model (Section 5.1) classified interviews in 3655 negatives, 956 neutrals and 17297 positives. As we can see in Table 1, sentiment classification is more clearly related to employee satisfaction than motivation classes provided by HR-BPs, although they someText (after preprocessing) risorsa brillante neodirigente clima positivo ansioso molto positivo (brilliant resource new executive positive mood anxious very positive) assenteista risorsa molto critico non riuscire nulla (absentee very critical resource don’t succeed in anything) non valorizzare poco riconoscimento non potere rimanere (don’t valorize inadequate recognition can’t stay) molto scontento non credere azienda reale meritocrazia interessare piano esodo (very unhappy don’t believe company real meritocracy interest retirement plan) stabile routinario non proattivo scarso impegno (stable routine not proactive scarce effort) assumere direttamente assistente seguire particolare sicurezza vedere capo (hire directly assistant follow particular safety see boss) HR-BP Motivation
Sentiment Motivated Demotivated Motivated Motivated Neutral Neutral +1 -1 -1 -1 -1 0 times are aligned.
A different subset of 200 manually labeled texts (test set), labeled with the same methodology as described in Section 5.1, was used for evaluating model performance. Accuracy and recall were both 64%, while precision was 70%. For more details about the sentiment classification performance, see confusion matrix in Table 2.
The clustering algorithm was applied only on the 2392 negative interviews and it identified 8 clusters that we were able to precisely label, while for the remaining 4 clusters labeling was unfeasible (see Table 3). Labels were applied by manually looking at the most frequent bigrams within clusters, trying to identify common significant topics.
The most frequent identified issues preventing employee satisfaction were health problems, the will to change activity, compensation and the high workload. The most frequent bigrams for clusters 0–3 were not specific enough to lead to a precise labeling, since they refer to work activity and job in general and they don’t focus on clear issues.
In Figure 1, we represented clustering results
by means of t-SNE, a popular method for
exploring high-dimensional data
The very same model could also be used on neutral and positive subjects, so that HR could check whether the quality of life at work of these employees could be somehow improved, and understand which are the essential key factors for the employees’ well-being.
From a technical point of view, one possible improvement in order to strengthen the solidity of the present approach could be to manually annotate a subset of (anonymized) texts, developing a gold standard of HR interview clusters, to be used as a test set for techniques like the one presented in this study. This gold standard may be made available company-wise, in order to encourage collaboration and to foster the creation of a data science community, to help bring a data driven way of thinking even to those areas which have been traditionally more reluctant to adopt a rigorous digital transformation.
This is a first step to improve how HR Departments operate nowadays. We strongly believe that the introduction of a data driven approach can support critical HR decisional processes and improve companies’ productivity, without having to sacrifice each individual’s quality of life.
This research was supported by Enel. We thank our colleagues from HR System Integration dep. who provided the data analyzed in this study.