Industry 4.0 envisions a human machine interaction that goes beyond the classical paradigms. Humans supervise semi-autonomous interoperable and reconfigurable machines that participate to the production line. In this way, humans become more and more part of a socio-technical production system and develop an intricate relationship with their (smart) tools and (smart) products.
AI is playing a critical role in advancing this vision [1], however, the path is more intricate then expected. Contrary to what is presented in [2], when we do not consider IoT devices, the so-called "big data" in many contexts presents itself as a collection of islands of "small data". These islands are scattered, fragmented, and unusable alone. Consequently, this raises the need of advancing and advocating for interoperability in several mini and micro contexts for sharing their data for creating meaningful and actionable information.
This paper presents a concrete example of this situation by considering a case study of human resources. We intend to investigate the possibility of an employee to experience attrition in his working environment by using a dataset of 1470 cases. The information collected are typical HR data such as number of years in the position, distance from work, job title etc. As we will see in section 3, each single datapoint is uncorrelated with the possibility of an employee to experience attrition. Consequently, it is impossible for a human to devise a meaningful heuristic. However, our results (presented in section 4) show that it is possible to build a neural network that achieves 80+% accuracy in a validation set and 100% in training set. At the same time, these kinds of solutions require a relatively large amount of data and companies may not have 1000+ employees that cover similar roles [3]. Consequently, a similar solution may not be applicable to simpler contexts such as, for example, in SMEs. However, this problem may be overcome if we consider an entire industry vertical where the participants will decide to share some of their information to the larger group. Let us consider, for example, the case of a hotel: One single unit may not have a sufficient amount of employees for applying a machine learning approach. However, a collection of hotels in a specific region may reach a meaningful threshold number and therefore benefit of tools that leverage an AI model, like one proposed in this paper, for supporting HR in conducting their everyday activities.
The rest of this paper is organized as follows: in section (ii) we will contextualize this research presenting a few related works, in section (iii) we will introduce the dataset and the methodology used for developing the predictor, in section (iv) we will analyze the results, and in section (v) we will conclude by presenting the interoperability issues that we are currently facing in developing such solutions.
According to authors in [4], Industry 4.0 has embraced AI technology as one of the key enablers. At the same time, also predicting employee attrition using machine learning is not a novel idea and, in some cases, raises also ethical concerns due to the possibility to inject AI-based biases in the decision process [5]. This is not a show stop for research. A recent survey presented by authors in [6] summarizes the work of 100 papers that are trying to propose AI techniques for solving HR issues. Authors in [3] also report the successful integration of AI tools in HR practices in a large bank like Credit Suisse.
The dataset that we are using in this paper has been publicly released by IBM [8] and a few exploratory research have been already performed. This is the case of authors in [7] where they suggest a simple ML approach achieving a recall rate of 0.54. The same data have been analyzed also using deep learning techniques as presented by authors in [8]. In this specific case they have been able to achieve a 91% accuracy.
In this paper, we present a neural network model, enhanced by genetic algorithm for finetuning the model to the specific dataset. Our model achieves a better accuracy; however, it is not the point of this paper. In this respect we intend to outline the importance of a proper dataset for developing such tools. In addition, we consider the difficulties of implementing such tools without creating a shared dataspace [9,10] where companies engage in data sharing for developing win-win solutions.
The core idea of this research is to develop a machine learning predictor that uses a dataset containing typical information collected from HR. Clearly, the possibility to model the problem is dependent by the quality and quantity of available information. Consequently, we can investigate the learning capability of the model reducing the size of samples. In this way, we will be able to observe the degradation of performances and define a threshold where the system becomes instable and therefore useless for addressing the specific problem of predicting the possibility that an employee will experience attrition. In the next subsection (3.1) we will present the dataset and in the consequent subsection we will describe the selected ML approach.
The dataset used in this study is a collection of 35 different datapoints from a sample of 1470 employees. It does not contain missing information and all the employees have been classified in terms of experiencing attrition or not experiencing it. Figure 1 presents a normalized version of the datapoints and gives an intuitive idea of the information collected. In particular, a histogram of each variable is presented. During the normalization process we removed 87 samples as their values were deviating more than 3 times from the norm (i.e. the threshold was 3 times sigma).
At the same time, Figure 2 analyzes the correlations that each datapoint has with the others. The whiter the color (as opposed to purple, see the column at the right), the more a datapoint is correlated with the others. In addition, for our specific case, we are interested in predicting the datapoint "attrition" that is represented in the second row of the matrix using the rest of the variables contained in the dataset.
We can observe that none of the datapoints in the dataset is significantly correlated to the variable that we would like to predict. This may explain why detecting employees that are experiencing attrition is a particularly challenging task given the information that the usual HR department collects regarding their employees. In the coming section, we will present a neural network model used in conjunction with a genetic algorithm for finetuning the parameters of the model in order to best fit the problem.
Given the dataset presented in section 3. 1 [11] we intend to create a neural network model that receives a portion (70%) of the dataset as training set, a second portion (20%) for testing and the remaining part (10%) for validating the final model to avoid overfitting cases. A neural network has a collation of parameters that can influence their learning process. Example includes the number of hidden layers, the number of neurons in each layer, the learning rate, batch size, etc. We decided to use a genetic algorithm approach for finetuning these parameters and present the best model to the final validation step. Figure 3 presents the idea in a graphical manner: The features of a particular neural network are encoded in a particular chromosome and the best (most performing) member of a set of models generates additional members of the population by exchanging the value of the given features.
Following the approach described in section 3.2, we generated a population of 10 neutral networks with their defining features initialized randomly in a given range and set. The population is then evolved for several generations until we find one or more neural networks that achieve precision 1 in the testing data (20% of the dataset). Lastly, the selected model (i.e the neural network that achieve precision 1) is tested against the remaining 10% of fresh data. We repeated the test varying the number of samples in the dataset in order to understand how the performance of the model degrades in relation to the amount of available data. The samples (i.e. raw in the dataset) have been removed randomly to avoid injecting bias in the dataset.
For this specific case, 3 generations were sufficient for achieving accuracy "1" in the training set. Given the relatively small number of features and samples of the dataset, the training can occur almost in real time. Figure 4 presents the experienced ROC curves and related AUC when the selected model (that achieve accuracy 1 in the testing data) was validated against the 10% of fresh samples that were randomly removed prior to feeding the data to the model for training.
As we can observe, a dataset of 100 samples is not sufficient for training a model that is capable to generalize from the data as the AUC is significantly lower the other models that use a larger sample. In addition, as we increase the size of the available data, we obtain better performances as well as a sounder model. Finally, the accuracy moved from 0.5 for 100 samples to 0.82 for the complete dataset (1470).
Although the data are clearly specific of a particular use case, it is reasonable to assume that this method of modelling cloud predicts similar performances if similar datapoints were collected from a different domain. Thus, confirming that tools that take advantage of a similar AI based model may be developed for companies where it is possible to identify and monitor a relatively large (500+) number of employees working at similar jobs. This intrinsic limit of the technology may limit the competitiveness of SMEs unless they properly organize.
In this paper, we present an example of how data can be used for developing a more human-centric approach and improve the quality of work of employees. We use a machine learning approach for developing the possibility of predicting attraction in the workforce with reasonably sound performance. As we are advancing in the 4 th and 5 th industrial revolution, more tools that take advantage of a similar approach will be developed and adopted by companies. However, this paradigm is challenging the interoperability community as the amount of data needed for the developing of the models may not be available in only one company, in particular if we consider the case of SMEs. Our experimental results show that is reasonable to assume that we need more than 500 samples if we want to develop an AI based model. This represents a clear limitation if we want to adopt these solutions in companies that have a limited number of employees.
We could overcome this limitation by developing safe dataspaces where multiple stakeholders can share their information to achieve a minimum dataset that may enable AI-based solutions.
In conclusion, this work intends to advance the conversation towards the development of interoperable non-production efficiency AI solutions by:
• Presenting a case where we face data scarcity and the need to jointly overcome this obstacle.
• Providing a working proof of concept coming from the HR domain. Within this specific case, we define a minimum size threshold for implementing AI based models in the context of predicting the possibility of an employee experiencing attrition. • Advocating for the creation of meaningful dataspaces where people and organizations can share their data in a GDPR compliant manner. • Outlining the challenge for modern interoperability best practices as these data are from different and scattered entities.