According to the Central Bank of Russian Federation the number of organizations providing banking services in Russia is more than 400 [1]. One of the key features of mature and saturated market is competition between participants in either price or service quality [1]. Despite the spread of text communication instruments call center still remains one of the key channels for providing client services and promotion of products [2, 3]. The development of digital technologies based on machine learning algorithms such as speech recognition, sentiment analysis and semantic analysis opens new opportunities for financial organizations to achieve outstanding level of services offered by call centers by means of scrutinizing internal processes and inferring best practices of top performing employees. This paper addresses application of machine learning technologies in contact centers with focus on banking organizations. In the locus of the study are two business cases: • Correspondence of operator's speech to a call script • Increase of product sales. The article presents conceptual framework and the architecture for perspective software application based on convolutional neural networks (CNN) and recurrent neural networks (RNN) and designed for automation of banking call center operations analysis.
According to the Central Bank of Russian Federation the number of organizations
providing banking services in Russia is more than 400 [
This paper addresses application of machine learning in contact center operations with focus on banking organizations. In the locus of the study are two business cases: Business Case 1 (BC1): Correspondence of operator's speech to a call script.
Business Case 2 (BC2): Increase of product sales via contact center.
The structure of the rest of this paper is as follows. In section 2 the results of literature review and key theoretical foundations are presented. Section 3 explains the approach to the transformation of business problems into formal problems of machine learning domain. In Section 4 a theoretical framework for perspective software application is presented. The conceptual architecture of perspective application is designed in Section 5. Section 6 summarizes the findings and proposals and presents further directions of the research. 2
The fundamentals of machine learning are based on the seminal paper by Rosenblatt
[
Modern methods of speech recognition are based mostly on one of the following algorithms:
Hidden Markov Models (HMM) combined with Gaussian mixture models
(GMM) [
HMM are based on the concept of Markov chain representing the interconnection of
a set of variables or states and the probabilities of transition between them [
The key assumptions underlying the model are:
Prediction of future state is independent of past observations; The probability of the predicted observation depends only on the state that predetermined it.
Thus, only current state is analyzed, in order to predict the further transition.
Based on the HMM three formal problems were defined: likelihood, decoding and
learning. Tackling these three issues allows to construct a predictive model for
realtime speech recognition [
GMM refers to the class of probabilistic models assuming that within a dataset all
the observations are produced following Gaussian distributions with unknown
parameters. The formal problem is stated as definition of distribution parameters applying
expectation-maximization algorithm given a set of observations [
Various combinations of these models were proposed by researchers and proved to
be more effective than single model algorithm. For example, GMM might be
implemented to derive the observation probabilities of certain states in HMM, or a decision
tree might be designed, to maximize the likelihood value [
Unlike HMM, Bayesian approach presumes introduction of all model variables
during model design and the posterior distribution of the variables is derived using Bayes
rule [
Compared to HMM and GMM the Bayesian distribution approach provides the
following advantages:
The predicted observation depends on a set of prior states;
Marginalization of model parameters yields improved classification;
The selection of model is performed using the probability maximization of
model components posterior distribution [
The approach requires considerable computations restricting the areas of implementation.
DTW refers to the algorithms applied to time series. The prerequisite for its
application is possibility to shrink or stretch one of the time series along the time axis. The
«warping» procedure results in two time series with distributed proportionally along
each other and serves as normalization phase. On the stage the distance between time
series patterns is calculated providing a similarity measure [
Neural networks are presented in the list of the algorithms by RNN and RBM. RNN
is a neural network with input, hidden and output layers. Apart from learning during
training procedure such design facilitates learning along network utilization [
RBM are constructed of nodes of two types, visible and hidden. Nodes between
layers are interconnected by fully bipartite graph. The resulting model is stochastic and
generative [
Over the last decade implementation of RBM for modeling input data yielded
significant improvement in recognition rate and motivated academic researchers and
industry experts to study application of deep learning to the speech analysis. RNN
models extended the research field and outperformed the RBM networks reaching the
recognition error rate of 17.7 percent [
The next academic field relevant for this research is Natural Language Processing
(NLP). NLP is a cross disciplinary field involving linguistics, machine learning and
psychology [
Inspired by recent implementations and successes the researchers intensively promoted new approaches to solve NLP tasks pushing forward the whole field. The stateof-the-art techniques applied for the problems of NLP include:
MEL is log-linear conditional probability model. The formal problem underlying the technique is selection of a model with maximum entropy from a set of models satisfying the training dataset. The process of selection is organized iteratively, therefore, requiring substantial computations [36].
MBL represents a straightforward machine learning algorithm based on a simple approximation approach. Every piece of data is stored in database and predictions are performed based on the similarity of input dataset to the stored ones characterized by a distance metric [37].
Another technique for classification problems of NLP is DT. It can be depicted as a hierarchical structure with every node representing a decision and every leaf representing a predetermined output class. DT is one of the most efficient machine learning algorithms for the NLP problem of part of speech definition [38].
CNN is considered to be a powerful deep learning algorithm designed as a multilayer perceptron with each neuron interconnected with every neuron of the next layer. This algorithm proved itself in the field of image recognition and text mining [39]. However, its effectiveness comes at price of significant computational costs [40].
Combination of the mentioned research areas provides theoretical basis for the development of the model appropriate for application to the defined business cases of the improvement of banking contact center efficiency. 3
In order to transform the baseline business cases into formal research problems I define the key operations in the business process flow. The first case refers to the problem of quality control and defined in this paper as correspondence of operator’s speech to a call script. The operator’s interaction with a client is recorded and stored on a dedicated server as a phonogram. Further the phonogram is transcribed using a speech-to-text algorithm. The next step includes analysis using text mining algorithms, in order to calculate the metrics of the correspondence of the communication flow to the predefined script. The above described case is depicted as the logical diagram in Figure 1.
Speech recognition relates to the classification problem of the machine learning domain. It can be defined as the task of designing an algorithm which allocates an arbitrary object to one of the predefined groups with a certain probability [41]. Assume I = (i1, i2 … i3) denotes a set of fragments received after recording an audio stream and O = (01, 02, …, 03) is a set of phonemes or words we expect to obtain. It is necessary to build a function f which computes the most probable sequence of phonemes or words O corresponding to the given set of audio fragments I: (1)
At the next step text data analysis is performed, to extract quality metrics of operator’s interaction. For instance, one of the text data clustering methods may be applied, to calculate the correspondence of recognized text to a call script. Additionally, classification of certain text fragments may be used, to discover the presence of welcome and farewell expressions in operator’s speech [42]. The number of similar metrics depends on the specific business model of quality evaluation.
The second business case extends the previous one. The accumulated text data and metrics describing the correspondence of interaction to a call script are enriched by sales data. On the next stage the resulting dataset is processed using regression analysis, to infer what characteristics of operator’s speech or parts of call scripts influence sales rates. Figure 3 describes the defined process.
With text metrics and additional sales data it is possible to provide classical regression analysis to explore the patterns influencing sales via contact center mostly. As a baseline model it is possible to design a simple linear regression model with successful sales as depend variable and a set of speech metrics as independent variables. Nonlinear regression may be constructed in case the resulting model does not fit well the dataset [43].
Following the conceptual framework constructed in the previous section the first
operation to be performed is classification of audio fragments. State of the art
approach yielding excellent results that may be employed for this task is RNN
modeling. Graves et al. achieved the result of 17.7 per cent phoneme error rate on the
TIMIT speech dataset [
The training process for the LSTM RNN includes several methods: connectionist temporal classification (CTC), RNN transducer, decoding and regularization. The approach provides simultaneous training of the network to classify the input acoustic information and to seek for the most appropriate following phonemes, thus constituting a joint acoustic and language model. 4.2
Text mining process consists of standard operations over the initial dataset including:
Preprocessing includes basic procedures of tokenization, stemming and stop words removal. Initially a text stream, produced by the speech recognition algorithm is broken up into a set of separate elements, for instance, words, abbreviations, interjections. Further the tokenized set of textual data reduced to word stems, base or root forms from the inclined or derived forms. This process represents the stemming procedure. Lastly, using the list of stop words the dataset is cleared off them.
For the matter of calculation operator’s interaction quality two metrics are proposed: correspondence of the speech to a call script, presence of greetings, farewell and gratitude expression. Additionally, for the purpose of further analysis automatic topic of interaction is needed, therefore key words detection is also included into the text mining process.
In order to evaluate the correspondence of speech to a call script, a text matching model is proposed. Recent experiments by Pang et al. showed that it is possible to obtain with their model called MatchPyramid the level of accuracy comparing to texts up to 75.94 per cent on the MRSP dataset [44]. The research group applied image recognition algorithms designing a CNN with a matching matrix constituted of indicators representing similarity between words of compared texts. Optimal results for the model were achieved when processing fragments of 30-50 words. Considering that interaction of operator in banking contact center and client is dialog such fragments fit well the length of speech of every side.
The second metric can be estimated as well the MatchPyramid model. It is necessary to define the expressions dictionary as a baseline for comparison. 4.3
Final operation is building a regression model using enriched dataset consisting of information regarding operator, such as time of job shift, client’s gender, client’s age and others, defined as c1, c2, … ci, their metrics, denoted as m1 and m2 and sales results as sj. Putting sj as a response variable and ci with mj as explanatory variables yields the following relating model:
(2)
Assuming a simple linear relation between variables the equation can be transformed into the following model: (3)
The rule of thumb for a linear regression sample size suggests that the number of observations should be 20 times the number of independent variables. Therefore, it is needed 100 – 200 calls per day, to perform daily analysis or the analysis may be performed on a weekly or monthly basis.
In case the linear regression model provides poor results, it can be replaced with non-linear regression model. 5 5.1
It is shown in previous section that two of three key modules of the perspective speech analytics application can be designed employing RNN and CNN. The application can be built a 3-layer architecture: data access web-server ensuring security of data transfer, analysis server with deployed speech recognition and text mining models and database server storing recognized audio records, algorithms settings and application logs. All interactions with corporate information systems, such as corporate data warehouse and audio records storage is performed via data access web server. The overview of the landscape is presented in Figure 4.
On the application level the architecture of the solution is constructed of a set of components for speech recognition and text-mining operations, controllers for training and modelling, regression analysis component and representation layer. Although for the tasks of speech recognition and text analysis are used two different networks, the processes are consequent. Therefore, it is possible to utilize same hardware.
CRISP-DM standard suggests evaluation stage, to control modelling results [45]. To incorporate this practice application includes components for models update and retraining.
The architecture is displayed in Figure 5. 5.2
More than 2/3 of requests for banking services are still applied via telephone. Moreover, the telemarketing is still one of the key financial products distribution channels. In both cases the quality of communication depends on the performance of operators in contact center.
While classical instruments for quality control, such as supervisory and selective checks allow to overview only limited number of interactions, automated control allows to cover all communications with clients bringing a holistic view over the call center operation. Moreover, the necessity of selective checks turns redundant and the number of supervisors and managers may be reduced providing a drop in staff cost. Apart from the presented outcomes the data derived from the phonograms integrated with information from other data sources, such as CRM systems, social networks, external and internal scoring systems within corporate data warehouse can produce additional value by expanding customer’s profile. Enriching customer data improves predictive power of recommendation models such as next best offer and provides improved conversion rates and sales figures.
Moreover, the designed architecture can easily be integrated with other communication channels, for instance, chat platforms. Chats deployed on the websites or implemented in mobile applications present ready-for-analysis data that can be processed using text mining algorithms. 6
This paper introduces a conceptual model and architecture for a software application based on CNN and RNN networks designed for automation of call center operations analysis. BC1 and BC2 defined in the introduction are of value for the majority of banking organizations.
The next step in the research includes developing a prototype of the application and performance of tests, to evaluate the recognition rate and quality of text analysis in one of the top 10 banks in the Russian Federation.
Further research directions may as well include definition of new business cases that can be solved using machine learning technologies or enhancement of the discussed cases applying new algorithms. 36. Neumann, S., N. Ahituv, and M. Zviran: A measure for determining the strategic relevance of IS to the organization. Information & management, 1992. 22(5): pp. 281-299 37. Lin, J.-H. and J.S. Vitter: A Theory for Memory-Based Learning. Machine Learning, 1994. 17(2): pp. 143-167 DOI: 10.1023/A:1022667616941. 38. Quinlan, J.R.: Induction of decision trees. Machine Learning, 1986. 1(1): pp. 81-106 DOI: 10.1007/BF00116251. 39. Gu, J., et al.: Recent advances in convolutional neural networks. Pattern Recognition, 2018. 77: pp. 354-377 DOI: 10.1016/j.patcog.2017.10.013. 40. He, K. and J. Sun: Convolutional neural networks at constrained time cost. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. DOI: 10.1109/CVPR.2015.7299173. 41. Goodfellow, I., Y. Bengio, and A. Courville: Deep Learning. 2016: The MIT Press. 42. Hotho, A.: A Brief Survey of Text Mining. GLDV-Journal for Computational Linguistics and Language Technology, 2005. 20(1): pp. 19-62 43. Draper, N.R.: Applied Regression Analysis. Wiley Series in Probability and Statistics, 1998. 29 DOI: 10.1002/9781118625590. 44. Pang, L., et al.: Text matching as image recognition, in Proceedings of the Thirtieth AAAI
Conference on Artificial Intelligence. 2016, AAAI Press: Phoenix, Arizona. p. 2793–2799 45. Chapman, P., et al.: CRISP-DM 1. 0: Step-by-Step Data Mining Guide. The CRISP-DM Consortium, 2000