A key business challenge of process mining is to appeal to decision-makers who seek to diferentiate, with the ambition to go beyond operational optimization. One way to position process mining as a diferentiator is to integrate operational process and experience journey perspectives, with the ultimate goal to better align operations with the needs of customers and other external stakeholders. To exemplify this direction, this demonstration presents SAP's journey-to-process analytics capabilities that fuse experience with process data, allowing organizations to generate insights about how operations afect experience.
CEUR
ceur-ws.org Screencast video Value 1.0 Proprietary Journey-to-Process Analytics https://staging.signavio.com/g/statics/labs/journey2process https://url.sap/7o0crn N/A https://url.sap/vm93si
To thrive in today’s fast-moving, interconnected world, organizations must not only maintain
tight control over their processes but also continuously transform them to meet the needs of
their stakeholders. Traditional business process management (BPM), however, focuses on an
organization’s inside-out perspective, thereby missing valuable improvement opportunities that
arise from leveraging the knowledge and experiences of stakeholders [
Recognizing both the lack of process observability and the recent interest in Large Language
Models (LLM) for BPM [
The remainder of the paper is structured as follows. Section 2 presents our tool for the integrated analysis of journey and process data. Section 3 elaborates on the maturity of the tool. Section 4 outlines directions for future work and concludes the paper.
The tool provides two main functionalities for leveraging the journey perspective for BPM: the textual analysis of journey data and the mapping of journey data to process data and models. Below, we elaborate on the details of these functionalities.
The journey-to-process analytics tool starts with the analysis of journey data as the cornerstone of providing an outside-in perspective. Users first need to create a new dashboard and provide a CSV file containing the data to be analyzed. The tool then determines the sentiment of the data, classifies the data into process topics, and identifies groups of similar stakeholder experiences. Sentiment Analysis. To get insights into how stakeholders perceive the processes of an organization, the tool classifies each text of the stakeholders as either positive, negative, neutral, or unknown. Users can then filter by sentiment type and review each text individually. Alternatively, users can analyze the development of the overall sentiment of the journey data over time, as shown in Figure 1. The tool provides the overall sentiment both as a net sentiment score and broken down into the sentiment types. The net sentiment score net is defined as follows: net =
pos − neg pos + neg + neu where pos is the number of positive items, neg is the number of negative items, neu is the number of neutral items
To determine the sentiment of a stakeholder’s text, we use an LLM that receives a prompt that includes the text and the sentiment types. A text that might violate the LLM’s content restrictions is classified as unknown.
Process Topic Analysis. To provide a starting point for a targeted in-depth analysis by the user, the tool classifies the data according to six expert-defined process topics, as shown in Figure 1. To help users to identify process issues that require attention, the tool provides, for each topic, a sentiment score and a brief summary for each sentiment type. Based on this information, users can drill down into relevant topics. Table 1 provides a description of each topic and a corresponding example.
We implemented this analysis as a multi-label classification task, where each text from a stakeholder can be associated with zero or more process topics. To obtain the topics for a text, we provide an LLM with a prompt that includes the process topics, their descriptions and the stakeholder’s text. In addition, we generate a summary by using a prompt that takes in texts from the specific topic and sentiment type for which the summary is generated. Cluster Analysis. Similar to the classification of journey data into predefined topics, the tool classifies the data into ”What goes well” and what ”Needs to Improve”, as shown in Figure 2. Within these two categories, the tool provides more fine-granular topics that emerge from the specific experience data. To achieve this, we use the BERTopic library 1 for clustering the data. For each emerging cluster, our tool also provides a topic label and a summary. We generate them using an LLM with a prompt that leverages representative keywords and texts according to BERTopic from the cluster for which the label and summary are generated. As this functionality does not require process data or models, it enables even organizations with very low levels of process management maturity to get an outside-in perspective on their operations. 1https://maartengr.github.io/BERTopic
understanding of the process that team’s targets. feedback is given to.
cution of a process, with regards to the system to approve the targets of costs, efort, time. each team member.
process. Explicit subcategory of pro- quickly. cess eficiency.
Any reference to the success of the I joined in the middle of the year and outcome of a process in one instance I could not set any targets for the reof execution or the overall success of maining months. the process and its alignment to the overall goal.
Any reference to the systems, tools The target setting system is easy to use. and integrations or UI and UX serving as an underlying layer to a process.
My team lead was very nice and provided constructive feedback when we discussed my targets.
To combine the outside-in and inside-out perspectives into a holistic view, users can link a process model (BPMN file) or an event log (XES file) to their provided journey data. The tool then maps each text of a stakeholder to zero or more activities of the model or nodes of the mined directly-follows graph (DFG). The mapping dashboards provide an overview of the analyzed process with sentiment scores attached to each process step. As shown in Figure 2, users can also select a particular step to get more in-depth insides about how stakeholders perceive this step, including a breakdown of the sentiment into the diferent types. Similar to the analysis of process topics, the mapping of journey data to process data and models is implemented as a multi-label classification task. To obtain the relevant process elements for a stakeholder’s text, we provide a prompt that includes the activities or nodes and the text to an LLM.
In this section, we report on the evaluation results of the novel process topic analysis and journey-to-process mapping features. For the topic matching experiment, we used a synthetic dataset that is inspired by real-life customer and employee journey data of a variety of business scenarios, including performance review, hiring, and order-to-cash processes. The dataset consists of 100 feedback comments. For the evaluation of the activity matching feature, we used a synthetic dataset inspired by the real-life order-to-cash process mentioned above. The dataset consists of 16 process activities and 57 feedback comments. To evaluate the features’ performance, we used the average dice score. The dice score dice is defined as follows: dice = 2 ⋅ | ∩ | | | + | | where is the set of human-provided labels, is the set of predicted labels, | ∩ | is the size of the intersection between sets and , | | is the size of set , | | is the size of set
The process topic analysis feature and the journey-to-process mapping feature experiments
were performed in a one-shot setting and a zero-shot setting, respectively. For both experiments,
we used GPT-4 [
In the future, we aim to enable users to adapt the tool to their data by confirming, rejecting, or modifying the sentiment assessments of their journey data and the associations of the data with process topics and process elements. Additionally, we aim to provide a more granular sentiment analysis that determines the sentiment towards each individual element addressed in a journey data point. Finally, the tool’s mapping functionality can be improved by considering more element types, by mapping journey data collected in a process-oriented manner to event logs instead of DFGs, and by providing a two-step mapping process. In the first step, the process model or event log that is related to the journey data would be identified within a collection. In the second step, the journey data would be mapped to individual process elements.
We sincerely thank Timotheus Kampik and Peyman Badakhshan for their continuous support in this research project and for revising an earlier draft of this manuscript.