Deep Dive Survey Miner (DDSM) is a web-based tool that analysis the verbatim records of business stakeholders' answers to open-ended survey questions. These records are traditionally manually annotated or in the best cases, analyzed by tools ensuring a coarse analysis. DDSM allows business analyst to go deeper through an unsupervised fine-grained sentiment analysis. It allows them to get insights on not only the distribution of the sentiments or the topics expressed by business stakeholders, but also, the reasons of these sentiments/topics. This could be useful for quickly deducing problems and great performances of organisation services starting from several verbatim records.
• Approaches based on supervised learning (e.g. [
In this demo paper, we present the Deep Dive Survey Miner (DDSM) tool. It discovers in an unsupervised way, the reasons of business stakeholders’ sentiments for deducing negative and positive performances. The tool ensures a fine-grained sentiment analysis allowing users to go deeper in their analysis; it generates results according to three levels of granularity: (i) a coarse level concerning the sentiments expressed in the verbatim records, (ii) a second coarse level concerning the topics in relation to each sentiment, and (iii) a finer level to understand the reasons in relation to each topic. In what follows, we describe the main functionalities of the tool, discuss its maturity and conclude with future works.
DDSM is a web-based tool allows business experts to quickly analyse the verbatim records of their surveys. The general architecture of DDSM tool consists of three key components. Below, we summarize the functionality of each one.
A- Load Data: DDSM contains an upload page where users can load their verbatim records while specifying the column names corresponding to their textual contents, IDs and timestamps.
B- Data Analysis: It uncovers, from the verbatims records, topics and sub-topics of the most recurrent sentiments reasons. We show, in Figure 1, the main implemented phases: - Phase 1: This phase pre-processes each verbatim record and split it into textual segments. It applies a pretrained BERT Model to associate to each segment a positive or negative sentiment. - Phase 3: This phase combines the occurrences of topics and sub-topics with the identified sentiments of their corresponding textual segments (see the example of results at Phase 3 in Figure 1). This enables the generation of two outputs: The first is a knowledge graph resuming: () Sentiment distribution around each topic and sub-topics; () Sub-topics around there are mostly positive sentiment; they are considered as the elementary reasons of satisfactions or positive performances; () Sub-topics around there are mostly negative sentiments; they are considered as the elementary reasons of dissatisfaction or negative performances. The second output is a structured log of sentiment reasons: it associates to each occurrence of a sentiment reason the following attributes: () the verbatim record ID and the related verbatim question, () the associated topic, sub-topic and sentiment generated by the tool, () the precise textual segment where it occurred and the origin words expressing it;
C- Results Visualization: This component enables exploring the results generated by the tool. It resumes them in a sunburst graph (see the left part of Figure 2) to visualize at the first level (near the center) all topics discovered by the tool (eg: screen, computer, etc.). Topics size is proportional to their occurrences number. For each topic, the associated reasons (i.e. sub-topics) are presented in the third level of the sunburst and are categorized by sentiments (positive or negative) in the second level. The tool ofers the possibility of displaying all verbatim records associated with a specific sub-topic by clicking on it (see the right part of Figure 2). It highlights its positions of occurrences in the verbatim records to quickly locate them. This enables business experts to better interpret the business context of each discovered topic/sub-topic.
D- Results Customization: Using the tool, business experts can customize visualizations based on their business needs and their own vocabulary. In fact, it is possible to:(i) Modify topics and sub-topics labels (globally or at each occurrence) and their degree of granularity. For instance, they can change the assignation of a set of sub-topics having in-common aspects to create new topics. In the video demo, we show how it is possible to easily create "Slowness" topic from sub-topics automatically associated to other topics related to computer workstation equipments; (ii) Load topics or sub-topics from answers of close-ended questions to combine them with the tool results (e.g. visualize sentiment reasons related to each ofice application); and (iii) Export the log of sentiment reasons obtained from the data analysis component (See the example provided in the figure 3). This provides explainability concerning the words leading the tool to assign a specific topic and sub-topic. Such functionality enables business experts to generate customized visualizations and reports according to their need (e.g. Pareto Diagram).
The implemented solution is accessible in our organisation via openshift. The front-end of
DDSM is implemented using Dash and the backend-end is implemented using Python. We
show in this video an example of application of the tool on a survey containing 1349 verbatim
records of employee workstation equipements. The solution that was the object of extension and
implementation in DDSM was validated in [
In this paper, we presented DDSM wich analyses the verbatim records. We intend to leverage the studied use cases to communicate across all directions within our organisation and visually demonstrate potential gains. Currently, we are studying how the large language models (LLM) could enhance DDSM performances. In fact, with the actual publicly available LLM, we could not exceed a limited number of tokens (i.e. 4000) which leads to the analysis of only some dozens of verbatim records at a single run; a normalization step is needed for covering thousands of records. Additionally, a huge resources in terms of RAM and GPU (e.g. 140 GB for LLAMA2) is needed, which complicates their integration in our tool. Getting data out of our organisation to be processed in a data center (as the case of ChatGPT) is also unfeasible for confidentiality purposes. Instead, we see the opportunity to use these LLM to generate more learning examples to improve some key steps (e.g. regrouping similar expressions, generate sub-topic labels).