In the domain of application stores and marketplaces, user reviews are crucial for supporting multiple requirements engineering tasks. Feature extraction, emotion classification, topic analysis, review type identification, and polarity analysis are key components in requirements prioritization, feedback gathering, and release planning. Empirical evaluation of these techniques is challenging due to data collection complexities and a lack of reproducible methods and available tools. Furthermore, existing studies often focus on isolated tasks, hindering a comprehensive analysis of user perceptions. This paper introduces RE-Miner 2.0, a work-in-progress tool that integrates multiple data extraction and analysis methods in a distributed environment (RE-Miner Ecosystem), enabling a multidimensional and detailed analysis of user feedback. It ofers a web-based service for task integration and comparison, supported by persistent storage and a web application that allows analytical visualization of reviews. As a result, RE-Miner 2.0 provides a platform for task integration, replication, and comparison of review mining techniques. Bringing advancements in deep review analysis for requirements engineering. A demo of the tool is showcased here: https://www.youtube.com/watch?v=a11bHSCYqqM.
User reviews are critical for requirements engineering tasks like feature elicitation [1], bug detection [2], and release planning [3]. While natural language processing advancements (e.g., transformers, generative AI) have improved review analysis through automated requirement extraction [4], feature extraction [5], and emotion classification [ 6], existing tools remain fragmented. Recent solutions like knowledge graphs [7] and open-source miners [8] address isolated tasks but lack integration with complementary techniques such as polarity detection [9], review-type classification [ 10], and feature-based topic modeling. This fragmentation leads to incomplete analyses, added to the absence of open-source tools that integrate these multidimensional tasks. Furthermore, method evaluation is hindered by data collection complexities [11], replication barriers, and resource-intensive deployments [12]. Practitioners also struggle to select domain-appropriate approaches [13], often due to unavailable implementations [14].
Building on our prior work [15], we present RE-Miner 2.0, a holistic framework addressing these gaps
with two contributions: (
Natural Language Processing for Requirements Engineering (NLP4RE) is a research field focused on software-based solutions for automating language processing and detection techniques within the Requirements Engineering (RE) domain. Despite surveys identifying over 130 tools [13], only 13% are publicly available for replication, resulting in dificulties when conducting comparative studies or trying to implement one of those systems. Key NLP4RE tasks [14] in mobile app review analysis include: • Feature Extraction, identifying features in reviews using methods like the SAFE approach [17], which employs syntactic pattern matching and semantic similarity. • Type Classification , categorizing reviews by intent (e.g., bug report, feature request) [18]. • Topic Classification , using semantic analysis to identify review aspects such as aesthetics, compatibility, cost, efectiveness , eficiency , enjoyability, learnability, security, or usability [19]. • Sentiment Analysis, which involves evaluating and extracting information from text to understand attitudes and emotions, is divided into various subtasks within the field of opinion mining. In RE-Miner 2.0, tasks such as polarity analysis, which determines the negativity or positivity of a text [14], and emotion detection (e.g., identifying emotions like anger and sadness) using models based on Ekman’s six universal emotions [20] are applied.
Despite the diversity of tools, several challenges apply across multiple review mining tasks. Furthermore, studies highlight the limited use of NLP techniques for multidimensional review analysis. D¸abrowski et al. emphasize the scarcity of tools integrating various analyses [21]. Tools like SAFE [17] and GuMa [22] (a syntactic-semantic approach combining feature extraction, polarity analysis, and topic modeling to infer features and sentiments), rely on syntactic patterns but struggle with the informality of reviews [23], a problem addressed by newer deep learning methods [5]. However, most tools remain unavailable or non-extensible [14]. Progress has also been made in other tasks, such as emotion classification, which has evolved from basic sentiment analysis to transformer-based models [24]. However, integrated solutions capable of handling multiple tasks remain scarce. While some tools attempt to address this gap, such as the Appsent tool [25], which extracts and visualizes end-user feedback through sentiment, emotion, and feature-issue analysis, holistic solutions are still limited. Additionally, less popular techniques, such as topic or type classification, face challenges due to a lack of available solutions and tool integration. The initial version of RE-Miner filled these gaps by integrating feature extraction and emotion classification in an open-source tool, laying the basis for RE-Miner 2.0, which expands it into a multidimensional solution by ofering an open-source, extensible framework with packaged web services, detailed documentation, and a sample dataset, integrating state-of-the-art models for multi-task review analysis.
RE-Miner 2.0 is designed to be user-friendly, accessible, and reusable, making it suitable for both
researchers and mobile application designers [26]. The software focuses on five core objectives: (
The RE-Miner Ecosystem (Figure 1) is a distributed, microservice-based, four-layer architecture that is both interconnected and independent. Each layer can be used either in isolation or orchestrated through RE-Miner 2.0, aligning with the tool’s flexibility objective. The techniques and models in Layer 3, as well as the datasets in Layer 4, are based in literature research and our own work. As a result, the tool reflects the state of the art and related work.
• Layer 1 - Core Infrastructure (RE-Miner 2.0): The base of the ecosystem is the RE-Miner 2.0,
the entry point for users to access the whole ecosystem via an user interface. It is detailed in
Section 3.2.
• Layer 2 - Extraction Micro-Services: This layer contains all extraction and clustering services,
organized as micro-services accessible via REST APIs. Its features include (
The core of RE-Miner 2.0 acts as an entry point for users to access the system via an user interface (two
view examples can be seen in Figure 4). It is composed of five components: (
RE-Miner 2.0 expands from feature extraction and emotion classification to multidimensional analysis the two existing use cases (Single-review analysis, Batch-review analysis and Visual analytics) introduced in the first version [ 15] and introduces an additional use case (Feature Clustering Analysis) for review analysis. Users must complete sign-up, login, and upload their apps and reviews before using these features.
• Single-Review Analysis: Extended from the initial RE-Miner version [15]. This version introduces a new enhanced Reviews Directory, allowing users to search and filter reviews by application package, features, or descriptors (e.g., topic, type, polarity, or emotion). The review processing wizard now supports the selection of multiple tasks and methods for multidimensional review analysis, including new tasks such as topic classification, type classification, and feature clustering. Results can be viewed in the updated Review Analyzer by clicking the icon (see Figure 4), where new additional outputs such as detected polarities, topics, and types can be seen. • Batch-Review Analysis and Visual Analytics: This version extends the original use case [15] by introducing improved analytical plots, including new charts such as the Descriptor Polar Area and a temporal Descriptor Histogram, improving the visualizations from the initial version.
Existing plots have also been improved for a better user experience.
• Feature Clustering Analysis: RE-Miner 2.0 introduces a new hierarchical feature clustering
view, designed to visualize results from the Hierarchical Clustering Analysis (HCA) technique
introduced in Layer 3 (see Figure 1). This view is accessible through the Tree Analyzer tab,
allowing users to explore and interact with the hierarchical clustering of features. In this view,
shown in Figure 3, users can select an application package and a feature family from the generated
clusters. An interactive visualization of the feature hierarchy then appears, enabling users to
analyze and manipulate the clustered data. Users have the following options for further analysis
and customization: (
5.1. Plan Below we summarize the main steps of the planned (and ongoing) evaluation.
1. Data collection and annotation. We built the tool based on our previous work on mobile app
repository mining for collecting multiple reviews using the mentioned datasets in Subsection 3.1.
For the remaining tasks, more precisely in emotion classification, we are currently developing a
dataset of tagged emotions through iterative annotation processes of subsets of reviews following
structured guidelines. For future work, we plan to expand our datasets by including review topics
and types, using an iterative process that measures annotation agreement and applies strong
evaluation criteria.
2. Experimentation. We plan to conduct an empirical evaluation of all tasks described in Layer 2 of
Subsection 3.1. This will involve multiple cross-validation analyses on the full dataset annotated
with features, emotions, types, topics, polarities, and clustered features. Our evaluation will
include a quantitative ground-truth evaluation to assess and compare the efectiveness of each
technique. Additionally, we will evaluate the tool’s overall software product quality with a focus
on two key aspects defined by ISO/IEC 25010 [ 27]: (
Based on the taxonomy proposed by Wohlin et al. [28], we identify key threats to the successful implementation and evaluation of RE-Miner 2.0 and the evaluation plan described in Section 5.1. We focus on internal and external validity, given their relevance to this study. Concerning internal validity, potential biases in data collection and model selection may afect our findings. The datasets used in REMiner 2.0 (Layer 4) originate from prior studies and public sources, which may introduce inconsistencies in annotation quality and domain specificity. Additionally, our evaluation relies on predefined tasks and configurations, which may not account for all real-world variations. To mitigate these risks, we plan to employ cross-validation and benchmark against multiple state-of-the-art approaches. Additionally, we facilitate the integration of new methods and algorithms for further evaluation.
Concerning external validity, generalization remains a key challenge, particularly regarding the usability, learnability, and functional suitability of RE-Miner 2.0. The tool’s performance is highly dependent on the descriptors and services (Layer 2), models and algorithms (Layer 3), and datasets (Layer 4) selected. However, these choices are based on a comprehensive analysis of app review mining literature, ensuring broad coverage of efective methods. Furthermore, RE-Miner 2.0 is designed as a lfexible framework rather than a gold-standard selection of services. Its modular architecture allows for easy customization, enabling users to adapt datasets, models, and descriptors to diferent contexts.
RE-Miner 2.0 addresses the limitations of existing review mining tools by providing a flexible and
holistic solution for comprehensive app review analysis. It also extends the features and usability
of the first version. The tool introduces three main contributions: (
With the support from the Secretariat for Universities and Research of the Ministry of Business and Knowledge of the Government of Catalonia and the European Social Fund. This paper has been funded by the Spanish Ministerio de Ciencia e Innovación under project / funding scheme PID2020-117191RB-I00 / AEI/10.13039/501100011033. ture extraction and emotion classification, in: Joint proceedings of REFSQ, 2024. doi: 10.1007/ 978-3-031-19759-9_9. [16] Q. Motger, X. Franch, J. Marco, Mapp-kg: Mobile app knowledge graph for document-based feature knowledge generation, in: Intelligent Information Systems, Springer Nature Switzerland, 2024. doi:10.1007/978-3-031-61000-4_15. [17] T. Johann, C. Stanik, A. MollaAlizadeh Bahnemiri, W. Maalej, Safe: A simple approach for feature extraction from app descriptions and reviews, in: Proc. 25th IEEE Int. Requirements Engineering Conf. (RE), 2017. doi:10.1109/RE.2017.71. [18] A. F. Araujo, M. P. S. Gôlo, R. M. Marcacini, Opinion mining for app reviews: an analysis of textual representation and predictive models, Automated Software Engineering (2022). doi:10.1007/ s10515-021-00301-1. [19] N. Alturaief, H. Aljamaan, M. Baslyman, Aware: Aspect-based sentiment analysis dataset of apps reviews for requirements elicitation, in: Proc. 36th IEEE/ACM Int. Conf. on Automated Software Engineering Workshops (ASEW), 2021. doi:10.1109/ASEW52652.2021.00049. [20] P. Ekman, Basic Emotions, 1999. doi:10.1002/0470013494.ch3. [21] J. Dąbrowski, et al., Analysing app reviews for software engineering: a systematic literature review, Empirical Software Engineering (2022). doi:10.1007/s10664-021-10065-7. [22] E. Guzman, W. Maalej, How do users like this feature? a fine grained sentiment analysis of app reviews, in: 2014 IEEE 22nd International Requirements Engineering Conference (RE), 2014. doi:10.1109/RE.2014.6912257. [23] F. A. Shah, K. Sirts, D. Pfahl, Is the safe approach too simple for app feature extraction? a replication study, in: Requirements Engineering: Foundation for Software Quality, Springer International Publishing, 2019, pp. 21–36. [24] D. Carneros-Prado, et al., Comparative study of large language models for emotion and sentiment analysis: Gpt vs. ibm watson, in: Proc. 15th Int. Conf. on Ubiquitous Computing & Ambient Intelligence (UCAmI 2023), Springer, 2023. [25] S. Malgaonkar, C. W. Lee, S. A. Licorish, B. T. R. Savarimuthu, A. Tahir, Appsent: A tool that analyzes app reviews, arXiv (2019). URL: https://doi.org/10.48550/arXiv.1907.10191. [26] Q. Motger, M. Tiessler, et al., Gessi - nlp4se github, 2024. URL: https://github.com/gessi-chatbots. [27] ISO/IEC, System and Software Quality Models (ISO/IEC 25010), Technical Report, 2011. URL: https://iso25000.com/en/iso-25010. [28] C. Wohlin, P. Runeson, M. Hst, M. C. Ohlsson, B. Regnell, A. Wessln, Experimentation in Software Engineering, Springer Publishing Company, Incorporated, 2012.