In recent years, smart cities have increasingly recognized the importance of citizen input in enhancing public services and optimizing urban infrastructure. As urban populations grow and services become more complex, understanding resident sentiments and opinions is crucial for effective governance. Sentiment analysis, a technique rooted in natural language processing (NLP), serves as a powerful tool for gauging public opinion onurban services, particularly public transportation. This paper presents a sentiment analysis framework using an advanced XLNet-Bidirectional Long Short-Term Memory (BiLSTM) model, developed with a custom dataset of citizen reviews related to the Delhi Metro, a key element of India's public transportation. The dataset was meticulously scraped from various platforms and manually labeled for accuracy. Initially, the model was trained on the IMDb dataset, achieving an impressive accuracy of 93.1%. It was then evaluated on the Delhi Metro dataset, yielding an accuracy of 1.00. However, this high accuracy may indicate overfitting due to the smalldataset size, suggesting the findings are exploratory. This study highlights how sentiment analysis can improve decision- making and enhance public transportation services. By analyzing feedback on the Delhi Metro, city planners can identify areas for improvement and address citizen concerns. In conclusion, the paper underscores the potential of advanced sentiment analysis techniques in understanding public opinion and calls for further research with larger, more diverse datasets and refined models to assess citizen sentiment in smart cities comprehensively.
Urbanization has led to the rapid development of smart cities, which depend on advanced technologies and citizen engagement to enhance public services and infrastructure. As urban populations increase, the need for efficient and responsive public transportation systems becomes more critical. Citizen feedback plays a pivotal role in improving these services by providing insights into user experiences, satisfaction levels, and areas requiring enhancement.
Sentiment analysis is a powerful tool to gauge public opinion, enabling city planners and
policymakers to make data-driven decisions that address the needs and concerns of urban residents. By
leveraging natural language processing (NLP) techniques, sentiment analysis extracts subjective
information from textual data, transforming unstructured feedback into quantifiable insights. Several
algorithms have been employed for sentiment classification, each with its respective strengths and
weaknesses. Traditional machine learning approaches, such as Support Vector Machines (SVM) and
Naive Bayes, are frequently used due to their simplicity and effectiveness. For instance, Ajmera
[
As the field advances, deep learning techniques have gained prominence, with models such as Long
Short-Term Memory (LSTM) networks demonstrating superior performance in varioussentiment
analysis applications. Abdirahman et al. [
Hybrid models that combine the strengths of multiple approaches have also emerged, pushing the boundaries
of sentiment analysis further. Garg and Sharma [
The introduction of transformer-based models, such as BERT (Bidirectional Encoder
Representations from Transformers), has revolutionized sentiment analysis. Sousa et al. [
This study proposes a novel sentiment analysis framework utilizing the XLNet-BiLSTM model, focusing on citizen reviews of the Delhi Metro. XLNet, an improvement over traditional transformer architectures, enhances contextual understanding by using a permutation-based training approach. By integrating this with a BiLSTM architecture, the proposed framework captures both contextual information and sequential dependencies in textual data.
To assess the effectiveness of this approach, we created a custom dataset comprising citizen reviewsof the Delhi Metro, which were manually scraped and labeled. Initial results from training the model on the IMDb dataset indicated a high accuracy of 93.1%, demonstrating the model’s effectiveness in sentiment classification. Additionally, the model achieved perfect accuracy (1.00) on the custom dataset, underscoring the exploratory nature of this research as a proof of concept rather than a definitive evaluation. This paper contributes to the evolving field of sentiment analysis in smart cities by presenting an innovative framework leveraging state-of-the-art techniques. By focusing on the Delhi Metro case study, we provide insights into citizen sentiment and highlight the potential of sentiment analysis for enhancing urban transportation systems. Our findings not only advance the theoretical understanding of sentiment analysis but also offer practical recommendations for improving public services through effective citizen engagement.
Sentiment analysis has become a pivotal area of research, driven by the exponential growth of socialmedia
and online platforms filled with user-generated content. Bonta et al. [
Grana [
Yogi et al. [
The impact of text preprocessing techniques on sentiment analysis was examined by Garg and
Sharma [
On the deep learning front, Srinivas et al. [
A hybrid approach combining SVM and lexicon-based methods, as explored by Muhammadi et al.
[
Further advancements in sentiment analysis methodologies were showcased by Mulyo and Widyan- toro
[
Mahadevaswamy and Swathi [
Lastly, Imran et al. [
This research introduces a novel sentiment analysis framework based on an XLNet-BiLSTM model, which integrates the advanced capabilities of the XLNet architecture with the sequential processing strengths of a Bidirectional Long Short-Term Memory (BiLSTM) network. The primary objective of this model is to enhance the understanding of sentiments expressed in complex and opinionated texts, such as citizen reviews and social media posts.
XLNet is a state-of-the-art transformer-based model that addresses limitations in traditional trans- former architectures. Unlike conventional models relying on fixed context windows, XLNet employs a permutationbased training method, allowing it to capture dependencies among all words in a se- quence more effectively. This capability is particularly beneficial for sentiment analysis as it enables the generation of contextualized word embeddings that reflect the nuanced meanings of words based on their surrounding context. By considering multiple permutations of word sequences during training, XLNet learns richer representations of language, crucial for understanding subtleties in sentiment.
Once contextualized embeddings are generated by XLNet, they are fed into a Bidirectional Long Short-Term Memory (BiLSTM) network. The BiLSTM architecture processes sequential data in both forward and backward directions, enabling it to capture information from both past and future contexts. This bidirectional processing is advantageous for sentiment analysis, where the meaning of a word canbe influenced by the words that precede and follow it. By leveraging this dual context, the BiLSTM enhances the model’s ability to discern complex sentiment nuances and relationships within the text. The integration of XLNet with BiLSTM is crucial in overcoming challenges commonly faced in sentiment analysis, such as ambiguity and contextual variability.For instance, in opinionated texts, thesame word may convey different sentiments depending on its context. The XLNet-BiLSTM model’s architecture effectively handles such complexities by using contextualized embeddings to capture dynamic word meanings, while the BiLSTM interprets these embeddings sequentially.
The proposed model is trained using a custom dataset consisting of citizen reviews, providing a richsource of opinionated content. The training process involves optimizing the model to minimize the loss function and maximize the accuracy of sentiment classification. By focusing on real-world data, the model is trained not only to recognize generic sentiment patterns but also to understand specific sentiments expressed by citizens regarding public services and transportation systems.
In summary, the XLNet-BiLSTM model presents a sophisticated approach to sentiment analysis, combining advanced contextualized embeddings with robust sequential processing capabilities. This innovative architecture aims to provide deeper insights into sentiments expressed in complex texts, facilitating more informed decision-making by city planners and policymakers in smart cities.
The training of the XLNet-BiLSTM model began with the IMDb dataset, a well-established benchmark for sentiment analysis. The dataset includes 50,000 movie reviews with an equal distribution of positive and negative sentiments, providing a comprehensive and diverse training set.
The IMDb dataset’s wide variety of reviews allows the model to encounter multiple contexts and sentiment expressions. By training on this data, the model learns to identify subtle nuances in sentiment, such as sarcasm, humor, and emotional complexity, commonly present in human-written texts.
During training, the XLNet-BiLSTM model utilized the rich contextual embeddings generated by XLNet, which effectively capture intricate relationships between words and phrases within the reviews. The training process involved optimizing the model to minimize the loss function and adjust its parameters over multiple epochs, improving its performance progressively. Techniques such as dropout regularization and gradient clipping were employed to prevent overfitting and enhance the model’s generalizability.Upon completing the training phase, the model achieved an accuracy of 93.1% on the IMDb dataset.
This high accuracy highlights the model’s ability to classify sentiment effectively across various contexts and expressions. The successful performance on the IMDb dataset demonstrates that the XLNet-BiLSTM model generalizes well, making it a strong candidate for rea-lworld sentiment analysis tasks, especiallythose involving more nuanced opinionated texts.
The insights gained from training on the IMDb dataset validate the model architecture’s effectivenessand lay the foundation for further evaluation on a custom dataset of citizen reviews. By establishing strong baseline performance in a controlled environment, the model’s potential to analyze and unde-r stand citizen sentiment in practical applications, such as public transportation feedback, is enhanced.
The custom Delhi Metro dataset consists of approximately 50 rowsof citizen reviews, sourced from aYouTube video discussing user experiences with the Delhi Metro. This dataset encapsulates a variety of opinions and sentiments reflecting individuals’ interactions with the transit system. Each entry in the dataset includes two essential columns: •
Cleaned_Comment: This column contains preprocessed user comments detailing their expe- riences with the Delhi Metro. Preprocessing was performed to standardize the text, making it suitable for sentiment analysis.
Sentiment: This column represents the manually labeled sentiment of each comment, categorized as either positive or negative. Careful manual classification was applied to ensure accuracy in capturing the sentiment conveyed by the user reviews.
Despite the limited size of the dataset, rigorous manual labeling and preprocessing have been conducted to maximize data quality for both training and evaluation purposes.
To optimize the performance of the XLNet-BiLSTM model on the Delhi Metro dataset, a detailed series of preprocessing steps was systematically applied to transform the raw text into a suitable format for analysis. These steps not only ensure the removal of irrelevant information but also help in aligning the preprocessing of the Delhi Metro dataset with the IMDb dataset, thus maintaining consistency indata handling across different datasets. By ensuring the integrity and quality of the input data, the preprocessing phase plays a pivotal role in improving the overall model performance. The following preprocessing operations were performed: • • • • •
Lowercasing: All text data was converted to lowercase to maintain consistency and eliminate discrepancies caused by case sensitivity. This step ensures uniform treatment of words regardless of capitalization. Words like "Metro" and "metro," for instance, are treated the same, helping the model focus on semantic meaning rather than variations in text presentation.
Removing URLs: URLs present in the comments were removed, as they typically do not contribute meaningful sentiment information. These hyperlinks could distract the model and introduce noise into the dataset. By removing them, the model’s focus is redirected to more sentiment-relevant features of the text.
Removing Special Characters: Along with URLs, special characters (e.g., punctuation marks, hashtags) were also removed, as they often do not carry meaningful sentiment. This step ensures that the remaining text is clean and more interpretable by the model, reducing noise.
Tokenization: The cleaned comments were then tokenized using the XLNetTokenizer, which breaks down the text into tokens. Tokenization is crucial for preparing the text for input into the XLNet model, allowing the model to process each word and sentence structure individually. Proper tokenization helps the model capture linguistic nuances and sentiment patterns more effectively. Removing Stop Words: Common stop words such as "the," "is," and "and" were removed as they do not contribute significantly to the overall sentiment. This ensures the model focuses on moresentimentrich parts of the text, improving the relevance of the processed data.
These preprocessing steps are crucial in reducing noise and standardizing the dataset, helping the model accurately capture the nuances of sentiment expressed in the input data. By applying a consistent preprocessing strategy across both the IMDb and Delhi Metro datasets, the model can better generalize its learning and perform more effectively on unseen data.
Upon evaluating the XLNet-BiLSTM model on the custom Delhi Metro dataset, the model achieved an outstanding accuracy of 1.00. This perfect accuracy suggests that the model classified all sentiments correctly. However, it is important to approach this result with caution. The small size and limited diversity of the dataset may have significantly contributed to this outcome. With only 50 reviews, the model may have learned specific patterns that do not generalize well to broader datasets or varied sentiments. Thus, while the accuracy reflects the model’s performance on this particular dataset, it may not necessarily indicate its effectiveness in real-world scenarios.
The primary limitation of this evaluation lies in the small size of the dataset, which raises concerns about overfitting. Overfitting occurs when a model performs exceedingly well on the training data but struggles to generalize to new, unseen data. In addition, the dataset shows a significant imbalance,with a much higher proportion of positive reviews compared to negative ones, which could skew the model’s predictive capabilities. To develop a more robust and generalizable model, future research should consider the following approaches: • • • •
Collecting Larger Datasets: A larger and more diverse dataset should be gathered from various sources, including user-generated reviews from social media platforms, online forums, and public discussion boards concerning the Delhi Metro and urban transportation experiences.
Enhancing Dataset Diversity: Future datasets should include reviews from different demographic groups, geographic regions, and user experiences to provide a richer dataset. This will allow the model to learn more generalized sentiment patterns, improving its predictive capabilities. Addressing Dataset Imbalance: Given that negative reviews are significantly less represented than positive ones, future work should explore techniques to address this imbalance. Strategies such as oversampling the minority class (negative reviews), undersampling the majority class (positive reviews), or employing advanced methods like Synthetic Minority Over-sampling Technique (SMOTE) can be implemented to ensure the model does not become biased toward the majority class. Implementing Cross-validation: Future evaluations should employ cross-validation techniques to assess the model’s robustness and ability to generalize across different data subsets. Cross- validation will help detect overfitting and ensure the model performs well on a variety of datasets.
By addressing these limitations and expanding the dataset, future research can enhance the effec- tiveness of sentiment analysis models applied to urban transit systems, providing better insights and enabling improvements in public transportation services.
The XLNet-BiLSTM model achieved the following performance metrics on the IMDb dataset: • Accuracy: 93.1% • Precision: 0.93 • Recall: 0.93 • F1-score: 0.93 • Accuracy: 100% • Precision: 1.00 • Recall: 1.00 • F1-score: 1.00
The XLNet-BiLSTM model achieved the following performance metrics on the Delhi Metro dataset: The following visualizations provide additional insights into the model’s performance:
The XLNet-BiLSTM model exhibited remarkable performance on the IMDb dataset, achieving an accuracy of 93.1%, which underscores its effectiveness in processing and classifying sentiment in complex textual data. This performance aligns with existing literature on sentiment analysis models,demonstrating that advanced architectures like XLNet, combined with BiLSTM, can significantly enhance sentiment classification accuracy compared to traditional methods. The XLNet’s ability to generate contextualized embeddings, coupled with BiLSTM’s capability to understand sequential data,allowed the model to capture nuanced sentiments expressed in movie reviews.
In contrast, the model’s testing on the Delhi Metro dataset resulted in an extraordinary accuracy of 1.00, indicating perfect classification of sentiments within this limited dataset. While such results are highly encouraging, they also raise concerns regarding potential overfitting. The small size of the dataset— comprising only 50 reviews—limits the diversity of the input data, which can lead the model to memorize specific examples rather than generalizing from them. This phenomenon is a common pitfall in machine learning, particularly in NLP tasks where context and variability are crucial. To achieve more robust and generalizable results, it is imperative to validate the model against larger datasets that capture a broader spectrum of sentiments and opinions. Future studies should focus on augmenting the Delhi Metro dataset with additional reviews and possibly integrating data from other sources, such as socialmedia, to enhance the model’s training process.
Sentiment analysis presents a powerful tool for understanding public sentiment and enhancing services within the framework of smart cities. By employing sentiment analysis techniques like theone demon- strated with the XLNet-BiLSTM model, city planners and decision-makers can gain invaluable insights into the feelings and opinions of the public regarding various urban services, including transportation systems like the Delhi Metro. This information can be used to assess public satisfaction and identify specific areas that require improvement, such as service efficiency, safety, and accessibility.
For instance, analyzing sentiments from user-generated comments can reveal patterns in public opinion, highlighting both positive feedback and areas of concern. If the sentiment analysis indicates a consistent negative sentiment towards certain aspects of the transit system, decision-makers can prioritize these areas for enhancement. Furthermore, sentiment analysis can facilitate real-time moni- toring of public reactions to new policies or changes inservice, allowing for quicker responses to publicconcerns.
In the context of smart cities, where the integrationof technology and data analysis plays a pivotalrole, sentiment analysis can drive data-informed decision-making. By continuously gathering and analyzing feedback from citizens, urban planners can create more responsive and adaptable transit systems that not only meet current needs but also anticipate future demands. Ultimately, the application of sentiment analysis in smart cities can lead to improved public services, enhanced citizen engagement, and a higher overall quality of urban life.
This study presents a novel sentiment analysis model that effectively combines XLNet and BiLSTM architectures to analyze citizen feedback on urban services, specifically focusing on the Delhi Metro. The model demonstrated exceptional performance on the widely recognized IMDb dataset, achieving an impressive accuracy of 93.1%. This high level of accuracy indicates the model’s capability to understand and classify sentiments in complex, opinionated texts, reinforcing the effectiveness of integrating advanced natural language processing techniques.
In addition to its success with the IMDb dataset, the model was further evaluated using a custom dataset comprising citizen reviews related to the Delhi Metro. The model performed flawlessly, at- taining a perfect accuracy of 1.00. While such results are undoubtedly encouraging, it is crucial to approach these findings with caution. The limited size of the Delhi Metro dataset—consisting of only 50 reviews—raises concerns regarding the model’s potential overfitting to this small and specific set of data. Overfitting occurs when a model learns to recognize patterns in the training data but fails to generalize thesefindings to new, unseen data. As a result, while the model’s perfect accuracy on this dataset is promising, it should not be construed as definitive proof of its robustness in real-world applications.
To address these concerns, future research should prioritize the collectionand analysis of larger andmore diverse datasets. By expanding the range of inputs, researchers can better assess the model’s generalizability and reliability across different contexts and settings.Gathering feedback from varioussources, such as social media platforms, public forums, and other transportation systems, will provide a more comprehensive understanding of public sentiment and allow for a more robust evaluation of themodel’s performance.
Moreover, exploring the implications of sentiment analysis for smart city initiatives is a promising avenue for further investigation. The insights gleaned from citizen feedback can significantly informurban planning and decision-making processes, leading to improved public services and enhanced citizen engagement. By continuously monitoring and analyzing public sentiment, city planners can make data-driven decisions that address the needs and concerns of their constituents, ultimately fostering amore responsive and adaptive urban environment.
In conclusion, this research not only demonstrates the potential of the XLNet-BiLSTM model in sentiment analysis but also underscores the importance of validating findingswith broader datasets to ensure the model’s effectiveness in real-world applications. Future studies will play a critical role in advancing our understanding of sentiment analysis within the context of smart cities, paving the way for innovative solutions that enhance urban living and promote citizen satisfaction.
The authors would like to express their sincere gratitude to Dr. Shallu Juneja for her invaluable guidance and support throughout the research process. This work was conducted as part of a minor project for the 7th semester, as outlined in the syllabus of the Department of Computer Science and Engineering at Maharaja Agrasen Institute of Technology. The authors also acknowledge the resources and facilities provided by the department, which significantly contributed to the completion of this project.