The research addresses two fundamental tasks in parliamentary discourse analysis: the first one is identifying the political ideology of speakers, and the other one is categorizing their party afiliation as either governing or opposition. Analyzing parliamentary speeches involves discerning the ideological leanings of speakers' parties based on their articulated beliefs and determining whether a party is in a governing or opposition role within legislative contexts, which is crucial for understanding their political dynamics. These tasks are formulated as binary classification problems. Pixel Phantoms at Touché presents a framework for classifying speakers' political ideology and party afiliation in parliamentary debates by using a diverse set of models including Linear SVC, Logistic Regression, and DistilBERT. We proposed three diferent models, In the first model, DistilBERT was used for extracting the embeddings, and then Logistic Regression was applied for classification. In the other two models, Linear SVC and Logistic Regression models were built and trained for classification based on the extracted TF-IDF vectorization features. The study utilizes a multilingual corpus derived from ParlaMint, encompassing parliamentary speeches across various languages and jurisdictions. The proposed system was evaluated using F1 score and found Linear SVC emerging as the top-performing model. It achieved F1 scores of 0.5921 and 0.66 in the orientation and power test data, respectively, showcasing its robustness in handling complex political discourse data.
Understanding the intricate landscape of political discourse within national parliaments necessitates a thorough analysis of two critical variables: political ideology and party alignment. This research aims to elucidate the ideological leanings of parliamentary speakers and their afiliations with either governing or opposition parties by systematically analyzing parliamentary speeches. Through this study, the multifaceted nature of political communication within legislative bodies is explored, contributing to a deeper comprehension of the forces shaping public policy and governance.
Traditionally, the task of categorizing parliamentary speeches based on political ideology and party alignment has been approached through manual methods, relying on human expertise and subjective judgment. However, this manual process is not without its drawbacks. It is time-consuming, labor-intensive, and susceptible to biases inherent in human interpretation. Moreover, the sheer volume of parliamentary data poses significant challenges in maintaining consistency and accuracy across analyses. These limitations underscore the need for innovative automated solutions capable of handling the complexity and scale of political discourse data.
In recent years, advancements in machine learning and deep learning have revolutionized the ifeld of natural language processing, ofering promising avenues for automated analysis of textual data. Techniques such as neural networks, recurrent neural networks (RNNs), and transformers have demonstrated remarkable capabilities in tasks ranging from text classification to sentiment analysis. Leveraging these techniques, researchers have begun exploring automated approaches to political discourse analysis, aiming to overcome the limitations of traditional manual methods.
In this context, the research work represents a significant innovation in the field of political discourse analysis. By harnessing state-of-the-art deep learning models and leveraging advanced computational techniques, the team aims to develop a robust framework for classifying parliamentary speeches based on political ideology and party alignment. The approach seeks to address the shortcomings of traditional methods by providing a scalable, eficient, and objective means of analyzing political discourse. Through the innovative solution, the team endeavors to contribute to the advancement of computational methods in political science and facilitate deeper insights into the dynamics of parliamentary debates.
The dataset [
In [
The research work [
In [11], Hardik Jadia conducts a comparative analysis of sentiment analysis techniques, including Support Vector Machines (SVM), Logistic Regression, and TF-IDF feature extraction. This research provides insights into the efectiveness of these methods for text classification tasks. The findings ofer guidance for optimizing models to accurately identify political ideologies and party afiliations in parliamentary speeches, contributing to more reliable computational political analysis. In [12], the comparison of supervised classification models on textual data highlights the efectiveness of diferent algorithms for text classification tasks. This research work underscores the importance of selecting and optimizing classification models for accurately identifying political ideologies and party afiliations in parliamentary speeches, thereby improving the overall performance and reliability of the analysis.
The literature review emphasizes the application of advanced NLP techniques and computational methods to enhance the accuracy of identifying political ideologies and party afiliations in parliamentary discourse. Insights from referenced studies inform the integration of transformer models like DistilBERT, logistic regression, and LinearSVC, ofering a structured approach for computational political analysis.
This research work is structured around two primary objectives within parliamentary discourse analysis: distinguishing speakers’ political ideologies and classifying their party afiliations into governing or opposition categories. To achieve these goals, separate binary classification models were employed for both orientation (political ideology) and power classification. Models such as Linear SVC and Logistic Regression were utilized for both tasks, alongside a DistilBERT-based model for generating text embeddings, followed by Logistic Regression for prediction.
In this task, all training was conducted on the English translations of the parliamentary speeches. The dataset includes both the original texts and their English translations (where available), and we focused on the English texts to maintain consistency and leverage the capabilities of pre-trained language models like DistilBERT, which are primarily trained on English data. By doing so, it is ensured that the feature extraction using TF-IDF and sentence embeddings was uniform across the dataset, allowing for efective training and evaluation of our models.
Separate models were developed for both distinguishing political orientation and categorizing power dynamics within parliamentary discourse using approaches like Linear SVC, Logistic Regression, and DistilBERT combined with Logistic Regression.
The dataset for this task is derived from ParlaMint, a multilingual comparable corpus of parliamentary debates. The data is provided in tab-separated text files, containing the following fields: id, speaker, sex, text, text_en, and label. Each entry represents a speech, with information on the speaker and automatic translations where available. The dataset covers parliamentary speeches from various national and regional parliaments, and is designed to minimize confounding variables like speaker identity.
During the data preprocessing phase, missing values were removed and the columns with null values were replaced with empty strings. TF-IDF features were extracted from the English text using Tfidf Vectorizer with specified parameters, and were used in Linear SVC and Logistic Regression. The text data (X_train and X_val) was transformed into sentence embeddings using the pre-trained DistilBERT model from the Sentence Transformers library (SentenceTransformer(’distilbert-base-nli-mean-tokens’)). This step converted the text data into numerical vectors suitable for machine learning algorithms, which were used in DistilBERT.
DistilBERT excels in capturing semantic meaning and relationships within sentences, making it highly versatile for various natural language processing tasks like sentiment analysis, question answering, document classification, and named entity recognition. Its embedding process converts text into numerical representations that encode deep contextual information, enabling precise model predictions across nuanced linguistic contexts. This capability to extract meaningful embeddings from input text is essential for a wide array of NLP applications.
After preprocessing, the model initiates by utilizing the "distilbert-base-nli-mean-tokens" pre-trained transformer from the Sentence Transformers library to convert parliamentary speeches into dense semantic embeddings. These embeddings serve as high-dimensional features fed directly into a Logistic Regression classifier.
Following embedding generation, the Logistic Regression classifier is directly trained on these embeddings. This training phase involves optimizing the Logistic Regression’s parameters, including regularization strength (’C’) and penalties (’l1’ and ’l2’), using GridSearchCV. This hyperparameter tuning ensures that the model maximizes its predictive accuracy based on the embeddings’ semantic content.This methodology guarantees that the model efectively leverages the semantic information distilled by DistilBERT to make precise predictions regarding the political ideologies and party afiliations of speakers based solely on their parliamentary speeches.
The adoption of an alternative model from considerations of computational complexity and resource requirements, particularly given the size of the dataset and the robust nature of the classification task. Moreover, the interpretability ofered by Logistic Regression’s feature coeficients seemed more congruent.
Logistic Regression is fundamental in supervised learning for its interpretability and eficacy across various domains, prominently in natural language processing (NLP). In tasks like sentiment analysis and political discourse classification, Logistic Regression harnesses TF-IDF vectorization to convert text into numerical features. This process optimizes parameters such as the maximum feature count (10,000), inclusion of both unigrams and bigrams (-grams of length 1 and 2), and thresholds for document frequencies (min_df =2 and max_df =0.9). Additionally, common English stop words are removed during the vectorization process.
GridSearchCV refines these parameters to maximize predictive accuracy, ensuring the model performs optimally. Post-vectorization, Logistic Regression incorporates these optimized features, adjusting class weights to manage dataset imbalances and ensure balanced learning outcomes. Utilizing a fixed random state guarantees consistent results across diferent runs. Leveraging the Scikit-learn library, these methodologies provide robust and scalable solutions for diverse NLP applications. This integrated approach underscores Logistic Regression’s versatility and pivotal role in converting raw textual data into actionable insights through parameter optimization and efective model training.
LinearSVC is renowned for its eficiency and efectiveness in text classification tasks, particularly suited for high-dimensional data and widely applied in areas such as spam detection and sentiment analysis. Like Logistic Regression, LinearSVC starts with TF-IDF vectorization to convert textual data into numerical features. It employs similar TF-IDF settings, including the limitation of features, exclusion of rare terms, and removal of common terms and stop words to enhance the quality of the feature set.
Despite using similar TF-IDF settings, there is a distinction between Logistic Regression, which predicts class probabilities using a logistic function, and LinearSVC, which separates classes with a hyperplane based on maximum margin criteria, emphasizing clear class distinction rather than probability estimation.
During model training, LinearSVC adjusts class weights to manage imbalanced data distributions efectively. Hyperparameter optimization via GridSearchCV focuses on parameters like the regularization parameter (tuned across values of [0.1, 1]) and maximum iterations (explored with [1000, 1700, 2500]), aiming to enhance model performance without overstating capabilities. Evaluation metrics, including the F1 score, provide rigorous assessment of the model’s classification accuracy, ensuring reliable performance validation in practical applications.
We have experimented with three diferent models and performance of each model was analysed using diferent measures such as F1 score, Recall and Precision. The results were tabulated in the Table 1 and Table 2.
Comparing the performance of three distinct models—Linear SVC, Logistic Regression, and DistilBERT approach—reveals their respective strengths and nuances in handling the tasks of identifying political ideology and categorizing party afiliation in parliamentary debates. These models were evaluated for both tasks, assessing their capabilities and limitations in computational political analysis.
The Linear SVC model achieves the highest training and testing F1 scores of approximately 0.74 and 0.66, respectively, for categorizing party afiliation (Power) and 0.77 and 0.5921 for identifying political ideology (Orientation). This showcases its robust capability in accurately classifying whether speakers’ parties are governing or in opposition and their political beliefs. Known for its efectiveness in handling linearly separable data, Linear SVC constructs hyperplanes to delineate data points, ensuring high precision and recall in this context.
The Logistic Regression model achieves the training and testing F1 scores of approximately 0.73 and 0.657, respectively, for categorizing party afiliation (Power) and 0.77 and 0.5926 for identifying political ideology (Orientation). Enhanced by Tfidf Vectorizer for feature extraction and optimized hyperparameters, Logistic Regression efectively manages collinear features and provides precise class probability estimates. This probabilistic framework makes it adept at scenarios where the relationship between features and target variables is probabilistic rather than strictly linear.
The DistilBERT model, achieves the training and testing F1 scores of approximately 0.74 and 0.453, respectively, for categorizing party afiliation (Power) and 0.74 and 0.563 for identifying political ideology (Orientation). DistilBERT’s strength lies in its ability to encode complex, non-linear relationships within the feature space, leveraging its capability as a powerful feature extraction tool. DistilBERT is employed primarily for feature extraction, harnessing its capacity to uncover intricate data patterns relevant to political discourse analysis.
Overall, while Linear SVC demonstrates superior performance in this study for the task of categorizing party afiliation and Logistic Regression for identifying political ideology, each model presents tradeofs between complexity, interpretability, and performance. These insights underscore the nuanced considerations necessary when applying machine learning to computational political analysis.
In conclusion, the Linear SVC model exhibits the highest validation F1 score, showcasing its robustness in handling linearly separable data. Logistic Regression closely follows, demonstrating competitive performance by eficiently managing collinear features and providing precise class probability estimates. Meanwhile, the DistilBERT model ofers flexibility in capturing complex relationships but may be prone to overfitting. [10] V. Dogra, A. Singh, S. Verma, Kavita, N. Jhanjhi, M. Talib, Analyzing distilbert for sentiment classification of banking financial news, in: Intelligent Computing and Innovation on Data Science: Proceedings of ICTIDS 2021, Springer, 2021, pp. 501–510. [11] H. Jadia, Comparative analysis of sentiment analysis techniques: Svm, logistic regression, and tf-idf feature extraction (2023). [12] B.-M. Hsu, Comparison of supervised classification models on textual data, Mathematics 8 (2020) 851.