Support vector machine is a machine learning algorithm that is widely used in sentiment analysis, but they rely on a set of hyper-parameters which greatly influence their performance. Fine-tuning these hyper-parameters is a complex problem that needs experience and domain knowledge. Thus, this study considers it as a combinatorial optimization problem and a swarm-based optimization technique is used to tune the SVM hyper parameters in sentiment analysis by using the recently introduced termite alate optimization algorithm. The performance of the proposed approach is evaluated using five metrics: accuracy, precision, recall, F1-measure and computation time on five well known datasets in the field of sentiment analysis. The experimental results result in significant improvement in the SVM performance with optimized hyper-parameters by the proposed approach compared to SVM with default parameters in all datasets.
In recent decades, user-generated content on the Web and social media has experienced an extraordinary
explosion. This has led to significant new challenges for companies, which are giving more attention to
the content of communities on the Web in order to follow their various trends. An important element
of such analysis is to characterize the sentiment expressed in comments on a specific topic, which is
called sentiment analysis [
Sentiment Analysis (SA), also known as Opinion Mining, is a rapidly growing field with numerous
applications. It involves assessing whether a piece of text is positive, negative, or neutral, and can be
applied to various types of content such as reviews, articles, and social media posts. Supervised machine
learning methods are commonly utilized to detect and classify user opinions on the Internet [
The performance of SVM hinges significantly on the meticulous selection of parameters. In the
SA field, SVM is used generally by their default parameters [
SVM’s HPs optimization is a process of finding the best combination of HPs’s values that optimize the
performance of the SVM model. Several methods are introduced in the literature which are classified
into Grid Search (GS) and Random Search (RS). Grid search, as a traditional method for HPs optimization
[
RS methods ofer an alternative to GS by sampling the HPs space and evaluating sets from a specified
probability distribution, RS randomly selects HPs sets to assess their performance. However, in RS
methods the complexity scales linearly with the number of evaluations [
In [11] the authors demonstrated significant enhancements in SVM performance through parameter optimization. They employed two methods: GS and Genetic Algorithm (GA). Comparative analysis revealed that GA outperformed GS. As described in [12], the authors advocated for employing an evolutionary algorithm called SHADE to optimize the deep learning model tailored for SA of Spanish tweets. Their findings illustrated that the HPs identified by the SHADE notably improved the performance of the deep learning approach.
Here’s a brief description of Support Vector Machines and the Termite Alate algorithm:
Support vector machine is a machine learning method that employs the principle of Structural Risk Minimization to identify the optimal hyperplane for distinguishing between two classes within the input space [13]. SVM is highly eficient in addressing challenges associated with textual data due to its ability to handle high-dimensional datasets [14], and efectively deal with correlated features by creating linear boundaries between distinct categories [15]. However, a kernel function does used to resolve the linearly inseparable problem by mapping it to linearly separable problem [16]. The objective is to find a hyperplane according to Equation 1 that optimally separates the data while Figure 1 shows that there are several valid hyperplanes for classification, all capable of separating the data set into diferent classes.
f() = + = 0 (1) Where is the weight vector (normal to the plane) and b is the bias term.
Before turning to SVM hyper-parameters, let’s explore the diference between hyper-parameters and parameters.
The terms ”parameter” and ”hyper-parameter” are often used in machine learning to describe two diferent types of variable. Parameters are the internal variables of the model that are automatically adjusted during training. They are learned from the training data and represent the weights or coeficients of the model. Hyper-parameters are configurations external to the model, often defined prior to training. They are not learned directly from the data, but are adjusted manually through testing or optimization techniques. Support vector machines have several hyper-parameters.
Support vector machines have several hyper-parameters: C, Kernel, Gamma, degree, coeficient... Using an SVM with an RBF kernel often yields excellent results for classification tasks across various datasets. However, it’s crucial to fine-tune the hyper-parameters C and according to the specific dataset [ 17]. Where C (type: Scalar) is the regularization parameter in SVM , it balances between maximizing margin and minimizing the training error, while (type: Scalar) decides that how much curvature we want in a decision boundary. Hyper-parameter selection poses a non-convex optimization challenge, prompting the development of numerous algorithms to address it. These include grid search, random search, Bayesian optimization, simulated annealing, particle swarm optimization, Nelder-Mead, and various others [18].
The study in [
one. The 3 is used to select the new alate.
= + ( − ) (3) Where alatea, alateb and alatec are random alates from the best alates group. is the adaptation factor with a value between 0 and 1. A lower value of improves the exploitation process by favoring the selection of alatenew closer to the best alates.
In this study, we aim to utilize TAOA to optimize the two hyper-parameters C and gamma in SVM for sentiment analysis. The diagram in Figure 3 illustrates the sequential flow from pre-processing to the optimization phase, showcasing our methodology optimizing the SVM’s hyper-parameters in the goal of enhancing SVM model performance.
The pre-processing phase is composed of the following tasks: • Conversion to lowercase using Python’s lower() function. • Removal of HTTP links using the split() function to filter out words containing specific prefixes. • Cleaning up special characters, numbers, and normalizing spaces using regular expressions from
Python’s “re” module. • Tokenization using NLTK’s word_tokenize function to segment text into tokens. • Removal of stop words using NLTK’s stopwords module. • Stemming each token to its linguistic root using NLTK’s PorterStemmer class.
After the preprocessing phase, the TF-IDF dataset is partitioned into training dataset (80%) and testing
dataset (20%). The TAOA optimization algorithm is called to search the optimal values for the two
hyper-parameters C and gamma in several iterations. The best values obtained by TAOA are used
by SVM for the training using the training TF-IDF dataset for generating a classification model. The
performance of the model is then evaluated by using the test dataset.
4.2.1. TAOA for optimizing SVM’s hyper-parameters
For resolving the HPs optimization problem by TAOA algorithm [
TAOA’s particle position encoding and initialization Because the HPs C and gamma are continuous parameters in rages [0.1, 1000 ] and [0.001, 100] respectively, the position of each particle in TAOA is encoded as a 2-dimensional real vector, the first one for the C HP and the second for the gamma HP. The initial values of each of N particles of TAOA Algorithm 1 Pseudo Code Termite Alate 1: 2: ∈ {1 , 2 , ..., } 3: for = 1 do 4: while <= do 5: ( ) 6: = + 1 7: end while 8: 9: while <= do 10: if ̸= then +1 = 11: ( ) 12: end if 13: = + 1 14: end while 15: ︀{ 1+1 , 2+1 , ..., +1 }︀ 16: = * 17: = − 18: ← 19: 20: 21: 22: 23: 24: are randomly initialized i.e. set random values for C and gam
ma in their ranges.
Fitness function The fitness function employed in the TAOA’s algorithm aims to maximize the classification accuracy obtained by SVM on the TF-IDF test dataset. The fitness function is presented in Equation 4 . The SVM’s accuracy is a crucial metric in determining the quality of the hyperparameter configuration. By maximizing the classification accuracy, the fitness function guides the swarm towards hyperparameter configurations that lead to improved model performance.
Fitness (alate) = (4)
This section summarizes the obtained result of the comparison of TAOA against default hyperparameters and PSO.
There are several datasets in the SA field, to evaluate the proposed approach; five datasets were selected due to their common use in the literature. Semeval_2016 (SE-2016) with 3198 tweets, Semeval_2017 (SE-2017) with (1252), Stanford with 2999 tweets, Polarity Movie Data (PMD) with 1001 tweets, and Movie Review Data (MRD) with 1200 tweets.
We used the following five metrics in the experiment: Accuracy, precision, recall, F1-score and Computation time.
• Accuracy
It is a metric that measures how often a machine learning model is correct overall. The accuracy is calculated as in Equation 5.
() =
All Predictions • Precision
It is a metric that measures how often a machine learning model correctly predicts the positive class. The precision value is calculated as in Equation 6.
( ) =
True Positives + False Positives • Recall
It measures how often a machine learning model accurately identifies positive instances (true positives) among all actual positive samples present in the dataset. The recall value is calculated as in Equation 7.
() =
True Positives + False negatives • F1-score
In scenarios where both precision and recall are equally critical, the F1-score serves as a valuable metric. The F1-score, calculated as the harmonic mean of precision and recall, provides a single measure that balances both precision and recall. A high F1-score means good precision value and good recall value . The F-score value is calculated as in Equation
Precision * Recall 1 − ( 1) = 2 * Precision + Recall • Computation Time
In addition to the usual evaluation metrics: Accuracy, Precision, Recall and F1-score this section also considers the computational time of the algorithms. It is a crucial factor in our study.
Where, true positive are instances correctly identified as positive. False positives are the instances wrongly identified as positive. False negatives are the instances wrongly identified as negative.
In our analysis, we compared the performance of the TAOA algorithm with the PSO algorithm for optimizing the hyper-parameters of the SVM on five well-known datasets in sentiment analysis. For both algorithms, we utilized a configuration with a fixed number of particles set to 100 and a total of 20 iterations. Specifically, for PSO, we selected parameters C1 = 2, C2 = 2, and weight (w) = 0.3. Conversely, for the TAOA, we opted for parameters Pe = 0.25 and = 0.7.
The five tested metrics accuracy, precision, recall, F1-measure and Computation time of the SVM’s
model was evaluated using: first, the default values of the SVM’s HPs that specified by the Python
scikit-learn library package, second the HPs generated by the PSO algorithm [19], and finally the HPs
generated by the TAOA optimization algorithm[
This research aims to improve the performance of SVM models in sentiment analysis through hyperparameter optimization. We focused on tuning the two main hyper-parameters of SVM (C and gamma) using Termite Alate Optimization Algorithm (TAOA). After experiments on five sentiment analysis datasets, we observed that the TAOA can increase the accuracy of SVM models in all datasets compared to SVM with default hyper-parameters on one hand, and exhibited quicker performance than PSO i.e. it completes its optimization process in less time. For future work, an improvement of TAOA by hybridization with other metaheuristics can significantly improve the performance.
[10] L. Villalobos-Arias, C. Quesada-López, J. Guevara-Coto, A. Martínez, M. Jenkins, Evaluating hyper-parameter tuning using random search in support vector machines for software efort estimation, in: Proceedings of the 16th ACM international conference on predictive models and data analytics in software engineering, 2020, pp. 31–40. [11] I. Syarif, A. Prugel-Bennett, G. Wills, Svm parameter optimization using grid search and genetic algorithm to improve classification performance, TELKOMNIKA (Telecommunication Computing Electronics and Control) 14 (2016) 1502–1509. [12] N. Rodríguez-Barroso, A. R. Moya, J. A. Fernández, E. Romero, E. Martínez-Cámara, F. Herrera, Deep learning hyper-parameter tuning for sentiment analysis in twitter based on evolutionary algorithms, in: 2019 Federated Conference on Computer Science and Information Systems (FedCSIS), IEEE, 2019, pp. 255–264. [13] A. D. N. W. Susanto, H. Suparwito, et al., Svm-pso algorithm for tweet sentiment analysis# besoksenin, Indonesian Journal of Information Systems 6 (2023) 36–47. [14] N. K. Wardhani, S. K. Rezkiani, H. Setiawan, G. Gata, S. Tohari, W. Gata, M. Wahyudi, Sentiment analysis article news coordinator minister of maritime afairs using algorithm naive bayes and support vector machine with particle swarm optimization, J. Theor. Appl. Inf. Technol 96 (2018) 8365–8378. [15] G. Defuant, S. Martin, L. Chapel, Utiliser des" support vector machines" pour apprendre un noyau de viabilité, in: MajecSTIC 2005: Manifestation des Jeunes Chercheurs francophones dans les domaines des STIC, 2005, pp. 195–202. [16] P. S. Hutapea, W. Maharani, Sentiment analysis on twitter social media towards shopee e-commerce through support vector machine (svm) method, JINAV: Journal of Information and Visualization 4 (2023) 7–17. [17] R. Amami, D. B. Ayed, N. Ellouze, Practical selection of svm supervised parameters with diferent feature representations for vowel recognition, arXiv preprint arXiv:1507.06020 (2015). [18] J. Wainer, P. Fonseca, How to tune the rbf svm hyperparameters? an empirical evaluation of 18 search algorithms, Artificial Intelligence Review 54 (2021) 4771–4797. [19] J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of ICNN’95-international conference on neural networks, volume 4, ieee, 1995, pp. 1942–1948.