Empirical performance analysis of classification methods on cultural heritage database Israr Ur Rehman1, Zulfiqar Ali2, Zahoor Jan1, Muhammad Rashid2 and Waqar Ali3 1 Department of Computer Science, Islamia College University, Peshawar, Pakistan. 2 Department of Computer Science, National University of Technology, IJP Road, Islamabad, Pakistan. 3 Center for Future Media, University of Electronics Sciences and Technology of China, China. Abstract Classification is the process of labeling data items as belonging to a given class from a model which is built from a selected set of data. The classification methods promis-ingly and effectively exploit the classification of cultural heritage unstructured data objects. There is a variety of classification methods available in contemporary litera-ture. The choice of a suitable and effective classification method for the classification of cultural heritage data is challenging and nontrivial. In this paper, we are providing an empirical study of classification methods to investigate the performance of a cul-tural heritage database. The classification algorithms for under focus empirical study are namely i.e. Bayesian (BayesNet, NaiveBayes, and NaiveBayesUpdateable), Function (Logistic, Multilayer Perceptron, Simple Logistic, and SMO), Lazy (IBK, KStar, and LWL), Meta (Bagging, Regression, LogitBoost, MultiClassClassifier, and Mul-tiClassClassifierUpdateable) and Rule-based (DecisionTable, JRip, OneR, and PART) classifiers provided in the Weka implementation by considering the av-erage values of each classification method category. This paper contributes in three aspects; 1) to provide a comparison of state-of-the-art classification methods on cultural heritage database; 2) secondly provides evaluation performance within the classification methods category as well as intra categories; 3) classification methods evaluation on the state-of-the-art evaluation metrics. Under focus study use speaker accent recognition data set available publically in UCI Ma-chine Learning repository for all the selected classification methods. The implemen-tation of the selective classifiers in Weka is exploited for the empirical study. The performance of classifiers is evaluated in terms of TP Rate, FP Rate, Precision, Re-call, F-Measure, MCC, ROC Area, PRC Area, and Kappa statistic. Keywords 1 Classification, Cultural Heritage, Classifiers, Empirical Study, Instance-based Learning. Introduction 1. Introduction Cultural heritage resources most valuable, non-renewable, scarce, and finite of any specific civilization era in the world. The cultural heritage represents our collective memory, shapes our identity, and also drives the economy [1, 2]. A reliable medium for history and knowledge transferring of a specific civilization is cultural heritage resources in the world. There are vast and diverse types of cultural heritage like paintings, sculptures, coins, manuscripts, monuments, archaeological sites, historical buildings, etc [3, 4]. These cultural heritage resources' accurate and efficient classification is valuable and useful for tourism and future generations. The contemporary literature reveals that there are several classification methods for processing variable performance. The choice of an efficient, accurate, and robust classification method is a challenging and non-trivial task. VIPERC2022: 1st International Virtual Conference on Visual Pattern Extraction and Recognition for Cultural Heritage Understanding, 12 September 2022 EMAIL: israr.rehman@gmail.com(A. 1); zulfiqarali@nutech.edu.pk (A. 2); 1zahoor.jan@icu.edu.pk (A. 3); muhammad.rashid@nutech.edu.pk (A. 4) ; waqar.uestc@yahoo.com (A. 5) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Wor Pr ks hop oceedi ngs ht I tp: // ceur - SSN1613- ws .or 0073 g CEUR Workshop Proceedings (CEUR-WS.org) The contemporary literature shows that there are several comparative studies done to evaluate the performance of classification methods in different domains like image processing, computer vision, disease detection systems, predictive systems, etc. Jia Wu et.al provided an empirical comparative study on classification methods on public data sets in [5]. S.Taruna and Mrinal Pandey a performance analysis of Classification Techniques to predict the performance of students in the academic field in[6]. Arvinder Kaur and Inderpreet Kaur evaluated the performance of classifiers to predict the faults in open source projects in [7]. Zulfiqar Ali and Waseem Shahzad provided the comparative analysis ACO based Rule Miners in [8]. Zulfiqar Ali et.al presents an empirical study to evaluate the performance of associative classifiers on imbalanced Datasets in KEEL in [9]. Radmila Jankovic et.al investigated the performance of ensemble classifiers on energy consumption in the Balkans in [10]. Zulfiqar Ali et.al provided the empirical performance analysis of machine learning methods to estimate the software development of effort in [11]. Yuanshun Yao et.al provided the analysis of machine learning-based classification models in the context of the complexity of the algorithms and performance [12]. Shelja Sharma et.al evaluated the performance of Bayesian classifiers on public data sets and used evaluation metrics accuracy, kappa statistics, and absolute mean error [13]. Umar Ishfaq et.al presents a comparative analysis of machine learning and deep learning-based classifiers for multiclass prediction in [14]. Chris Zhu et.al exploited augmented reality in the domain of cultural heritage assets in [15]. The contemporary literature reveals that several studies have been proposed for empirical comparison of classification methods in various domains of knowledge like the medical field, image classification, medical image classification, etc. However, these studies do not fully analyze the classifier performance on a cultural heritage database. It demands a more comprehensive empirical study for the classification methods to reveal the comparative performance analysis on cultural heritage databases. This study provides the empirical performance analysis of diverse and widely used classification approaches in the domain of cultural heritage in terms of state-of-the-art metrics. Therefore, the objectives of this paper include: 1. To compare classification models in the context of a cultural heritage database 2. Evaluate the performance of various types of classification methods within the specific classifiers’ category as well as in the other classification categories. 3. The performance of classifiers is evaluated in terms of TP Rate, FP Rate, Precision, Recall, F-Measure, MCC, ROC Area, PRC Area, and Kappa statistic. The paper is organized as follows: Section 2 describes Related Work. Section 3 provides the first- hand introduction of selected classification models. Section 4 describes the data and methodology exploited for empirical study in this paper. Section 5 consists of the analysis performed on the cultural heritage data set. Finally, Section 6 concludes the under-focus comparative study. . 2. Related Work The contemporary literature reveals that several studies have been proposed for empirical comparison of classification methods in various domains of knowledge like the medical field, image classification, medical image classification, etc. However, these studies do not fully analyze the classifier performance on a cultural heritage database. Marijana ´ Cosovi´ et.al [3] provided a review of classification methods exploited to classify the cultural heritage data objects. This article surveys the contemporary literature for the classification of cultural heritage data and provided the taxonomy for cultural heritage information. Jose Llamas et.al [16] used a convolutional neural network for the classification of images obtained from the architectural heritage. Various deep learning-based classification approaches are exploited and evaluated. Uday Kulkarni et.al proposed a crowdsourcing platform and designed a transfer learning-based classification model in [17]. This crowdsourcing platform collects Indian digital heritage space monuments data, performs classification tasks on images, and labels images retrieved based on queries. Radmila Jankovi'c provided the performance analysis of decision tree-based classification methods by using the Weka data mining tool in [18]. In this, J48, Hoeffding Tree, Random Tree, and Random Forest algorithms are applied for the heritage image classification task. Marijana Ćosović and Radmila Janković used a Convolutional Neural Network for the classification of cultural heritages images in [19]. The CNN approach evaluated architectural heritage images i.e. bell tower, stained glass, vault, column, outer dome, altar, apse, inner dome, flying buttress, and a gargoyle in this study. Feng Zhipeng and Hamdan Gani [20] provided a new Indonesia cultural events dataset and applied a deep learning-based Hyperparameter Optimization approach for the classification of images related to cultural events. The Keras library is used for the development of the CNN model and experiments are performed in Python v.3.7. Sathit Prasomphan attempted to develop a cultural heritage information management system by using the deep learning approach [21]. Waqar Ali et.al. applied contextual recommendation systems in [22, 23]. A federated learning approach for privacy protection in heritage data recommendation is proposed in [24]. The story of the archaeological site is developed by using machine learning and image processing through features of the archaeological site and linked to the era of the ancient monument's architecture. Heri Kurniawan et.al proposed a framework, called eCultural Heritage and Natural History (eCHNH) in [4]. The proposed framework (eCHNH) is developed based on Zachman Architecture Framework. 3. Classification Algorithms A classifier is a machine learning model that is used to discriminate different objects based on certain features. The following subsections provide the basic description of the classification methods used in the focus empirical performance analysis on the cultural heritage database. 3.1. Bayesian-based Classifiers We have selected BayesNet, NaiveBayes, and NaiveBayesUpdateable classification learning algorithms for the comparative and empirical study in the field of cultural heritage. The Bayesian family of classifiers works based on the Bayes theorem. Table 1 shows the comparative performance analysis of Bayesian-based classifiers namely as BayesNet [25], NaiveBayes [26], and NaiveBayesUpdateable [27] given Weka. 3.2. Function-based learners The function-based classification methods work to estimate the performance of the function like LibSVM, Logistic, Multilayer Perceptron, RBF Network, Simple Logistic, SMO, SPegasos and voted perceptron. For the comparative analysis of the cultural heritage database, the selected Function-based classifiers are namely i.e. Logistic[28], MultilayerPerceptron [29], Simple Logistic[30], and SMO [31] implemented in Weka 3.3. Instance-Based Classifiers The paradigm of the machine in which the process of generalization of the training is delayed until a query is made to the system is known as instance-based learning or Lazy learning classification methods [32]. In this article, we have selected tree instance-based learning classification methods i.e. IBk [33], KStar [34], and LWL[35]. The Weka implementation is exploited for this empirical research study. 3.4. Rule-Based Classifiers The paradigm of classification techniques exploits the rules like conjunctive rules for the task of classification [36]. The rule-based classification method is also known as the separate–and–conquer approach. This type of classification method follows an iterative process consisting of the generation of rules that covers a subset of the training samples. Then starts removing all examples covered by the rule from the training set. For the comparative performance analysis, four Rule-based classifiers are selected namely as DecisionTable [37], JRip [38], OneR [39], and PART [40] implemented in Weka. 3.5. Meta Classifiers The type of classification algorithms that use or combine multiple algorithms are known as Meta classification methods like AdaBoostM1, Attribute Selected Classifier, Bagging, Grid Search, Metacast, etc. This study selected five meta classifiers that are Bagging [41] , Regression [28], LogitBoost [42] , MultiClassClassifier and MultiClassClassifierUpdateable implemented in Weka Data Mining tool. 4. Data and Methodology This empirical study uses a speaker accent recognition data set (https://archive.ics.uci.edu/ml/datasets/Speaker+Accent+Recognition) [UCI Machine Learning Repository] for all the classification methods under the focus of the empirical study. The number of instances 329, number of attributes 12 and number of web hits 24149. All attributes of the data set are real and multivariate. We have used 10-fold data validation for the training and testing in this empirical analysis. For experimentation, we use Weka 3.9 implementation of classification methods. For empirical analysis investigation, we used Bayesian-based Classifiers (BayesNet, NaiveBayes, and NaiveBayesUpdateable) [26], Instance-Based Classifiers (IBk, KStar, LWL), Regression-based Classifiers (Logistic, MultilayerPerceptron, SimpleLogistic and SMO) and Rule-Based Classifiers (DecisionTable, JRip, OneR, PART). 5. Experimental Results This section provides the analysis of classification methods on cultural heritage data-base namely the "speaker accent recognition data set" publically available. The subsec-tions of this section discuss the performance of classification methods belonging to the major five classification families i.e. Bays Based Classifiers, SVM-based Classifiers, Instance-Based Classifiers, Rule-Based Classifiers, and Meta Classification Methods. All classification algorithms are evaluated on the same and single heritage data set. The performance of each algorithm is measured in terms of TP Rate, FP Rate, Precision, Recall, F-Measure, MCC ROC Area, PRC Area, and Kappa statistic. 5.1. Bays Based Classifiers Table 1 shows the comparative performance analysis of Bayesian-based classifiers namely as BayesNet [25], NaiveBayes[26], and NaiveBayesUpdateable [27] given Weka. The performance of Bayesian classifiers is measured in terms of TP Rate, FP Rate, Precision, Recall, F-Measure, MCC ROC Area, PRC Area, and Kappa statistic as shown in Table 1. The performance of each method is also represented in terms of average, maximum, and minimum values. The results depict that the performance of NaiveBayes and NaiveBayesUpdateable remains the same as compared to BayesNet. Table 1 Comparative performance analysis of Bays-based classification methods on accent recognition database. Bays Based Classifiers Evaluation BayesNet NaiveBayes NaiveBayes_ Average Max Min Measures Updateable TP Rate 0.728 0.755 0.755 0.746 0.755 0.728 FP Rate 0.091 0.082 0.082 0.085 0.091 0.082 Precision 0.734 0.751 0.751 0.745 0.751 0.734 Recall 0.728 0.755 0.755 0.746 0.755 0.728 F-Measure 0.728 0.749 0.749 0.742 0.749 0.728 MCC 0.640 0.672 0.672 0.661 0.672 0.640 ROC Area 0.910 0.913 0.913 0.912 0.913 0.910 PRC Area 0.778 0.788 0.788 0.785 0.788 0.778 Kappa 0.637 0.673 0.673 0.661 0.673 0.637 statistic 5.2. Function-based Classifiers Table 2 shows the comparative performance analysis of Function-based classifiers namely Logistic[28], MultilayerPerceptron [29], Simple Logistic[30], and SMO [31] implemented in Weka. The performance of Bayesian classifiers is measured in terms of TP Rate, FP Rate, Precision, Recall, F-Measure, MCC ROC Area, PRC Area, and Kappa statistic as shown in Table 2. The performance of each method is also represented in terms of average, maximum, and minimum values. The results depict that the performance of MultilayerPerceptron is promising for other competitive classifiers as shown in Table 2 with boldface values. Table 2 Comparative performance analysis of SVM-based classification methods on accent recognition database. Function-based Classifiers Evaluation Logistic Multilayer Simple SMO Average Max Min Measures Perceptron Logistic TP Rate 0.705 0.803 0.798 0.795 0.775 0.803 0.705 FP Rate 0.098 0.066 0.067 0.068 0.075 0.098 0.066 Precision 0.704 0.799 0.798 0.794 0.774 0.799 0.704 Recall 0.705 0.803 0.798 0.795 0.775 0.803 0.705 F-Measure 0.704 0.800 0.797 0.794 0.774 0.800 0.704 MCC 0.606 0.735 0.730 0.726 0.699 0.735 0.606 ROC Area 0.857 0.933 0.946 0.907 0.911 0.946 0.857 PRC Area 0.660 0.860 0.870 0.737 0.782 0.870 0.660 Kappa 0.607 0.737 0.730 0.727 0.700 0.737 0.607 statistic 5.3. Instance-Based Classifiers Table 3 shows the comparative performance analysis of Instance-based Classifiers known as Lazy Learning Classifiers namely as IBk [33], KStar [34], and LWL[35] implemented in Weka. The performance of Bayesian classifiers is measured in terms of TP Rate, FP Rate, Precision, Recall, F- Measure, MCC ROC Area, PRC Area, and Kappa statistic as shown in Table 3. The performance of each method is also represented in terms of average, maximum, and minimum values. The results depict that the performance of the KStar classifier is promising for other competitive classifiers as shown in Table 3 with boldface values. The performance of LWL is lowest to others in terms of FP Rate as shown in Table 3 while IBK outperformed in terms of F-Measure to other classifiers. Table 3 Comparative performance analysis of instance-based classification methods on accent recognition database. Instance-Based Classifiers Evaluation IBk KStar LWL Average Max Min Measures TP Rate 0.638 0.648 0.628 0.638 0.648 0.628 FP Rate 0.121 0.118 0.124 0.121 0.124 0.118 Precision 0.629 0.645 0.635 0.636 0.645 0.629 Recall 0.638 0.648 0.628 0.638 0.648 0.628 F-Measure 0.631 0.626 0.623 0.627 0.631 0.623 MCC 0.513 0.528 0.507 0.516 0.528 0.507 ROC Area 0.759 0.868 0.842 0.823 0.868 0.759 PRC Area 0.507 0.717 0.698 0.641 0.717 0.507 Kappa 0.517 0.530 0.503 0.517 0.530 0.503 statistic 5.4. Rule-Based Classifiers Table 4 shows the comparative performance analysis of Rule-based classifiers namely DecisionTable [37], JRip [38], OneR [39], and PART [40] implemented in Weka. The performance of Bayesian classifiers is measured in terms of TP Rate, FP Rate, Precision, Recall, F-Measure, MCC ROC Area, PRC Area, and Kappa statistic as shown in Table 4. The performance of each method is also represented in terms of average, maximum, and minimum values. The results depict that the performance of PART is promising to other competitive classifiers as shown in Table 4 with boldface values. The DecisionTable classifier and JRip classifier outperformed in terms of ROC Area and PRC Area respectively. The performance of OneR classifier remains behind others in terms of FP Rate as shown in Table 4 with boldface value. Table 4 Comparative performance analysis of Rule-based classification methods on accent recognition database. Rule Based Classifiers Evaluation DecisionTable JRip OneR PART Average Max Min Measures TP Rate 0.638 0.668 0.535 0.710 0.638 0.710 0.535 FP Rate 0.121 0.111 0.155 0.097 0.121 0.155 0.097 Precision 0.647 0.667 0.537 0.709 0.640 0.709 0.537 Recall 0.638 0.668 0.535 0.710 0.638 0.710 0.535 F-Measure 0.640 0.667 0.535 0.709 0.638 0.709 0.535 MCC 0.521 0.557 0.381 0.613 0.518 0.613 0.381 ROC Area 0.852 0.846 0.690 0.818 0.802 0.852 0.690 PRC Area 0.658 0.659 0.413 0.603 0.583 0.659 0.413 Kappa 0.517 0.557 0.380 0.613 0.517 0.613 0.380 statistic 5.5. Meta Classification Methods The Table 5 shows the comparative performance analysis of Meta based classifiers namely as Bagging [41] , Regression [28], LogitBoost [42] , MultiClassClassifier and MultiClassClassifierUpdateable implemented in Weka. The performance of Bayesian classifiers is measured in terms of TP Rate, FP Rate, Precision, Recall, F-Measure, MCC ROC Area, PRC Area, and Kappa statistic as shown in Table 5. The performance of each method is also represented in terms of average, maximum, and minimum values. The results depict that the performance of Regression is promising to other competitive classifiers as shown in Table 5 with boldface values. Table 5 Comparative performance analysis of Meta based classification methods on accent recognition database. Meta Classification Methods Evaluation Baggin Regression LogitBoost MultiClass MultiClass Average Max Min Measures g Classifier Classifier Updateable TP Rate 0.763 0.765 0.758 0.738 0.733 0.751 0.76 0.733 5 FP Rate 0.079 0.078 0.081 0.088 0.089 0.083 0.08 0.078 9 Precision 0.760 0.764 0.759 0.736 0.777 0.759 0.77 0.736 7 Recall 0.763 0.765 0.758 0.738 0.733 0.751 0.76 0.733 5 F-Measure 0.761 0.762 0.757 0.736 0.731 0.749 0.76 0.731 2 MCC 0.682 0.686 0.677 0.649 0.664 0.672 0.68 0.649 6 ROC Area 0.913 0.920 0.923 0.905 0.901 0.912 0.92 0.901 3 PRC Area 0.799 0.818 0.807 0.773 0.759 0.791 0.81 0.759 8 Kappa 0.683 0.687 0.677 0.650 0.643 0.668 0.68 0.643 statistic 7 5.6. Comparison of Classification Methods w.r.t Average Results Table 6 provides the empirical performance analysis of Five Classification Methods domains namely Bayesian, Function, Lazy, Meta, and Rule-based classifiers provided in the Weka implementation by considering the average values of each classification method category. The performance of all classifiers is measured in terms of TP Rate, FP Rate, Precision, Recall, F-Measure, MCC ROC Area, PRC Area, and Kappa statistic as shown in Table 6. The category of classifiers included in Function outperformed Bays, Lazy, Meta, and Rules-based classification methods as depicted in Table 6 with boldface values. With the aspect average values of each category Bays and Function, algorithms remained in a similar position in terms of ROC Area and PRC Area while Lazy and Rules-based methods remained behind in terms of FP Rate. Table 6 Comparative performance analysis of classification families w.r.t average results on accent recognition database. Classification Methods Evaluatio Bays Functio Lazy Meta Rules Averag Max Mi n n e n Measures TP Rate 0.75 0.78 0.64 0.72 0.64 0.70 0.78 0.64 FP Rate 0.09 0.07 0.12 0.09 0.12 0.10 0.12 0.07 Precision 0.75 0.77 0.64 0.73 0.64 0.70 0.77 0.64 Recall 0.75 0.78 0.64 0.72 0.64 0.70 0.78 0.64 F-Measure 0.74 0.77 0.63 0.72 0.64 0.70 0.77 0.63 MCC 0.66 0.70 0.52 0.63 0.52 0.60 0.70 0.52 ROC Area 0.91 0.91 0.82 0.88 0.80 0.87 0.91 0.80 PRC Area 0.78 0.78 0.64 0.74 0.58 0.71 0.78 0.58 Kappa 0.66 0.70 0.52 0.63 0.52 0.60 0.70 0.52 statistic 6. Conclusion Classification plays an important role in data mining and machine learning for the discovery of valuable and nontrivial knowledge from labeled data items. The classification methods promisingly and effectively exploit the classification of cultural heritage unstructured data objects. There is a variety of classification methods available in contemporary literature. This paper provides, an empirical performance analysis of Classification algorithms namely Bayesian(BayesNet, NaiveBayes, and NaiveBayesUpdateable), Function(Logistic, Multilayer Perceptron, Simple Logistic, and SMO), Lazy(IBK, KStar, and LWL), Meta(Bagging, Regression, LogitBoost, MultiClassClassifier, and Mul- tiClassClassifierUpdateable) and Rule-based (DecisionTable , JRip , OneR, and PART) classifiers provided in the Weka implementation by considering the average values of each classification method category. The performance of classifiers is evaluated in terms of TP Rate, FP Rate, Precision, Recall, F- Measure, MCC, ROC Area, PRC Area, and Kappa statistic. The empirical results analysis shows that the performance of NaiveBayes and NaiveBayesUpdateable classifiers remain the same within the Bayesian classifiers' completion. In the group of Function classifiers, the performance of MultilayerPerceptron is promising to others. For Lazy classifiers, KStar is a winner in the context of most of the evaluation metrics. In the case of rule-based classifiers, PART is leading in terms of evaluation metrics. The Regression classifier outperformed the class of Meta classifiers. With the consideration of average evaluation metrics values of each category of classifiers, the Function category significantly better performed concerning other competitive classification categories. In future work, we consider the larger number of cultural heritage data sets to evaluate the performance of the state of art classification algorithms. 7. Acknowledgements I am thankful to Higher Education Commission (HEC) of Pakistan for supporting and funding for higher education in the form PhD Scholarship. References [1] R. Jigyasu et al., "Heritage and resilience: issues and opportunities for reducing disaster risks," 2013. [2] R. J. J. o. H. M. Jigyasu, "Reducing disaster risks to urban cultural heritage: global challenges and opportunities," vol. 1, no. 1, pp. 59-67, 2016. [3] M. Cosovic, A. Amelio, and E. Junuz, "Classification Methods in Cultural Heritage," in VIPERC@IRCDL, 2019. [4] H. Kurniawan, A. Salim, H. Suhartanto, and Z. A. Hasibuan, "E-cultural heritage and natural history framework: an integrated approach to digital preservation," in International Conference on Telecommunication Technology and Applications (IACSIT), 2011, pp. 177-182. [5] J. Wu, Z. Gao, and C. Hu, "An empirical study on several classification algorithms and their improvements," in International Symposium on Intelligence Computation and Applications, 2009, pp. 276-286: Springer. [6] S. Taruna and M. Pandey, "An empirical analysis of classification techniques for predicting academic performance," in 2014 IEEE International Advance Computing Conference (IACC), 2014, pp. 523-528: IEEE. [7] A. Kaur, I. J. J. o. K. S. U.-C. Kaur, and I. Sciences, "An empirical evaluation of classification algorithms for fault prediction in open source projects," vol. 30, no. 1, pp. 2-17, 2018. [8] Z. Ali, W. J. I. J. o. A. C. S. Shahzad, and Applications, "Comparative analysis and survey of ant colony optimization based rule miners," vol. 8, no. 1, 2017. [9] Z. Ali, R. Ahmad, M. N. Akhtar, Z. H. Chuhan, H. M. Kiran, and W. Shahzad, "Empirical Study of Associative Classifiers on Imbalanced Datasets in KEEL," in 2018 9th International Conference on Information, Intelligence, Systems and Applications (IISA), 2018, pp. 1-7: IEEE. [10] R. Jankovic, A. Amelio, and Z. A. Ranjha, "Classification of Energy Consumption in the Balkans using Ensemble Learning Methods," in 2019 2nd International Conference on Advancements in Computational Sciences (ICACS), 2019, pp. 1-8: IEEE. [11] Z. Ali, I. U. Rehman, and Z. Jaan, "An Empirical Analysis on Software Development Efforts Estimation in Machine Learning Perspective," 2021. [12] Y. Yao, Z. Xiao, B. Wang, B. Viswanath, H. Zheng, and B. Y. Zhao, "Complexity vs. performance: empirical analysis of machine learning as a service," in Proceedings of the 2017 Internet Measurement Conference, 2017, pp. 384-397. [13] S. Sharma, V. Mittal, R. Srivastava, and S. Singh, "Empirical Evaluation of Various Classification Methods," in 2020 2nd International Conference on Advances in Computing, Communication Control and Networking (ICACCCN), 2020, pp. 105-109: IEEE. [14] U. Ishfaq et al., "Empirical Analysis of Machine Learning Algorithms for Multiclass Prediction," vol. 2022, 2022. [15] C. Zhu, L. H. N. Fong, and M. J. C. I. i. T. Gan, "Rethinking the consequences of postmodern authenticity: the case of a World Cultural Heritage in Augmented Reality," pp. 1-15, 2022. [16] J. Llamas, P. M Lerones, R. Medina, E. Zalama, and J. Gómez-García-Bermejo, "Classification of architectural heritage images using deep learning techniques," Applied Sciences, vol. 7, no. 10, p. 992, 2017. [17] U. Kulkarni, S. Meena, S. V. Gurlahosur, and U. Mudengudi, "Classification of cultural heritage sites using transfer learning," in 2019 IEEE fifth international conference on multimedia big data (BigMM), 2019, pp. 391-397: IEEE. [18] R. Jankovic, "Classifying cultural heritage images by using decision tree classifiers in WEKA," in Proceedings of the 1st International Workshop on Visual Pattern Extraction and Recognition for Cultural Heritage Understanding Co-Located with 15th Italian Research Conference on Digital Libraries (IRCDL 2019), Pisa, Italy, 2019, pp. 119-127. [19] M. Ćosović and R. Janković, "CNN classification of the cultural heritage images," in 2020 19th International Symposium INFOTEH-JAHORINA (INFOTEH), 2020, pp. 1-6: IEEE. [20] F. Zhipeng and H. Gani, "Cultural Events Classification using Hyper-parameter Optimization of Deep Learning Technique," International Journal of Advanced Computer Science and Applications, vol. 12, no. 5, 2021. [21] S. Prasomphan, "Cultural Heritage Content Management System by Deep Learning," in Proceedings of the 2020 Asia Service Sciences and Software Engineering Conference, 2020, pp. 21-26. [22] W. Ali, J. Kumar, and J. J. A. I. Shao, "Inferring context with reliable collaborators: a novel similarity estimation method for recommender systems," pp. 1-19, 2022. [23] Z. Deng, C. Li, S. Liu, W. Ali, and J. Shao, "Knowledge-aware group representation learning for group recommendation," in 2021 IEEE 37th International Conference on Data Engineering (ICDE), 2021, pp. 1571-1582: IEEE. [24] W. Ali, R. Kumar, Z. Deng, Y. Wang, and J. J. T. C. J. Shao, "A federated learning approach for privacy protection in context-aware recommender systems," vol. 64, no. 7, pp. 1016-1027, 2021. [25] N. Friedman, D. Geiger, and M. Goldszmidt, "Bayesian network classifiers," Machine learning, vol. 29, no. 2, pp. 131-163, 1997. [26] H. Zhang, "The optimality of naive Bayes," Aa, vol. 1, no. 2, p. 3, 2004. [27] G. H. John and P. Langley, "Estimating continuous distributions in Bayesian classifiers," arXiv preprint arXiv:1302.4964, 2013. [28] S. Le Cessie and J. C. Van Houwelingen, "Ridge estimators in logistic regression," Journal of the Royal Statistical Society: Series C (Applied Statistics), vol. 41, no. 1, pp. 191-201, 1992. [29] Y. Liu, Z. You, and L. Cao, "A novel and quick SVM-based multi-class classifier," Pattern Recognition, vol. 39, no. 11, pp. 2258-2264, 2006. [30] D. R. Cox, "The regression analysis of binary sequences," Journal of the Royal Statistical Society: Series B (Methodological), vol. 20, no. 2, pp. 215-232, 1958. [31] J. Platt, "Sequential minimal optimization: A fast algorithm for training support vector machines," 1998. [32] J. Tighe and S. Lazebnik, "Superparsing," International Journal of Computer Vision, vol. 101, no. 2, pp. 329-349, 2013. [33] D. W. Aha, D. Kibler, and M. K. Albert, "Instance-based learning algorithms," Machine learning, vol. 6, no. 1, pp. 37-66, 1991. [34] J. G. Cleary and L. E. Trigg, "K*: An instance-based learner using an entropic distance measure," in Machine Learning Proceedings 1995: Elsevier, 1995, pp. 108-114. [35] C. G. Atkeson, A. W. Moore, and S. Schaal, "Locally weighted learning," Lazy learning, pp. 11-73, 1997. [36] B. Qin, Y. Xia, S. Prabhakar, and Y. Tu, "A rule-based classification algorithm for uncertain data," in 2009 IEEE 25th international conference on data engineering, 2009, pp. 1633-1640: IEEE. [37] R. Kohavi, "The power of decision tables," in European conference on machine learning, 1995, pp. 174- 189: Springer. [38] W. W. Cohen, "Fast effective rule induction," in Machine learning proceedings 1995: Elsevier, 1995, pp. 115-123. [39] R. C. Holte, "Very simple classification rules perform well on most commonly used datasets," Machine learning, vol. 11, no. 1, pp. 63-90, 1993. [40] E. Frank and I. H. Witten, "Generating accurate rule sets without global optimization," 1998. [41] L. Breiman, "Bagging predictors," Machine learning, vol. 24, no. 2, pp. 123-140, 1996. [42] J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors)," The annals of statistics, vol. 28, no. 2, pp. 337-407, 2000.