within the IEEE WORLD CONGRESS ON COMPUTATIONAL INTELLIGENCE (WCCI2022)

-

Proceedings Workshop Organization Organizing Committee

Gabriella Casalino, University of Bari, Italy

Giovanna Castellano, University of Bari, Italy

Katarzyna Kaczmarek-Majer, Polish Academy of Sciences, Poland

Daniel Leite, Universidad Adolfo Ibáñez, Santiago, Chile

Program Committee

Sašo Blažič (University of Ljubljana, Slovenia)

Przemysław Grzegorzewski (Warsaw University of Technology, Poland)

Leandro Maciel (University of São Paulo, Brazil)

Corrado Mencar (Università degli Studi di Bari Aldo Moro, Italy)

Zied Mnasri (University of Tunis El Manar, Tunisia)

Daniel Peralta (Ghent University, Belgium)

Mahardika Pratama (Nanyang Technological University, Singapore)

Sławomir Zadrożny (University of Warsaw, Poland)

Choiru Za'in (Monash University, Australia)

10.00-10.10 Opening (Chairs: Gabriella Casalino and Katarzyna Kaczmarek-Majer) 10.10-11.00 Keynote Talk:

Plamen Angelov, Online Learning of Interpretable Deep Models from Uncertain Data (streams)

11.00-12.00 Session 1 (Chair: Giovanna Castellano) - P. V. Campos Souza and E. Lughofer, An explainable evolving fuzzy neural network in position identification of basketball players - B. Li and E. Müller, STAD: State-Transition-Aware Anomaly Detection Under

Concept Drifts

- D. Leite, State-Space Evolving Granular Control of Unknown Dynamic

Systems

Online Learning from

Uncertain

Data Streams:

Editorial

Daniel Leite3 Gabriella Casalino1, Giovanna Castellano1, Katarzyna Kaczmarek-Majer2 and

1Computer Science Dept., University of Bari Aldo Moro, Bari, Italy

2Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

3Faculty of Engineering and Science, Universidad Adolfo Ibáñez, Santiago, Chile

This editorial note provides an overview of the papers accepted to the First Workshop on Online Learning from Uncertain Data Streams (OLUD 2022) and related sub-areas. The OLUD workshop was intended to facilitate interdisciplinary discussion on recent advancements of state-of-the-art online machine learning and incremental pattern recognition methods. The uncertainty inherent to the data, model parameters and learning procedures, as well as the implication of the uncertainties on applied domains, was a concern of the discussions. Model explainability, especially by means of a rule base and linguisticallytranslated elements, was also emphasized. The workshop was held in Padua (Italy), on July 18, 2022, in conjunction with the IEEE World Congress on Computational Intelligence (IEEE WCCI 2022). This preface summarizes the motivations of the meeting, the contributing papers and their findings, and the open topics discussed by the participants, which may stimulate new research. 1. Area Overview and OLUD

Motivations Nowadays, applications in various domains (computer science, engineering, medicine, economy, etc.) are based on sensor data and/or depend on data transmission in the cloud. Efective modeling approaches to address such a massive amount of dynamically-changing data in a feasible period of time are of utmost importance. Traditional first principles and ofline datadriven modeling approaches for static datasets are very often insuficient or inefective in online data stream environments. In such environments, fast recursive procedures to capture spatio-temporal patterns and concept drifts and shifts from the data are needed. Often, the data lfow brings instances in high frequency. Narrow time and memory constraints are available for an incremental machine learning step per instance, or per a small set of instances.

Models should be parametrically and structurally updated as a consequence of multiple types of changes that may occur in the data sources. Moreover, data streams may carry statistical, possibilistic and fuzzy uncertainties that arise in specific technical and contextual domains, which need to be adequately addressed. Finally, computational models that present a higher level of human-understandability have been demanded in several domains. For example: (i) in nEvelop-O rOcid OLUD’22: Workshop on Online Learning from Uncertain Data Streams, July 18, 2022, Padua, Italy [e-mail] (G. Casalino); [e-mail] (G. Castellano);

0000-0003-0713-2260 (G. Casalino); 0000-0002-6489-8628 (G. Castellano); 0000-0003-0422-9366 (K. Kaczmarek-Majer); 0000-0003-3135-2933 (D. Leite)

© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Workshop Proceedings htp:/ceur-ws.org CEUR Workshop Proceedings (CEUR-WS.org)

ISN1613-073 domains in which the final users are non-technicians; (ii) to support expert decision making in medicine, engineering, meteorology, energy, logistics; (iii) to explain the actions taken by unmanned vehicles and mobile robots, which afect the human environment. Thus, new methods to linguistically explain the approximate reasoning behind the outcomes of a model are needed to achieve trustful and reliable results. The OLUD (Online Learning from Uncertain Data Streams) workshop (sites.google.com/view/olud) addressed topics on uncertainty in online machine learning, leaving room to several open questions: • How explainability can be achieved in online learning? • How uncertainty handling can improve online learning? • How hybrid methods and elements from diferent theories can be combined to exploit their benefits for online learning?

The workshop brought together theorists and practitioners who apply computational intelligence, statistics, and control methods for sequential and uncertain data analysis to exchange and discuss ideas that enrich traditional approaches useful for static datasets. The workshop was attended by experts and an audience from diferent research communities – such as: (i) incremental learning from stream data; (ii) soft methods for stream data; (iii) fuzzy statistics; (iv) Big data; (v) uncertainty modeling; (vi) evolving neural and neuro-fuzzy networks. 2. Event Synopsis The workshop started with a keynote talk by Plamen Angelov titled Online Learning of Interpretable Deep Models from Uncertain Data Streams, which is based on his recent findings [ 1, 2 ]. Current machine learning methods are often focused on accuracy, whereas overlook aspects such as explainability, semantic meaning of the internal model representation, reasoning, and the model link with the problem domain. They also overlook the eforts to collect and label training data and rely on assumptions about the data distribution that are often unsatisfactory. The keynote speaker has addressed open issues on developing highly eficient and accurate algorithms; and models that are transparent, explainable, and fair by design. Such models are able to continuously learn and improve their estimates or actions over time. Complete model re-training after significant changes is needless. Learning can start from a few training instances, and never-before-seen patterns can be detected on the fly. Such evolving models would be able to collaborate with humans and other such algorithms.

The OLUD workshop attracted 12 full-length papers from computational intelligence, control and statistics, from which 9 were accepted. The contributions are: 1) An Explainable Evolving Fuzzy Neural Network in Position Identification of Basketball Players, by Paulo Vitor Campos Souza and Edwin Lughofer. This paper applies an evolving fuzzy neural network to identify the position of players on a basketball court. The Spanish Basketball League dataset contains 4 classes (point guard, shooting guard, small forward, and center). An instance related to a player is described by 13 features (height, blocks, rebounds, assists, points, personal fouls committed and received, free throw percentage, 2-point and 3-point field goal percentages, turnovers, steals, and global assessment). In addition to provide accurate classification, the evolving neuro-fuzzy model displayed interpretable information to help decision making. For example, the model indicated that the player height is a determining factor for his position on the court. The ability on blocking, rebounding, and 3-point shooting can also facilitate the player’s on-court functionality. See [ 3, 4 ] for further applications and diferent types of evolving neuro-fuzzy networks studied by the authors. 2) STAD: State-Transition-Aware Anomaly Detection Under Concept Drifts , by Bin Li and Emmanuel Muller. This paper proposes an autoencoder-based approach called STAD for anomaly detection under potential concept changes. A state-transition-based model is used to map diferent data distributions within windows of the data stream into states, thereby addressing the model adaptation problem in an interpretable way. The state transition process was empirically evaluated and demonstrated for detecting anomalies in a Covid-19 dataset from Germany. While typical ofline-designed autoencoders for unsupervised anomaly detection become invalid after distributional drifts of the data stream, STAD overcomes this issue by exploring the temporal context. For additional information of the research area refer to [ 5, 6 ]. 3) Fuzzy Hoefding Decision Trees for Learning Analytics , by Gabriella Casalino, Pietro Ducange, Michela Fazzolari and Riccardo Pecori. This paper presents a case study of explainable stream data analysis in the educational domain. Students’ interactions with Virtual Learning Environment together with students’ information, have been proved to be suitable predictors of the students’ outcomes [ 7 ]. In this paper the intrinsic evolving nature of the students’ learning has been exploited though a stream data analysis algorithm. Particularly, Fuzzy Hoefding Decision Trees, proposed in [ 8 ], have been used to describe the students’ learning behaviors over sequential semesters, in form of “IF-THEN” rules, and to predict their outcomes. This is an innovative research topic combining stream data processing, explainability and fuzzy logic for the educational domain. These topics are individually addressed in the literature [ 9, 10, 11 ]. 4) Hoefding Regression Trees for Forecasting Quality of Experience in B5G/6G Networks , by José Luis Corcuera Bárcena, Pietro Ducange, Francesco Marcelloni, Alessandro Renda, and Fabrizio Rufini. This paper describes the use of Hoefding Regression Trees (HRTs) to evaluate the end-user satisfaction in terms of “Quality of Experience” (QoE). They measure the capability to play high-definition videos in real-time with B5G/6G networks. Standard Regression Trees (RTs) have been compared with HRTs to forecast real data. Diferent parameters have been considered in evaluating the two approaches, such as, accuracy, time for model updating, model complexity and memory occupancy. For additional information on forecasting QoE in networks refer to [ 12, 13, 14 ] 5) Fuzzy-based Process Mining to Discover the Coding Behavior: Challenges and Future Works, by Pasquale Ardimento, Lerina Aversano, Mario Luca Bernardi, and Marta Cimitile. This paper describes a learning environment for object-oriented coding, that is able to identify patterns in students’ coding behaviours, through a fuzzy logic based process mining approach. The interactions between students and the learning environment are collected over the time and are stored in form of logs. For additional information of the process mining applied to the educational and coding domain refer to [ 15, 16, 17 ]. 6) State-Space Evolving Granular Control of Unknown Dynamic Systems, by Daniel Leite. The paper describes a State-Space Evolving Granular Control method (SS-EGC) for unknown nonlinear dynamic systems. The approach is based on elements of granular computing, discrete state-space systems, and online machine learning. The structure and parameters of a granular model is developed from a stream of state data. The model is formed by information granules comprising first-order diference equations. A granular controller is derived from the granular model for parallel distributed compensation. Instead of diference equations, the content of a control granule is a gain matrix, which can be redesigned in real-time from the solution of a relaxed locally-valid linear matrix inequality derived from an energy Lyapunov function. For information on the research area of evolving granular control, and applications in stabilization of nonlinear systems refer to [18, 19, 20, 21, 22]. 7) Time series classification using F-transform , by Przemysław Grzegorzewski and Antoni Kędzierski. This paper describes a new method for time series classification. Two techniques were evaluated: (i) the fuzzy transform (F-transform), which provides a simple approximate representation of functions; and (ii) the well-known decision tree classifier. The objective is to compare the best distance measures to be used in both the proposed method and the most popular 1-NN method. Numerical experiments confirmed that the proposed F-transform-based classification method reveals the smallest standard deviation of the error, and one of the smallest mean errors. For further results on the use of F-transforms in time-indexed data refer to [23, 24]. 8) Evolving Membership Functions in Fuzzy Linguistic Summarization, by Katarzyna Kaczmarek-Majer, Aleksandra Rutkowska, and Olgierd Hryniewicz. Inspired by the general concept of evolving fuzzy systems, this paper introduces a time-dependent procedure for the construction of membership functions in linguistic summarization. Linguistic terms are automatically derived from a mathematical model to compose membership functions. In particular, stationary autoregressive and moving average (ARMA) models are estimated. Then, components of linguistic summarization are gradually changed by online learning from new data instances and statistical inference. The usefulness of the proposed approach is illustrated in an economic time series prediction problem. For additional information on the research area, refer to [25]. 9) Typicality based Fuzzy Gradual Rules Model for Real-Time Emotions Assessment through Physiological Signals, by Joseph Onderi Orero. This paper addresses online afective computing in order to enhance the quality of human-computer interactions. The essential idea is that computational models should automatically adapt themselves based on human afective states. Emphasis is put on promoting empathy between machine and autistic people. Emotional responses were accessed by means of uncertain and imperfect physiological measures using bio-sensors. As physiological patterns change over time and change from person to person, a fuzzy rule-based model supported by the concept of typicality of the flowing data was developed and updated gradually to characterize diferent afective states over the time. For additional information about online learning applied to emotion recognition and afective computing refer to [ 26, 27, 28, 29].

Overall the OLUD contributions highlighted the ability of online machine learning methods and evolving fuzzy and neuro-fuzzy systems to handle complex nonstationary and uncertain data; thus indicating multiple avenues for future research. Particularly, in addition to the paper topics, the on-site participants leveraged and discussed future and open research issues. Some issues are mentioned in the following. 3. Open Topics Interesting and persuasive practical solutions have been achieved in the area of intelligent modeling of uncertain data streams in the last decade. Some future directions for making adaptive and evolving methods suitable to a broader field of applications, especially for Big data processing, Internet of Things, eXplainable Artifcial Intelligence, Cyber-Physical Systems, and Smart Industry, are described in the following.

Propositions, lemmas, theorems and assurance that certain conditions are fulfilled are still lacking, in large part, in the field of online clustering and evolving fuzzy and neuro-fuzzy modeling from data streams [30]. For instance, necessary and suficient conditions to guarantee short-term adaptation and long-term survivability are still to be found. This is a major challenge because concept shift and concept drift afect the structure of the hypothesis space [ 30]. Systematic and formal methods to deal with the models’ structural stability-plasticity trade-of are still needed.

Characterization, design of experimental setups, and construction of workflows to guide development, performance evaluation, testing, validation, and comparison of methods in nonstationary environments require further elaboration. The evolution of rough-set models, DempsterShafer models, second-order granular rule-based models, and aggregation functions are also important topics to expand the current scope of the area [30]. T-norms and S-norms, Uni-norms and null-norms, and averaging functions are generally chosen a priori and kept fixed during model evolution. Approaches to switch aggregation operators based on properties of the data, and to update associated operator parameters are still to be undertaken.

Evolving and adaptive systems in parallel high-performance computing frameworks should be explored. The rule-base modular and granular structure of fuzzy models is an interesting aspect to be exploited in high-frequency stream applications. Moreover, a variety of particularities of diferent applications and evolution aspects in hardware using low resources – aiming at smarter evolving models – are still to be addressed.

We have witnessed the expansion of the OLUD topics to the area of deep learning, image processing, and autoencoders [ 1 ]; streams of text and sentences, and log parsing [31]; decision and regression trees in the educational domain [32]; autonomous robots with evolving capabilities [33]; linguistic summarization for augmented interpretability [34]; brain-computer interfaces [27, 28, 35]; weather prediction [36]; power systems [37]; fault detection in engineering systems [38, 39]; and patient monitoring and medical decision support [25, 40].

Acknowledgments The OLUD organizers thank the 24 authors who responded to the call, the program committee that helped with paper reviews, and the CEUR publication team. G. Casalino acknowledges funding from the Italian Ministry of Education, University and Research through the European PON project AIM (Attraction and International Mobility), nr. 1852414, activity 2, line 1. [18] D. Leite, R. M. Palhares, V. C. Campos, F. Gomide, Evolving granular fuzzy model-based control of nonlinear dynamic systems, IEEE Trans Fuzzy Syst. 23 (2015) 923–938. [19] Ž. Stržinar, I. Škrjanc, Self-tuned model-based predictive control using evolving fuzzy model of a non-linear dynamic process, in: J. Rayz, V. Raskin, S. Dick, V. Kreinovich (Eds.), Explainable AI and Other Applications of Fuzzy Techniques, Springer, 2022, pp. 406–421. [20] C. Aguiar, D. Leite, D. Pereira, G. Andonovski, I. Škrjanc, Nonlinear modeling and robust lmi fuzzy control of overhead crane systems, Journal of the Franklin Institute 358 (2021) 1376–1042. [21] R.-E. Precup, T.-A. Teban, A. Albu, A.-B. Borlea, I. A. Zamfirache, E. M. Petriu, Evolving fuzzy models for prosthetic hand myoelectric-based control, IEEE Transactions on Instrumentation and Measurement 69 (2020) 4625–4636. [22] L. A. Q. Cordovil, P. H. S. Coutinho, I. V. de Bessa, M. F. S. V. D’Angelo, R. M. Palhares, Uncertain data modeling based on evolving ellipsoidal fuzzy information granules, IEEE Transactions on Fuzzy Systems 28 (2020) 2427–2436. [23] M. Stepnicka, P. Cortez, J. P. Donate, L. Stepnickova, Forecasting seasonal time series with computational intelligence: On recent methods and the potential of their combinations, Expert Systems with Applications 40 (2013) 1981–1992. [24] S. Mirshahi, V. Novak, A fuzzy method for evaluating similar behavior between assets,

Soft Computing 25 (2021) 7813–7823. [25] K. Kaczmarek-Majer, G. Casalino, G. Castellano, O. Hryniewicz, M. Dominiak, Explaining smartphone-based acoustic data in bipolar disorder: Semi-supervised fuzzy clustering and relative linguistic summaries, Information Sciences 588 (2022) 174–195. [26] J. O. Orero, M. Rifqi, Design of a fuzzy afective agent based on typicality degrees of physiological signals, in: 15th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU’14), France, 2014. [27] C. Tan, M. Šarlija, N. Kasabov, Neurosense: Short-term emotion recognition and understanding based on spiking neural network modelling of spatio-temporal EEG patterns, Neurocomputing 434 (2021) 137–148. [28] D. Leite, V. F. Jr., R. Medeiros, Incremental fuzzy machine learning for online classification of emotions in games from EEG data streams, in: P. Angelov (Ed.), Handbook on Computer Learning and Intelligence, World Scientific, 2022, pp. 235–262. [29] L. Ghosh, S. Saha, A. Konar, Decoding emotional changes of android-gamers using a fused type-2 fuzzy deep neural network, Computers in Human Behavior 116 (2021) 14p. [30] D. Leite, I. Skrjanc, F. Gomide, An overview on evolving systems and learning from stream data, Evolving systems 11 (2020) 181–198. [31] L. Decker, D. Leite, D. Bonacorsi, Explainable log parsing and online interval granular classification from streams of words, in: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 2022, pp. 1–8. [32] G. Casalino, P. Ducange, M. Fazzolari, R. Pecori, Incremental and interpretable learning analytics through fuzzy hoefding decision trees, in: International Conference on Higher Education Learning Methodologies and Technologies Online, Springer, 2023. [33] M. Ferdaus, M. Pratama, S. Anavatti, M. Garratt, E. Lughofer, PAC: A novel self-adaptive neuro-fuzzy controller for micro aerial vehicles, Info. Sci. 512 (2020) 481–505. [34] K. Kaczmarek-Majer, G. Casalino, G. Castellano, M. Dominiak, O. Hryniewicz, O. Kamińska, G. Vessio, N. Díaz-Rodríguez, Plenary: Explaining black-box models in natural language through fuzzy linguistic summaries, Information Sciences 614 (2022) 374–399. [35] J. de Jesús Rubio, A. Bouchachia, Msafis: An evolving fuzzy inference system, Soft

Computing 21 (2017) 2357–2366. [36] G. R. Lencione, A. O. C. Ayres, F. J. V. Zuben, Online convex optimization of a multi-task fuzzy rule-based evolving system, in: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 2022, pp. 1–8. [37] K. S. T. R. Alves, E. P. de Aguiar, A novel rule-based evolving fuzzy system applied to the thermal modeling of power transformers, Applied Soft Computing 112 (2021). [38] M. R. Santos, B. S. J. Costa, C. G. Bezerra, G. Andonovski, L. A. Guedes, An evolving approach for fault diagnosis of dynamic systems, Expert Syst. Appl. 189 (2021) 10p. [39] G. Andonovski, G. Music, S. Blazic, I. Skrjanc, Evolving model identification for process monitoring and prediction of non-linear systems, Engineering Applications of Artificial Intelligence 68 (2018) 214–221. [40] F. Hassan, M. E. Shaheen, R. Sahal, Real-time healthcare monitoring system using online machine learning and spark streaming, International Journal of Advanced Computer Science and Applications 11 (2020) 650–658.

References

[1] P. Angelov , E. Soares, Towards explainable deep neural networks (xDNN) , Neural Networks 130 ( 2020 ) 185 - 194 .

[2] X. Gu , J. Han, Q. Shen , P. Angelov , Autonomous learning for fuzzy systems: a review , Artif Intell Rev 1 ( 2022 ) 1 - 47 .

[3] P. V. C. Souza , Fuzzy neural networks and neuro-fuzzy networks: A review the main techniques and applications used in the literature , Appl. Soft Comput . 92 ( 2020 ) 26p .

[4] P. V. C. Souza , E. Lughofer, An advanced interpretable fuzzy neural network model based on uni-nullneuron constructed from n-uninorms , Fuzzy Sets Syst 426 ( 2022 ) 1 - 26 .

[5] J. Lu , A. Liu , F. Dong , F. Gu , J. Gama , G. Zhang, Learning under concept drift: A review , IEEE Transactions on Knowledge and Data Engineering 31 ( 2018 ) 2346 - 2363 .

[6] J. Gama , I. Žliobaitė , A. Bifet , M. Pechenizkiy , A. Bouchachia , A survey on concept drift adaptation, ACM computing surveys (CSUR) 46 ( 2014 ) 1 - 37 .

[7] W. Chango , J. A. Lara , R. Cerezo , C. Romero , A review on data fusion in multimodal learning analytics and educational data mining, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery ( 2022 ) e1458 .

[8] P. Ducange , F. Marcelloni , R. Pecori , Fuzzy Hoefding Decision Tree for Data Stream Classification, Int . J. Comput. Intell . 14 ( 2021 ) 946 - 964 .

[9] H. Khosravi , S. B. Shum , G. Chen, C. Conati , D. Gasevic , J. Kay , S. Knight , R. MartinezMaldonado, S. Sadiq, Y.-S. Tsai , Explainable Artificial Intelligence in education, Computers and Education: Artificial Intelligence ( 2022 ) 100074 .

[10] M. Tsiakmaki , G. Kostopoulos, S. Kotsiantis , O. Ragos , Fuzzy-based active learning for predicting student academic performance using autoML: a step-wise approach , Journal of Computing in Higher Education 33 ( 2021 ) 635 - 667 .

[11] M. Y. Zavaleta-Sánchez , E. Benítez-Guerrero , Data stream mining for dynamic student modeling , in: Advances in Computational Intelligence: 21st Mexican International Conference on Artificial Intelligence , Springer, 2022 , pp. 161 - 172 .

[12] A. Renda , P. Ducange , G. Gallo , F. Marcelloni , Xai models for quality of experience prediction in wireless networks , in: 2021 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) , IEEE, 2021 , pp. 1 - 6 .

[13] V. Vasilev , J. Leguay , S. Paris, L. Maggi, M. Debbah , Predicting qoe factors with machine learning , in: 2018 IEEE Int. Conference on Communications (ICC) , IEEE, 2018 , pp. 1 - 6 .

[14] F. Tang , B. Mao , Y. Kawamoto , N. Kato, Survey on machine learning for intelligent endto-end communication toward 6G: From network access, routing to trafic control and streaming adaption , IEEE Communications Surveys & Tutorials 23 ( 2021 ) 1578 - 1598 .

[15] C. S. Garcia , A. Meincheim , E. R. F. Junior , M. R. Dallagassa , D. M. V. Sato , D. R. Carvalho , E. A. P. Santos , E. E. Scalabrin , Process mining techniques and applications-a systematic mapping study , Expert Systems with Applications 133 ( 2019 ) 260 - 295 .

[16] A. Bogarín , R. Cerezo , C. Romero , A survey on educational process mining , Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8 ( 2018 ) e1230 .

[17] M. I. Azeem , F. Palomba , L. Shi , Q. Wang , Machine learning techniques for code smell detection: A systematic literature review and meta-analysis , Information and Software Technology 108 ( 2019 ) 115 - 138 .