Capabilities of Symbiotic AI systems in FAIR (Future Artificial Intelligence Research)

Abstract

Nowadays, we need AI systems capable of engaging and working with people, perceiving and acting within changing contexts, being aware of their limits and adapting to new scenarios, behaving correctly in complex social settings, being aware of their security and trust perimeters, and of being aware of the environmental and societal efect that their implementation and execution may imply. In summary, we require a formerly undiscovered AI. Symbiotic-AI systems should disclose human cognitive capabilities to improve the efectiveness of information access and decision-making. This is achieved by combining methods for determining who is interacting with the system and how. The former includes the definition of strategies for acquiring and exploiting users' personal information gathered by combining diferent strategies and heterogeneous sources. In contrast, the latter includes detecting and interpreting human signals acquired from various sources, such as advanced machine learning (particularly deep learning) and natural language processing.

Keywords

• Artificial Intelligence Research)

1. Introduction

Future Artificial Intelligence Research (FAIR) is the Italian

ate in real-world environments; 4. adaptive: the ability to respond in rapidly changing circumstances; 5. highquality: safety-critical applications; 6. symbiotic: encourgic Program. FAIR accepts the challenge of defining the AI research community’s answer to the National Strate- ages eficient human-machine contact and collaboration; 7. edge/exascale: operates on the edge and on the cloud; research agenda for tomorrow’s AI approaches and tech- 8. pervasive: function ubiquitously; 9. environmentally forecasts. This is why fundamental, interdisciplinary re- its three primary goals: 1) enabling SAI system design centered: co-evolve with humans ”in-the-loop” both in- approaches to endow AI systems with human understandniques. Despite AI’s progress, its acceptance has been primarily in low-risk applications, with medium/highrisk applications, such as healthcare, public administration, and safety-critical industries, still lagging behind search is required to shape future AI. FAIR would like to work on future types of artificial intelligence: 1. humandividually and collectively; 2. integrative: a link between various AI approaches; 3. resilient: the ability to oper

2. Symbiotic-AI

The research is founded on the novel concept that AI aug

ments (and values) human cognitive talents rather than replacing them and hence supports and facilitates human activities [ 1 ]. In terms of human comprehension, we investigate strategies for learning who is engaging with signals obtained from various sources using deep/ma

In terms of AI system understandability, we will look into

2.1. Data curation and ingestion

human mental models and their key ability to deal with sentiment analysis [ 12, 13, 14, 15, 16, 17 ], machine transambiguity and imprecision in decision-making processes. lation, and question answering. Multimodal embeddings The activities will take two paths: i) Explainable AI for may be constructed using a variety of methods, includSymbiotic-AI [2]; (ii) methodologies for assessing and ing fusion-based and alignment-based approaches. The conveying uncertainty and imprecision. Research on the input from several modalities is integrated to generate acceptability of Symbiotic-AI will use an interdisciplinary a single embedding vector in fusion-based approaches. approach, with academics from AI, Law, and Ethics par- Deep learning architectures like CNNs and RNNs are ticipating. The long-term viability of Symbiotic-AI is common approaches for constructing multimodal embedexplored from two viewpoints: model recycling to con- dings. Multimodal embeddings have been employed in serve computing resources for model learning and tuning a variety of applications, including picture captioning, and data collection, labeling, and representation efort visual question answering, and multimodal sentiment reduction. analysis [18]. They allow the models to reason across multiple modalities and give a more thorough grasp of the underlying data.

This research task involves creating, organizing, and maintaining large volumes of data or corpora that are used to train and test large language models. The process of data curation and ingestion is a complex and timeconsuming task, but it is necessary to ensure the quality of the model’s output and to support explainability, transparency, and interoperability [3, 4, 5]. Additionally, data security and privacy need to be taken into account, such as gaining participants’ consent, anonymizing or de-identifying data, and ensuring that it is securely kept and managed. Data curation and intake are critical components of developing Large Language Models (LLMs) that can efectively understand and respond to human language [6]. This is especially crucial to allow for diversity in terms of the languages and topics covered [7].

Recognizing possible biases in the data curation and ingestion processes and taking proactive steps to remove them is critical. Data security, ethics, and privacy must be considered, especially when working with sensitive data. It is critical that we protect people’s privacy rights and prevent data exploitation [8].

2.2. Combination of exogenous and endogenous semantics

This research task involves a combination of exogenous and endogenous semantics, utilizing multimodal embeddings and generative tasks through attentive, selfsupervised training and large-scale-based transfer learning [9]. Exogenous semantics refers to external information that is used to train the model, while endogenous semantics refers to internal information that is subsequently learned by the model from the training data. Multidimensional embeddings involve representing words and sentences using multiple modalities, allowing the model to capture diferent facets of language, leading to improved performance [ 10 ]. Linguistic knowledge can be added into embeddings to increase the representations’ quality [ 11 ]. This can lead to better results in tasks like

2.3. Training, fine-tuning, and prompting

of Large Language Understanding

Models

The study of strategies for training, fine-tuning, and motivating Large Language Models is part of this research task. It is critical to train adaptive LLMs, which can learn and alter their knowledge depending on new data and feedback [19, 6]. This must be done while keeping multimodality in mind. The task goal is to solve this issue and examine options that may be available for less frequently spoken languages. Fine-tuning and prompting are two approaches often employed in Big Language Understanding, a subset of Natural Language Processing. These strategies are used to boost the performance of pre-trained language models like GPT-3 for specific tasks or domains. Fine-tuning entails picking a smaller dataset relevant to the job and then training the model on it. Except for the last layer, the weights of the pre-trained model usually are frozen. Fine-tuning can help with various NLP tasks, including sentiment analysis, question answering, and text categorization. Prompting, on the other hand, is sending a specific prompt or input to a pre-trained language model in order for it to respond. The prompt might be a single brief word or sentence that ofers context for the model to respond to. The purpose of prompting is to increase the model’s relevance and accuracy. Task-specific prompts can be created for writing a news story summary or answering a specific question.

2.4. Exploiting LLMs for intelligent information access

LLMs may be used to improve a variety of informationaccess applications, including information extraction, question answering, information search and seeking discovery, decision-making, and recommendation. One way of extracting information from large language models is named entity recognition (NER) [ 20 ], while another is relationship extraction (RE). Large language models have into ways to automatically map qualities that represent shown tremendous potential in the field of information preferences and requirements to KG components. In extraction and are expected to play an increasingly impor- this approach, reasoning and learning mechanisms may tant role in automating structured information extraction. be used to infer more general and accurate information. Large language models, such as GPT-3 and BERT, have Lastly, LLMs may be utilized to describe preferences and revolutionized quality assurance by achieving cutting- requirements accurately. Conversational data, KGs, and edge performance on a wide range of benchmarks and LLMs can represent information sources. In terms of real-world applications. They can also be used with other conversational data, the massive volume of information tools and technologies to create more robust information already accessible may be utilized to forecast more acsearch and retrieval systems. Large language models can curate information about the user [23, 24]. For instance, aid in discovering new information and insights from the psychological attributes of the user (i.e., personality) data. They can aid in identifying patterns and insights can be learned and inferred from such a data point [13]. that traditional data analysis methods may overlook, pro- There are now various ways to predict such qualities auviding tailored suggestions and insights to help users tomatically, but the concept of analyzing conversational make decisions, and assisting decision-makers in making data for these purposes remains unexplored. Similarly, intelligent and data-driven decisions [21]. They may also sources such as KGs and LLMs might be valuable for be used in finance to analyze market data and estimate learning about the characteristics of users. For example, stock prices or investment opportunities, in healthcare by using a user’s preferences and requirements stored as to review patient data, and in natural language process- an embedding, it is feasible to fine-tune models that allow ing applications to evaluate consumer comments, social reliable prediction of such attributes [25]. All information media postings, and other forms of communication [22]. about what users desire, as well as their characteristics, will be utilized to give customized information access to 2.5. Functional connectivity patterns them. In this situation, conversational data will also be crucial in learning new strategies and models [26, 27, 28].

This project aims to identify diferent functional connectivity patterns associated with diferent aspects of 2.7. Data preprocessing with higher-order cognitive functions, with particular attention to the diferent stages of memory processing. During outlier/anomaly detection functional magnetic resonance imaging (fMRI) scanning, Outliers are frequently caused by measurement mistakes participants will perform a memory task between two or data corruption, but in some instances, the source of resting state sessions. To investigate whether diferent the outliers is unknown. During the statistical analysis participants recruit diferent thalamic hubs as a function of the dataset, a robust automated approach for detecting of memory performance, the percentage of overlap be- outliers turns out to be critical. Novel techniques based tween each individually-segmented thalamic subdivision on statistical regression processes and using the idea of will be extracted from individual spatial maps of a fron- entropy to detect outliers contextually when fitting a toparietal network. Multiple support vector regression model to data will be addressed. The approach, already models will be used to validate and extend the method developed for polynomials [29], will be extended to the to reach the goal to whole-brain functional connectivity B-spline basis and nonlinear models and will be used to beyond the thalamus and to other cognitive functions remove noise from signals generated either synthetically beyond memory. or by specific sources of interest.

Noise reduction, imputing, and smoothing of long time 2.6. User Modeling series will also be accomplished by using numerical approaches based on B-spline quasi-interpolation [ 30 ]. In In this task, we will create techniques for automatically general, quasi-interpolation refers to a method for coneliciting information about users’ preferences and re- structing eficient local approximants to a given data quirements from disparate data sources. Initially, we will collection or function. There are two primary reasons look into how such data points may be extracted from for using quasi-interpolation. In practice, data acquired conversational data. This data, often available in nat- from natural settings are frequently contaminated by ural language format, is essential for modeling what a mistakes. Hence, interpolation methods may be overly user wants and requires when interacting with an intelli- rigorous, in addition to sufering from the overfitting gent system. Unfortunately, obtaining this information issue. Furthermore, since quasi-interpolation is based on is not easy. Hence approaches for exact elicitation are local construction, the computational cost is significantly becoming increasingly important. Similarly, we will look lower when compared to global alternatives such as ininto how Knowledge Graphs (KGs) and LLMs might be terpolation. The same methodologies will be examined, used for this purpose [ 10 ]. Regarding KGs, we will look together with statistical methods, for a novel anomaly detection strategy in massive datasets. This data pre- and computer vision. Deep neural networks’ excellent processing phase is critical because it enables the use of function approximation capabilities are at the root of higher-quality data as input to the algorithms that will these achievements. Modeling sequential data is a difibe built. cult challenge in NLP applications [38, 39]. Much work has been done to overcome the challenge of sequential 2.8. Matrix and tensor decomposition data modeling. Because of its recurrent nature, recurrent neural networks (RNNs) outperform the others. NotwithThe frontier of machine learning and deep learning in standing the capabilities of the RNNs, training them is a Symbiotic-AI is to evaluate enormous amounts of data challenging task. When the input sequence is large, the from many sources, such as text, photos, and signals multiplication term generated by the chain derivation from the IoT environment, to construct models capable method is numerically unstable, causing RNN to sufer of assisting human understanding, providing answers, from gradient explosion and gradient vanishing. To train knowledge, and synthesis. Nevertheless, sustaining such the model, backpropagation through time (BPTT) is remodels is too expensive in terms of computational and quired. In this challenge, we will investigate a novel data storage costs. Therefore, it remains fundamental family of deep neural networks that entail the usage of to use techniques capable of extracting information and Ordinary Diferential Equations (ODEs) inspired by the patterns from diferent types of data by reducing their relationship between RNNs and ODEs (ODE). Specific dimensionality while preserving the most important in- models in the ODE class are worth considering, such as formation. This is in the sense of learning closer to the delay ordinary diferential equations [ 40] and integrodhuman, which requires little information to learn, build iferential equations. Such delayed dynamics are signifnew knowledge, or provide answers. Within this task, icant in characterizing emergent phenomena in many new Matrix and Tensor decomposition techniques based real-world systems, including physical, chemical, ecologon SVD will be investigated [31, 32, 33]. This is essential ical, and physiological systems. This study area will be to obtain data in a low dimensional space that contains examined in conjunction with data representation utithe salient information discarding those redundant or lizing tensors and multiway arrays that account for the those that don’t contain crucial information. These tech- data structure. Moreover, the usage of decompositions niques turn out to be an important step when used for to address data dimensionality will be discussed. supervised and unsupervised learning processes. Indeed, numerous works in this field have shown that matrix and applied tensor decomposition in the task of classifica- 3. Conclusion tion, clustering, change detection, or salience detection algorithms are able to increase performance, not only from a computational point of view but also from the expected results [34, 35]. Lately, the Tensor and Matrix decomposition have been also used as a generative model to obtain synthetic data with the same characteristics as the real data. This is another aspect that can be taken into account, especially when there is a need to form a model, and there is not enough real data available.

The FAIR approach is comprehensive and multidisciplinary, with the goal of profoundly rethinking the foundations of AI while also studying the societal consequences of emerging kinds of AI. Depending on the path that the AI revolution takes, AI will either empower or reduce human agency; expand or replace the human experience; create new forms of human activity or reduce jobs; help distribute well-being for many or increase the concentration of power and wealth in the hands of a few; expand or endanger democracy in our societies; help fight climate change or increase emissions. FAIR researchers want AI to be part of the solution, not the issue, to the world’s social, financial, ethical, and environmental concerns. The merging of human intelligence with artificial intelligence (AI) to improve both capabilities is thought of as Symbiotic-AI. This form of collaboration has the potential to forge strong alliances by allowing people to profit from AI’s speed, eficiency, and processing capacity while giving machines human-like judgment, reasoning, and creativity.

2.9. Explore NN with a new family of deep neural network models

In the last decades, a wide range of neural networks has been proposed to address challenges in deep learning processes, achieving great success in a wide spectrum of applications. Despite the prominent success achieved by the neural network, this approach still lacks theoretical direction for designing an efective neural network model, and evaluating the performance of a model requires unnecessary resources. Recent research has shown that many existing models may be thought of as various numerical discretizations of diferential equations [36, 37]. Deep learning is also making rapid progress in natural language processing (NLP), speech recognition,

4. Acknowledgements

pp. 135–149. sition, Signal Processing 198 (2022) 108575. [ 20 ] M. Polignano, M. de Gemmis, G. Semeraro, Compar- URL: https://www.sciencedirect.com/science/ ing transformer-based ner approaches for analysing article/pii/S0165168422001128. doi:https: textual medical diagnoses., in: CLEF (Working //doi.org/10.1016/j.sigpro.2022.108575.

Notes), 2021, pp. 818–833. [32] L. Lathauwer, B. De Moor, A multi-linear singular [21] M. Polignano, V. Suriano, P. Lops, M. de Gemmis, value decomposition, Society for Industrial and G. Semeraro, A study of machine learning models Applied Mathematics 21 (2000) 1253–1278. doi:10. for clinical coding of medical reports at codiesp 1137/S0895479896305696.

2020., CLEF (Working Notes) 640 (2020). [33] T. G. Kolda, B. W. Bader, Tensor decomposi[22] P. Basile, D. Croce, V. Basile, M. Polignano, et al., tions and applications, SIAM Review 51 (2009) Overview of the evalita 2018 aspect-based senti- 455–500. URL: https://doi.org/10.1137/07070111X. ment analysis task (absita), in: EVALITA Evalu- doi:10.1137/07070111X. ation of NLP and Speech Tools for Italian, CEUR, [34] A. Falini, G. Castellano, C. Tamborrino, F. Mazzia, 2018, pp. 1–10. R. M. Mininni, A. Appice, D. Malerba, Saliency [23] C. Musto, M. Polignano, G. Semeraro, M. de Gem- detection for hyperspectral images via sparse-non mis, P. Lops, Myrror: a platform for holistic negative-matrix-factorization and novel distance user modeling: Merging data from social networks, measures, in: 2020 IEEE Conference on Evolving smartphones and wearable devices, User Modeling and Adaptive Intelligent Systems (EAIS), 2020, pp. and User-Adapted Interaction 30 (2020) 477–511. 1–8. doi:10.1109/EAIS48028.2020.9122749. [24] C. Musto, G. Semeraro, M. Polignano, A comparison [35] C. Tamborrino, F. Mazzia, Classification of hyperof lexicon-based approaches for sentiment analysis spectral images with copulas, Journal of Compuof microblog posts., in: DART@ AI* IA, Citeseer, tational Mathematics and Data Science 6 (2023) 2014, pp. 59–68. 100070. URL: https://www.sciencedirect.com/ [25] C. Musto, F. Narducci, M. Polignano, M. de Gem- science/article/pii/S277241582200030X. doi:https: mis, P. Lops, G. Semeraro, Towards queryable user //doi.org/10.1016/j.jcmds.2022.100070. profiles: Introducing conversational agents in a [36] P. Kidger, On neural diferential equations, 2022. platform for holistic user modeling, in: Adjunct arXiv:2202.02435. publication of the 28th ACM conference on user [37] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, D. K. modeling, adaptation and personalization, 2020, pp. Duvenaud, Neural ordinary diferential equations, 213–218. in: S. Bengio, H. Wallach, H. Larochelle, K. Grau[26] C. Musto, F. Narducci, M. Polignano, M. De Gem- man, N. Cesa-Bianchi, R. Garnett (Eds.), Advances mis, P. Lops, G. Semeraro, Myrrorbot: A digital in Neural Information Processing Systems, volassistant based on holistic user models for person- ume 31, Curran Associates, Inc., 2018. URL: https: alized access to online services, ACM Transactions //proceedings.neurips.cc/paper_files/paper/2018/ on Information Systems (TOIS) 39 (2021) 1–34. file/69386f6bb1dfed68692a24c8686939b9-Paper.pdf. [27] M. Polignano, P. Lops, M. de Gemmis, G. Semeraro, [38] M. Liu, W. Luo, Z. Cai, X. Du, J. Zhang, S. Li, NumerHelena: An intelligent digital assistant based on a ical‐discrete‐scheme‐incorporated recurrent neulifelong health user model, Information Processing ral network for tasks in natural language process& Management 60 (2023) 103124. ing, CAAI Transactions on Intelligence Technology [28] M. Polignano, F. Narducci, A. Iovine, C. Musto, (2023) n/a–n/a. doi:10.1049/cit2.12172.

M. De Gemmis, G. Semeraro, Healthassistantbot: a [39] B. Li, Q. Du, T. Zhou, Y. Jing, S. Zhou, X. Zeng, personal health assistant for the italian language, T. Xiao, J. Zhu, X. Liu, M. Zhang, Ode transIEEE Access 8 (2020) 107479–107497. former: An ordinary diferential equation-inspired [29] D. Giordano, F. Iavernaro, Maximal-entropy driven model for sequence generation, 2022, pp. 8335–8351. determination of weights in least-square approxi- doi:10.18653/v1/2022.acl- long.571. mation, Mathematical Methods in the Applied Sci- [40] F. Stelzer, S. Yanchuk, Emulating complex networks ences 44 (2021) 6448 – 6461. doi:10.1002/mma.7197. with a single delay diferential equation, The Eu[ 30 ] A. Falini, F. Mazzia, C. Tamborrino, Spline ropean Physical Journal Special Topics 230 (2021). based hermite quasi-interpolation for univariate doi:10.1140/epjs/s11734- 021- 00162- 5. time series, Discrete and Continuous Dynamical Systems - S 15 (2022) 3667–3688. URL: /article/ id/621c8f822d80b7479e4357ab. doi:10.3934/dcdss.

2022039. [31] M. Vandecappelle, L. De Lathauwer, From multilinear svd to multilinear utv decompo

References

30th ACM Conference on User Modeling , Adapta-

tion and Personalization , 2022 , pp. 354 - 356 . We acknowledge the support of the PNRR project FAIR - [10] M. Polignano , C. Musto , M. de Gemmis, P. Lops, Future AI Research (PE00000013) , Spoke 6 - Symbiotic AI G . Semeraro, Together is better: Hybrid recommen(CUP H97G22000210007) under the NRRP MUR program dations combining graph embeddings and contextufunded by the NextGenerationEU. alized word representations , in: Proceedings of the

15th ACM Conference on Recommender Systems, References 2021 , pp. 187 - 198 .

[11] D. Nozza , L. Passaro , M. Polignano , et al., Preface [1] P. Brusilovsky , M. De Gemmis , A. Felfernig , P. Lops, to the sixth workshop on natural language for ar-

M. Polignano , G. Semeraro, M. C. Willemsen , Joint tificial intelligence (nl4ai) , in: CEUR Workshop

workshop on interfaces and human decision mak- Proceedings , 2022 .

ing for recommender systems (intrs'22) , in: Pro- [12] M. Polignano , P. Basile , M. de Gemmis, G. Semer-

ceedings of the 16th ACM Conference on Recom- aro, A comparison of word-embeddings in emo-

mender Systems , 2022 , pp. 667 - 670 . tion detection from text using bilstm, cnn and self[2] C. Musto , N. Tintarev , O. Inel , M. Polignano , G. Se - attention, in: Adjunct Publication of the 27th Con-

models and personalized systems (exum 2021 ), in: alization, 2019 , pp. 63 - 68 .

Adjunct Proceedings of the 29th ACM Conference [13] M. Polignano , F. Narducci , M. de Gemmis, G. Se-

2021 , pp. 211 - 212 . systems: an afective coherence model based on [3] C. Musto , A. Delic , O. Inel , M. Polignano , A. Rapp, emotion-driven behaviors, Expert Systems with

G. Semeraro , J. Ziegler, Workshop on explainable Applications 170 ( 2021 ) 114382 .

user models and personalised systems (exum) , in: [14] M. Polignano , P. Basile , M. De Gemmis , G. Semer-

Adjunct Proceedings of the 30th ACM Conference aro , Hate speech detection through alberto italian

on User Modeling, Adaptation and Personalization, language understanding model ., in: NL4AI@ AI*

2022 , pp. 160 - 162 . IA, 2019 , pp. 1 - 13 . [4] M. Polignano , V. Basile , P. Basile , G. Gabrieli, M. Vas- [15] M. Polignano , M. de Gemmis , F. Narducci , G. Semer-

approach for explainable polarity detection, Infor- from social media , in: AI* IA 2017 Advances in Arti-

mation Processing & Management 59 ( 2022 ) 103058 . ifcial Intelligence: XVIth International Conference [5] M. Polignano , G. Colavito, C. Musto , M. de Gemmis, of the Italian Association for Artificial Intelligence,

G. Semeraro , Lexicon enriched hybrid hate speech Bari , Italy, November 14-17 , 2017 , Proceedings 16,

detection with human-centered explanations , in: Springer, 2017 , pp. 321 - 333 .

Adjunct Proceedings of the 30th ACM Conference [16] M. Polignano , P. Basile , G. Rossiello, M. de Gemmis,

2022 , pp. 184 - 191 . social media footprints , in: Proceedings of the [6] K. Florio , V. Basile , M. Polignano , P. Basile , V. Patti , 25th conference on user modeling, adaptation and

Time of your hate: The challenge of time in hate personalization , 2017 , pp. 383 - 384 .

speech detection on social media , Applied Sciences [17] M. Polignano , P. Basile , Hansel: Italian hate speech

10 ( 2020 ) 4180. detection through ensemble learning and deep neu[7] M. Polignano , P. Basile , M. De Gemmis , G. Semeraro, ral networks, EVALITA Evaluation of NLP and

V. Basile , et al., Alberto: Italian bert language under- Speech Tools for Italian 12 ( 2018 ) 224 .

standing model for nlp challenging tasks based on [18] M. G. de Pinto , M. Polignano , P. Lops , G. Semer-

2481, CEUR , 2019 , pp. 1 - 6 . ken language using deep neural networks and mel[8] F. Lorè , P. Basile , A. Appice , M. de Gemmis, frequency cepstral coeficients , in: 2020 IEEE con-

port decisions on gdpr compliance , Journal of In- tems (EAIS) , IEEE, 2020 , pp. 1 - 5 .

telligent Information Systems ( 2023 ) 1 - 28 . [19] M. Polignano , M. de Gemmis, G. Semeraro, Con[9] P. Lops , C. Musto , M. Polignano , Semantics-aware textualized bert sentence embeddings for author

mender systems (score) , in: Proceedings of the tational Science and Its Applications-ICCSA 2020 :

20th International Conference, Cagliari, Italy, July

1- 4 , 2020 , Proceedings, Part IV 20 , Springer, 2020 ,