Cortical Activity of Relevance Zuzana Pinkosova, Yashar Moshfeghi University of Strathclyde, Glasgow, UK {zuzana.pinkosova,yashar.moshfeghi}@strath.ac.uk, Abstract. Many theoretical approaches in information retrieval assume that relevance is based on mutual interaction of the system and user. Past studies have mainly focused on the system side of relevance, while user- centred studies are more recent. As a result, this work aims to focus on user relevance, which is characterised as a subjective process, dependant on the specific user mind state [19]. To gain a better insight into the nature of this internal and subjective process, it is crucial to examine the underlying behavioural, physiological and psychological mechanisms involved [1]. With the development of brain imaging, new research has begun to investigate user relevance by analysing neural brain activity. However, despite the available research, different strata of relevance pro- posed by Saracevic (1997), have not yet been investigated in terms of neuroscience. A better understanding of relevance is an important step towards improving personalisation in the information retrieval process. Keywords: relevance, EEG, information retrieval, information process- ing, cognitive relevance 1 Introduction The main goal of the information retrieval (IR) systems is to retrieve relevant information or information units that would help users to satisfy their informa- tion need and to achieve the search task goal [7]. Relevance is a central notion in the IR [9] and it plays a key role in the user-system interaction. Additionally, relevance is an important indicator of IR systems effectiveness and performance [15, 16]. However, despite the significance and importance of this concept [17], rel- evance is still not completely understood [9]. In addition, relevance is difficult to define [17] and the terminology has not been consistent. Different authors assigned different meanings to the concept [10], aiming to develop an ideal and widely accepted relevance model. Nevertheless, up to date, a universal theory of relevance does not exist. Instead, many competing theories and models have been Copyright c 2019 for this paper by its authors. Use permitted under Creative Com- mons License Attribution 4.0 International (CC BY 4.0). FDIA 2019, 17-18 July 2019, Milan, Italy. proposed, involving different relevance criteria and several distinct components [13, 20]. The main objective of this work is to contribute to the empirical evidence associated with Saracevic’s model and to increase the understanding of relevance. This work will focus on a single theory to investigate relevance while maintaining conceptual consistency during empirical evaluation. Since Saracevic’s stratified model of relevance has been identified as a framework which is complex enough to consider all significant relevance aspects, yet flexible and abstract enough to be empirically tested an applied [20]. Also, his model proposes that the information retrieval process results from a set of interactions between user and system. Thus, both user and system are represented by a set of layers that are interdependent [15], allowing us to focus on the user aspect of relevant within this model. The rest of this paper is organised as follows. First, we describe the back- ground in Section 2, which outlines the concept of relevance and related work in the area of neuroscience.Section 3 discusses the methodological approach this work aims to employ. The current stage of the PhD progress and next planned steps will be also explained in this section. Finally, Section 4 presents key con- clusion and potential implications of this work. 2 Background This section outlines Saracevic’s stratified model. It has been argued that rel- evance depends on the users subjective judgement [17] and hence it might be difficult to measure. Hence, this section will address how the employment of brain imaging techniques helped to tackle this problem and the findings of previous studies investigating users neural processes during relevance judgement will be summarised. 2.1 The Concept of Relevance According to Saracevic’s stratified model of relevance, IR is seen as an interaction between several layers or strata through an interface at a surface level. Relevance is therefore derived as a result of interaction among these strata [14–16]. Within this model, there are two main elements a user and a system. The user usually expresses the subjective information need (IN) through query formulation. The system then presents the user with retrieved information (system relevance), which then users interprets and relate to the problem at hand, cognitive state, and other aspects. In other words, the user retrieves information based on sub- jective relevance criteria (user relevance). According to Saracevic (1997), both, user and system side consists of several levels. The user side, which is the main interest of this work, is composed of cognitive, affective and situational level [14]. However, it is important to note that one of the main limitations of this model is that the model is not detailed enough for experimentation and verification [15]. However, Weigl and Guastavino in [20] discussed models potential application and usefulness in user-centred music information retrieval research. Investigating the role of different strata during relevance judgement consti- tutes a complementary and promising technique which can enhance the under- standing of this complex process. This work aims to do so through the employ- ment of neuroscientific approach. The study is based on the premise that user relevance is inter-subjective, systematic and measurable in its nature, as pro- posed by Schamber and Eisenberg in [18]. The above-mentioned premise has been recently supported by empirical evidence of previous studies investigat- ing relevance through the measurement of physiological signals. Results of these studies suggest that overall, physiological signals significantly differ during pro- cessing relevant content non-relevant content across individuals. Employing such an approach helped to provide valid insight into the relevance judgement process and to overcome the self-referential nature of direct and obtrusive methods, but still having an ability to focus on internal mental states of an individual. In addi- tion, past studies investigating this phenomenon have benefited from employing knowledge from multiple disciplines, such as neuroscience, computer science, and psychology, which we seek to implement in the present study. Relevance Feedback: The area of research interested in investigating relevance has a long theoretical background [15]. While system-oriented empirical research in this area is well established, examining the users internal processes happening during relevance judgement is relatively recent [11],[6]. Past research investigat- ing relevance relied on filtering relevant from non-relevant information through relevance assessment and selection process from users when examining specific information presented by the search system. The selection process is therefore complex, involving a series of interactions of various components [15], which is also known as the relevance feedback cycle. The relevance feedback cycle is an indicator of perceived relevance and can be based either on explicit or/and implicit feedback [1]. Relevance & Brain Imaging: Recently, with the development of brain imaging techniques, new research begun to investigate relevance analysing neural activ- ity in the brain. The earliest research conducted by Allegretti and colleagues in [1] using brain imaging revealed that neural signatures detectable with an elec- troencephalogram (EEG) along with other physiological signals could be used as a reliable indicator of relevance in real-time. In order to investigate how does relevance happen in the brain, Moshfeghi and colleagues [11] employed functional magnetic resonance (fMRI) technique, to localise the neural activity differences in the brain while processing of rele- vant and non-relevant information. The study found that the differences in brain activity are the greatest in 3 regions in the frontal, parietal and temporal cor- tex. Later, they found that brain regions playing a crucial role during relevance judgement are the inferior parietal lobe, inferior temporal gyrus, and superior frontal gyrus and their increased activation for relevant items were related to visuospatial working memory [12]. Another research conducted by Frey and colleagues in [5] found that postrel- evance judgement brain wave differences in processing relevant and irrelevant words that persisted for one word after a relevant word (from approximately 260 to 320 ms) and two words after an irrelevant word (from approximately 500 to 530 ms). Using EEG has become a popular tool in order to study relevance and researchers attempted to employ this tool to make information retrieval process even more effective. Eugster and colleagues provided further evidence that EEG is a valid tool to study relevance and moreover found that EEG signals can be used to automatically predict relevance [4]. They found that peak significant difference between processing relevant and non-relevant words was detected in Pz channel after 450 ms, maximising at 747 ms. Later, Eugster and colleagues [3] introduced brain-relevance paradigm which enables recommendation of in- formation without any explicit user interaction based on EEG signals evoked by users’ interests toward digital content. Allegretti and colleagues [1] used EEG in order to identify time intervals and brain activity shifting during relevance and non-relevance processing. They identified 3 time intervals: 180 - 300 ms an early process of implicit judgements of relevance (frontal areas F1; AF4) and stimuli processing. At this stage, there is no relevance judgement. Between 300 500 ms activity is shifted towards central areas C2 and CP2. During 500 800 ms, the most significant differences can be observed between the processing of relevant and non-relevant. They found that the region of interest is located in the center of the scalp - Cz, C1. In addition, Gwizdka [6] found significant differences in EEG-measured power of alpha frequency band and in EEG-detected attention levels during relevance judgement. 3 Approach Recent findings employing brain imaging to investigate relevance have shown that human mental experiences during information retrieval process can be un- derstood and accurately decoded using non-invasive measurements of the brain activity. Hence, recent application of neuroscientific approach has brought valid and valuable insight into better understanding of relevance. In addition, since relevance is a complex process, it is important to highlight the benefit of com- bining multiple data collection tools, which has become very popular in recent years. As Kelly and Belkin suggested in [8], tools such as questionnaires enable researchers to explore participant views of a task and topic familiarity, which in- fluence relevance perception. In addition, the authors highlighted the importance of the naturalistic approach, which optimises ecological validity [8]. It is essential to design the task, which will closely model real-life user-system interaction and place relevance judgement within the context of information retrieval. Current Work In the first study, we aim to explore aspects of cognitive rele- vance through the examination of the users physiological and behavioural signals. These signals will be obtained through naturalistic tasks designed for this pur- pose, placing cognitive relevance within the context of the information retrieval process and aiming to incorporate all its aspects, such as information need. The study will be built on the previous literature investigating relevance through the comparison of signals associated with relevant and non-relevant information [1, 4, 3]. An in-depth understanding of cognitive relevance might not only improve the understanding of the relevance process, but it can also improve user-system interaction and result in greater search success. If the level of users cognitive abilities is low and task difficulty is high, the user might be unable to effectively interact with retrieved information and as a result, the problem solving may fail to occur [2]. Next Steps Since relevance is a complex mental phenomenon, it is essential to also consider the underlying perceptual and cognitive processes. To do so, as men- tioned in Section 3, this work will aim to gather physiological and behavioural data in order to better understand participant’s experience during relevance judgement tasks. Also, as a future direction of this work, we aim to go beyond cognitive relevance and investigate other relevance strata, as outlined by Sarace- vic [14], such as situational and affective relevance through the employment of brain imaging techniques. 4 Conclusion Our understanding of relevance is still not complete, and thus there is a need to further investigate this key concept in IR. In this work, we aim to investigate the the concept of relevance from a neuropsychological perspective. In particular the work will focus on Saracevic’s stratified model of relevance through the employment of brain imaging techniques. Further understanding of neurological properties of relevance might provide valuable insight into personalisation within information retrieval [8]. This could also lead to a significant contribution to the improvement of information systems [13]. Additionally, empirical investigation of different relevance strata might help to provide scientific evidence to validate Saracevics theoretical concept of relevance. References 1. Allegretti, M., Moshfeghi, Y., Hadjigeorgieva, M., Pollick, F.E., Jose, J.M., Pasi, G.: When Relevance Judgement is Happening? Proceedings of the 38th Interna- tional ACM SIGIR Conference on Research and Development in Information Re- trieval - SIGIR ’15 pp. 719–722 (2015) 2. Back, J., Oppenheim, C.: A model of cognitive load for IR: Implications for user relevance feedback interaction. Information Research 6(2) (2001) 3. Eugster, M.J.A., Ruotsalo, T., Spapé, M.M., Barral, O., Ravaja, N., Jacucci, G., Kaski, S.: Natural brain-information interfaces: Recommending information by rel- evance inferred from human brain signals. Scientific Reports 6(11), 1–10 (2016) 4. Eugster, M.J., Ruotsalo, T., Spapé, M.M., Kosunen, I., Barral, O., Ravaja, N., Jacucci, G., Kaski, S.: Predicting term-relevance from brain signals. In: Proceedings of the 37th international ACM SIGIR conference on Research \& development in information retrieval. pp. 425–434 (2014) 5. Frey, A., Ionescu, G., Lemaire, B., López-Orozco, F., Baccino, T., Guérin-Dugué, A.: Decision-making in information seeking on texts: an eye-fixation-related poten- tials investigation. Frontiers in Systems Neuroscience 7(8), 1–22 (2013) 6. Gwizdka, J., Hosseini, R., Cole, M., Wang, S.: Temporal Dynamics of Eye-Tracking and EEG During Reading and Relevance Decisions. Journal of the American So- ciety for Information Science 68(10), 2299–2312 (2017) 7. Kagolovsky, Y., Möhr, J.R.: A new approach to the concept of ’relevance’ in infor- mation retrieval (IR). Studies in Health Technology and Informatics 84, 348–352 (2001) 8. Kelly, D., Belkin, N.J.: Display time as implicit feedback. In: Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval. p. 377. No. 1 (2004) 9. Mizzaro, S.: The whole history of relevance. Journal of the American Society for Information Science 49(4), 392–392 (1997) 10. Mizzaro, S.: How many relevances in information retrieval? Interacting with Com- puters 10(3), 303–320 (1998) 11. Moshfeghi, Y., Pinto, L.R., Pollick, F.E., Jose, J.M.: Understanding relevance: An fMRI study. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 7814 LNCS, 14–25 (2013) 12. Moshfeghi, Y., Triantafillou, P., Pollick, F.: Understanding Information Need: An fMRI Study An fMRI Study. ACM SIGIR conference on Research and Develop- ment in Information Retrieval pp. 335–344 (2016) 13. Ruthven, I., Kelly, D.: Interactive Information Seeking, Behaviour and Retrieval. Facet Publishing, London (2011) 14. Saracevic, T.: The stratified model of information retrieval interaction: Extension and applications. Proceedings of the Annual Meeting-American Society for Infor- mation Science 34, 313–327 (1997) 15. Saracevic, T.: Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance. Journal of the American Society for Information Science and Technology 58(13), 1915–1933 (2007) 16. Saracevic, T.: The Notion of Relevance in Information Science: Everybody knows what relevance is. But, what is it really? Tech. rep. (2016) 17. Schamber, L.: Relevance and Information Behavior. Annual review of information science and technology (ARIST) 29, 3–48 (1994) 18. Schamber, L., Eisenberg, M.: Relevance: The Search for a Definition. In: Proceed- ings of the 51st annual meeting of the American Society for Information Science. pp. 164–168 (1988) 19. Swanson, D.R.: Subjective versus objective relevance in bibliographic retrieval sys- tems. The Library Quarterly 56(4), 389–398 (1986) 20. Weigl, D.M., Guastavino, C.: Applying the stratified model of relevance interac- tions to music information retrieval. Proceedings of the ASIST Annual Meeting 50(1) (2013)