Developing a N400 Brain Computer Interface based on semantic expectancy

Abstract

In this study, we present a new application to the study of the N400 related event component applied to the Brain Computer Interfaces (BCI) field. The N400 is classically defined in literature as an index of semantic integration mechanisms and it is sensitive to the difficulty with which the reader integrates the input within the semantic context, based on their expectations. By varying the level of violation of expectations in the semantic context and presenting sentences lacking the final word (cloze probability test) we want to train a classifier so that it can always complete the sentences in accordance with the expectations of the participant. The online classification is based on the average peak differences in three different conditions (target, semantically related and unrelated), where the amplitude of the N400 should correlate with the progressive and greater violation of semantic expectation. The findings can contribute significantly to this area of research that is still left with several unanswered questions as this research is one of the first to exploit the N400 in an online experiment.

Keywords

• brain-computer interfacing
• electroencephalography
• N400
• humancomputer interaction
• reading
• expectancy
• language
• communication
• semantics

-

A brain-computer interface (BCI) provides a direct connection between the brain and an external device, translating brain signals into commands for electronic devices [ 1 ]. It is a communication system that does not use the normal brain output channels such as peripheral nerves and muscles, but is able to recognize and adapt the mind of the individual.

In the most common sense it is a device capable of monitoring the activity of the user and use certain signals to interpret and enforce their will. Some BCI methods of recording, like MEG (magnetoencephalography), detect the generated magnetic fields from electric currents in the brain; others use functional magnetic resonance imaging, fMRI, and others still use the near infrared spectroscopy, NIRS, to visualize the activity of the bloodstream of the brain. MEG or fMRI devices are bulky and expensive, thus limiting their applications to specialized environments [ 2 ]. NIRSs are relatively smaller and less expensive, they are based on dynamic flow response that crosses the brain and nearby tissues, but it requires very long analysis and not suitable for realtime applications as it does demand in a BCI system [ 3 ]. For these and other reasons, BCI research has focused on bioelectric signals recorded by methods of electroencephalography (EEG). Given the easy availability and easy use of the EEG method, most of the research has focused on analysing and deepening this recording technique.

The EEG has a good timing resolution and provides immediate feedback, with delays to be included in the order of milliseconds. As far as spatial resolution is concerned, it tends to be more approximate with a precision of about 2-3cm [ 3 ]. This type of system focuses on decoding and classifying signals derived from brain activity in order to provide controls for managing various applications, to promote communication or complete daily tasks as in the specific case of patients suffering from of locked-in syndrome (LIS) and amyotrophic lateral sclerosis (ALS) [ 5 ].

Research conducted over the past few decades has allowed to distinguish multiple approaches that allow to adopt different characteristics of the brain signal detected through the EEG. Such applications rely on modulation of frequency/amplitude composition of EEG tracks independently, which is expected to be the result of the training. In this context, the most frequently used signals are sensory motor rhythms and slow cortical potentials. Sensory motor rhythms are associated with cortical areas directly related to the control of motor networks [ 6 ]. These rhythms include a range of oscillations between 8 ÷ 12 Hz and 15 ÷ 32 [ 7,8 ]. However, the most commonly investigated rates in the BCI range are the rhythm between 8 ÷ 12 Hz and the beta rhythm between 18 ÷ 26 Hz [ 9 ] . Slow cortical potentials (SCPs), on the other hand, represent another characteristic of the EEG (in the frequency band below 1 Hz) which can be voluntarily modified after a training period of the participant [ 10, 11, 12 ].

During this training, the user learns from a display both the polarity and the amplitude of the SCPs he is producing on his scalp. In this way, it is possible for a subject to become aware of the psychological state that induces changes in these potentials and appropriately utilize that psychological state to induce the desired variations on the produced EEG [ 13 ]. Another approach adopted by the BCI for the detection of distinctive characteristics of the brain signal involves its modulation following the presentation of external stimuli.

This type of EEG signal is named Event-Related Potential (ERP) and the applications developed for the work of this study are based precisely on this variant of cortical activity. ERPs manifest themselves in conjunction with the activation of specific cognitive processes by the subject, such as decisions-making, or shifting attention from one stimulus to another in the external environment [ 14 ]. These potentials are dependent on the information content of the stimulus and appear only when the subject cares about the latter and attributes it a “meaning” [ 15 ]. A peculiarity of these potentials relates to the temporal relationship between stimulation and brain electrical response to the stimulus itself. An ERP can be seen as a variation of cortical electrical potential from the background activity to the one induced by the external event, which takes place at a fixed distance over time with respect to the event of interest.

These electrical events are distinguished by their duration, some hundreds of milliseconds, and by their magnitude, of a few microvolts [ 14 ] and consist of waveforms characterized by positive or negative polarity deflections [ 15 ]. A component must appear or not (or change) when the same stimulus is presented in different cognitive contexts or presentation modalities. Only in these cases, we can have the certainty about the endogenous nature of a component and that it reflects the neural processes associated to the cognitive activation induced in a particular psychological context [ 16 ]. This study focuses on the analysis of a particular electroencephalographic component occurring at 400 ms after stimulus presentation in the event of an inconsistency in the type of event being proposed, defined as N400.

More in detail, the N400 was identified by Kutas and Hillyard in 1980 [ 17 ] in a semantic paradigm in which the words of a sentence are presented below text form one by one at regular intervals. They noticed that inserting as final words of the sentence, two terms not congruent from the point of view of meaning (and not of syntax) with respect to the rest, instead of a form of wavelengths between 200 and 600 ms was a significant component of negative amplitude. Whereas predictable endings elicited a broad positive waveform from 200 to 600 ms, the incongruent words elicited a large negative wave in this latency range. The N400 is preceded by a series of exogenous components (P1-N1-P2) underlying the processing along the sensory channels of perceived stimuli and orientation of attention to the salience of stimuli [ 18 ].

The amplitude of the N400 is extremely sensitive to the context that precedes the critical or target word, whether it is a single word or a phrase: this context generates semantic priming [ 19 ]. N400 amplitude is also influenced by several lexical characteristics in addition to contextual factors: low frequency (less commonly used) words elicit larger N400s than high frequency words [ 20 ] . Moreover, it is also modulated by the type and degree of semantic association between the words and it seems to express the difficulty with which a word is recovered from the semantic memory: smaller N400s were also elicited by the second words of semantically-related (e.g., hot/cold) compared to semantically-unrelated (e.g., hot/noise) pairs [ 21, 22 ] and the difficulty is minimal if the word is expected and predictable, higher if unexpected or inconsistent.

The semantic context effect is also evident in printed, spoken and signed language [ 23, 24 ]. Furthermore, words with many orthographic neighbours (generated by taking a word, and replacing each letter in turn with every other letter e.g. brain – train, wave – wake) elicit a larger N400 than words with few orthographic neighbours [ 25 ]. This has been interpreted as stronger overall semantic activation due to orthographic neighbours (N) activating their semantic representations. A study with event-related potentials (ERPs) by Holcomb, Grainger, and O’Rourke (2002) [ 26 ] seems to indicate that orthographic neighbours activate their semantic representations. Holcomb et al. presented high-N and low-N words to the participants in their study and found that the ERP showed a bigger N400 for high than for low-N words. A better test of the claim that N400 effect is semantic would be to collate it with a manipulation supposed to involve the semantic level of representation, such as a manipulation involving the number of semantically associated words. To measure the semantic richness, Nelson et al. (1992) [ 27 ] proposed the number of associates (NoA). NoA can be defined as the number of different first associated word produced by the participant in a free association task, where participants are presented with a word and asked to write down the first words that comes to their mind. For example, given “garden”, they might write “flowers”. Generally, we could say that associates derived from the free association procedure refer to the semantic field of the target word. Therefore, NoA can be considered a reliable measure of semantic activation [ 28 ] . Neural bases of the N400 are being studied: it has been suggested that the N400 originates from several generators such as posterior temporal cortex and the angular gyrus29 . According to data obtained from intracranial recordings during speech reading, medial temporal structures near hippocampus and amygdala were considered to be possible locating the N400 generator [ 30 ] 2

Problem Statement and Proposed Approach

The goal of this study is to create a system that reproduces the user model, respecting user expectations and translating them into a satisfactory answer based on implicit data that the user does not control. An adaptive interface is able to customize the content and interaction mode with the user based on the information they have on the user. In many application domains, adaptive systems have proven, in many situations31, more effective and /or usable than corresponding non-adaptive systems.

The information gathered by the interface is useful in creating a "user model" that makes the interaction between man and machine more and more functional and in this sense “adaptive”. In a complementing approach, an adaptive interface exploits the asymmetry between man and machine to develop new interactions and collaboration possibilities [ 32 ].

A user model can be defined as a set of parameters (knowledge, expectations, preferences, and goals) that are relevant for the activity. Collecting informations, enables adaptivity, seen as interaction with the information domains to obtain a customized, contextualized, and environmentally compatible view [ 33 ].

The system therefore aims to create a user model that suits the user's expectations in the completion of sentences (i.e. by presenting at the end of the sentence the word he/she expects) and the choice of the N400 is justified by the fact that this wave represents the electrophysiological substrate of the violation of semantic expectancy [ 34 ].

In addition, the use of the N400 classification through online single-trial analysis is an element of novelty in literature, as the study and application in an online experiment of this brain wave are quite new in BCI research field [ 35 ].

Finally, we aim to identify the source of the signal by Standardized Low Resolution Electromagnetic Tomography (sLORETA). sLORETA consists of a method that allows the parametric estimate of the brain current density, locating non-invasive neural generators responsible for the electroencephalographic phenomena detected on the scalp. ERPs have a high temporal resolution, but distance between the electrodes applied to the scalp surface does not allow such precise spatial resolution.

sLORETA calculates a "reverse solution" of the electromagnetic problem, i.e. the calculation of images of neuronal electrical activity is performed from surface signals recorded on the scalp. This method provides information on the temporal trend and localization of brain functions. 2.1

Research Questions and Methods The objective of this study is to investigate the different amplitudes of N400 in function of the last word (which can be the target word or a semantically related word or an unrelated word), exploiting a Cloze-probability test. A Cloze-probability test is an exercise or a linguistic evaluation test consisting of a portion of text from which some words have been removed.

The cloze probability is the probability that a group of speakers completes a certain sentence with a given terminal word. In the test, the participant is required to enter the missing words that meet the expectation criteria (the last word in the sentence presented has .75 or higher probability of being completed by a specific single word). In the present experiment, a sentence database will be used which is also “validated” for N400 since all the sentences elicits a N400 effect produced by Block & Baldwin, 2010 [ 36 ]. The “last missing word database“ will be composed by using of the association rules produced by Nelson, McEvoy & Schreiber (2004) [ 37 ], the so-called University of South Florida Word Association Norms, which provides semantic connection databases between words obtained through a free association test.

The database was produced by a free word association task where the participant was shown a target word and after the participant was required to write the most semantically related word that came to his mind. This task is considered a “discrete” free association task as the participant is required to produce only one word. Free association rules provide a "forward strength" related information as they are sensitive to the number of other words that compete in the free association task.

Word-related probability values inform about memory access of the word by exploiting associative structures that involve word representations and represents how much a word is semantically related to the target word. The participant will be presented on sentence on the screen, missing the last word.

Randomly, the sentence will be completed (e.g. He loosened the tie around his….): - by the target word (chosen from the database of Block & Baldwin, 2010, e.g.

neck); - by a most common semantic associate produced by participants (word with the highest associated probability value chosen by database produced by Nelson, McEvoy & Schreiber, 2004 e.g. throat); - by an unrelated word (word with the lowest associated probability value chosen by database produced by Nelson, McEvoy & Schreiber,2004 e.g. chicken).

Once the expected word is displayed, the next sentence will be presented. To ensure attentive reading, participants will be asked to evaluate whether a word, after the presentation of all the sentences, was present or not within the phrases themselves. This operation will be done by pressing a left or right index finger (right and left mapping “present” and “absent” counterbalanced in all participants). Half of the words to be evaluated by the participant will be "extracted" from the previous sentences, while the other half will be “extracted” from stimuli of other phrases not yet presented. The words to be evaluated will be shown for 200 ms with 4300 ms blank screen before the next trial begins.

For the experiment, a total of 20 healthy English-proficient adult volunteers will be invited (approx. half male and female). The room will be kept dark and quiet during stimulus presentation to minimize interferences. Cloze probability sentences will be displayed in black Courier font at centre of the screen with one word at a time, with durations of 200 ms per word and 300 ms inter-word intervals. Each sentence will be followed by the pattern “XXXX,” displayed in the centre of the screen for 1000 msec, indicating the start of the next trial.

The system used in the experiment will include an EEG device, a computer, and two screens (one for the participant and one for the experimenter). EEG was recorded using 64 active electrodes, arranged according to the International 10-20 System (Acticap, BrainAmp, BrainProducts, Munich, Germany: sampling frequency 1000 Hz).

The ground electrode was placed on the participant's forehead while the reference electrodes at the linked-mastoids. The computer acquires the raw EEG signal from the device (via BrainVision Recorder software, BrainProducts, Munich, Germany). EEG data are then streamed to the ad-hoc software within the framework of the BBCIToolbox (https://github.com/bbci/bbci_public) executed with Matlab 2014b (MathWorks, Natic, USA).

Presentation of on-screen sentences (60 Hz, 1680 x 1050 pixels, 47.2 cm x 29.6 cm) was made possible through a custom software written in Processing 3.3 (https://processing.org). The continuous signal will be cut into time segments in the range of 200 to 500 ms after the closing word presentation.

The raw signal will be sub-sampled from 1000 Hz to 20 Hz to reduce the dimensionality of the features to be extracted. This operation is necessary to improve the classification output, avoiding the risk of overfitting. A specific brain response is expected by violating the user's expectations on words, showing detectable differences across conditions (target, related and unrelated). Based on these discernible differences, we want to create a user model of user expectations to reach the optimum state (presenting the word expected by the user).

Labelling of features vector will resemble the conditions, then a classification function will be trained with a regularized linear discriminant analysis [ 38 ] in order to discriminate the three classes (target, related, unrelated). A BCI based on supervised machine learning needs to be calibrated before it can be applied.

This calibration is typically performed on sets of recordings, usually EEG epochs, which are known to contain the signals that need to be detected later. On the basis of these epochs, a classifier is calibrated to optimally distinguish between the different classes of source signals.

We will then define three different classes of classification which will be compared to each other: 1. Online Classification Condition: to enable real-time detection of the individual, single-trial neuroelectric responses, a discriminative classification system will be calibrated. The extracted information will be used for reinforcement learning on the side of the target word, modifying the probabilities of presentation upcoming words such that the target word (word expected from the cloze probability test) would be more likely to appear, if classifications are correct. 2. Random Condition: no reinforcement takes place and words complete the sentence randomly; 3. Perfect Condition: in every trial, the target word always appears in the first attempt.

The participants will first run a calibration training session and then begin the experimental session. Calibration session will be composed of 50 trials while the experimental session will last 240 trials. 3

Expected Results and Limitations

The aim of this experiment is to combine the information collected from electrophysiological responses to qualitatively different stimuli to obtain a deeper analysis of the operator's cognition. The expected results refer to electrophysiological differences resulting from the comparison between the various semantic discrepancies. Referring to the literature, different works showed that N400 amplitudes gradually decrease with increasing expectancy of a given word in a given context [ 39, 40 ]. Therefore, we expect a lower mean amplitude of the N400 in the target state (word expected from the cloze probability test), slightly larger when the semantic associate is presented, and finally the greater mean amplitude of all should occur in the “neutral state (words with no semantic association chosen by database)”.

These differences, due to the nature of stimuli, should reflect a progressive violation of user expectations and by extension, its initial expectations and should be clearly recognized by the classification system. To enable real-time detection of each single-trial response, the discriminative classification system will be based on three classes of responses representing the above-mentioned conditions.

Regarding the analysis that will be performed after the data collection, one-way analysis of variance of the systematic peak differences around 250-450 ms will be completed. We will conduct a source analysis which, according to Lau et al., 2009, should predict the involvement of angular gyrus and inferior frontal cortex.

This study represents a novelty element in the BCI search field for distinct reasons. First, BCI tasks generally require stimulation recognition while in this experiment a semantic processing is required, reflecting a deep stimulation processing. In addition, the experiment allows the integration of implicit information through different types of semantic relation. A future experiment could consider multiple related or even whole texts, opening up a new challenge on how to draw the task, analyse and interpret data as there is a close semantic relationship between complex elements (composed of more words or phrases).

Regarding the limits of the present study and the possible new directions to be followed in the research, it can be safely stated that the completion of sentences does not surely resemble an effective communication. The interpretation of the meaning of a sentence and its communication goes far beyond the recognition of a word, albeit expected and fundamental to convey meaning.

For this reason and as said previously, next studies will have to consider more than one word or possibly the whole sentence. It would also interesting to propose to the participant an online decision-making task in which one or more sentences semantically related to the last word of the first sentence are presented. In this type of task, the participant chooses the final word to complete a sentence, and on the basis of the latter one or more sentences continue and deepen the meaning of the preceding sentence. For example, if we present to the participant the phrase "After high school I would like to attend the university of ... economics / law / engineering", in case the relevant word becomes "economics" can be presented successively a sentence like “After graduation I would like to work in bank / start-up company / financial consulting.” 4

Conclusions

BCIs applied to the field of communication offer great areas of application and wide development. Initially, researchers were interested in allowing communication to clinical populations with little to no communication capability to generate text. This type of technology will probably not be extended to healthy populations, and thus opens a wide field for innovative technologies that can go beyond the basics of communication and go towards the generation of speech.

Future BCIs applied to the field of communication should increase the space in which man and computer can communicate, by promoting understanding of information or even predicting the comprehension of information contexts. Strengthening the communication space between man and machine means increasing the usable information: in human interaction, for example, the information transmitted goes beyond words, body language, facial expressions and proximity space, and adds meanings to communication. This information could be combined with the information used by the BCIs to provide more information about the user's state and be integrated into human-computer communication.

Just think of future BCIs that exploit multiple system sensory recordings such as eye tracker and head movements in combination with other factors that affect the user's state of physical fatigue, cognitive workload, and arousal levels. This type of technology could combine these data to generate forecasts that could be used to modify the information provided to you to improve the effectiveness of communication.

In this research path, the study of N400 may play a significant role since, at present, this brain wave has not been widely used in the BCI domain and there can be numerous applications: for example, a device that can detect semantic misunderstanding in communication between two people. Using this type of information, the system could provide an indicator of mutual understanding or effectiveness of communication, or even provide unprofitable communications solutions, repeating or reformulating sentences, suggesting alternative words and making communication clearer and less ambiguous.

There are also technologies already developed that could allow computers to analyse and predict what users are trying to say [ 41,42 ]. These technologies can be applied across fields from search engines using collaborative filters to suggest useful terms to computer vision algorithms that use graph theory to find objects similar to the default set. Here too we can imagine future applications involved in the formation of human-computer semantic lexicons, including multimedia material such as images, sounds and videos that adapt to individual needs. These systems, in which we can already include some applications [ 43 ], could provide opportunities that would go beyond accessing a computer, allowing the computers themselves to create connections between the user's own concepts and returning different ideas back to the user based on his input.

The capillary diffusion of neuroscientific research and neuro-technologies offer opportunities to extend computer applications to predictive capabilities also of emotional and cognitive states. Using this information could revolutionize not only the design of interfaces, but also the relationships of users with the same systems. Beyond these rosy prospects, BCI technologies still have to face many challenges over the next few years. For example, the ability of people to adapt to complex tasks and demanding environments show difficulties in interpreting their neural substrates and in designing ecological tasks. Other difficulties arise from the overlapping of the neural processes generated by task or from multiple tasks, or from long-term changes within the same task, in addition to the great inter-individual variability of the electroencephalographic tracks. The progressive application and integration of neurotechnologies with other disciplines will promote wide insertion spaces for BCI technologies, which have the potential to profoundly influence daily life if researchers can overcome obstacles such as detecting and interpreting neural signal in an ecological setting with no constraints.

References

1. Wolpaw , J. R. , Birbaumer , N. , McFarland , D. J. , Pfurtscheller , G. , & Vaughan , T. M. ( 2002 ). Brain-computer interfaces for communication and control . Clinical neurophysiology , 113 ( 6 ), 767 - 791 .

2. Coyle , S. , Ward , T. , Markham , C. , & McDarby , G. ( 2004 ). On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces . Physiological measurement , 25 ( 4 ), 815 .

3. Mak , J. N. , Arbel , Y. , Minett , J. W. , McCane , L. M. , Yuksel , B. , Ryan , D. , ... & Erdogmus , D. ( 2011 ). Optimizing the P300-based brain-computer interface: current status, limitations and future directions . Journal of neural engineering , 8 ( 2 ), 025003 .

4. Tan , D. , & Nijholt , A. ( 2010 ). Brain-computer interfaces and human-computer interaction (pp. 3 - 19 ). Springer London.

5. Kübler , A. , & Neumann , N. ( 2005 ). Brain-computer interfaces-the key for the conscious brain locked into a paralyzed body . Progress in brain research , 150 , 513 - 525 .

6. Pfurtscheller , G. , Brunner , C. , Schlögl , A. , & Da Silva , F. L. ( 2006 ). Mu rhythm (de) synchronization and EEG single-trial classification of different motor imagery tasks . NeuroImage , 31 ( 1 ), 153 - 159 .

7. Pfurtscheller , G. , Guger , C. , Müller , G. , Krausz , G. , Neuper , C. ( 2000 ). Brain oscillations control hand orthosis in a tetraplegic . Neurosci. Lett . 292 , 211 - 214 .

8. Kostov , A. , Polak , M. ( 2000 ). Parallel man-machine training in development of EEG-based cursor control . IEEE Trans Rehabil Eng . 8 : 203 - 205 .

9. Lotte , F. , Congedo , M. , Lécuyer , A. , Lamarche , F. , & Arnaldi , B. ( 2007 ). A review of classification algorithms for EEG-based brain-computer interfaces . Journal of neural engineering , 4 ( 2 ), R1 .

10. Birbaumer , N. , Ghanayim , N. , Hinterberger , T. , Iversen , I. , Kotchoubey , B. , Kübler , A. , Perelmouter , J. , Taub , E. , and Flor , H. ( 1999 ). A spelling device for the paralyzed . Nature . 398 , 297 - 298 .

11. Kübler , A. , Neumann , N. , Kaiser , J. , Kotchoubey , B. , Hinterberger , T. and Birbaumer , N. ( 2001 ). Brain-computer communication: Self-regulation of slow cortical potentials for verbal communication . Archives of Physical Medicine and Rehabilitation , 82 , 1533 - 1539 .

12. Pham , M. , Hinterberger , T. , Neumann , N. , Kübler , A. , Hofmayer , N. , Grether , A. , Wilhelm , B. , Vatine , J. J. , Birbaumer , N. ( 2005 ). An auditory brain-computer interface based on the self-regulation of slow cortical potentials . Neurorehabil Neural Repair . 19 , 206 - 18 .

13. Neumann , N. , and Birbaumer , N. ( 2003 ). Predictors of successful self control during brain-computer communication . J. Neurol. Neurosurg. Psychiatry 74 , 1117 - 1121 .

14. Luck , S. J. , & Kappenman , E. S. (Eds.). ( 2011 ). The Oxford handbook of eventrelated potential components . Oxford university press.

15. Kutas , M. , Hillyard , S ,. A. ( 1982 ). The lateral distribution of event-related potentials during sentence processing . Neuropsychologia . 20 , 579 - 90 .

16. Donchin , E. , Ritter , W. , & McCallum , W. C. ( 1978 ). Cognitive psychophysiology: The endogenous components of the ERP . Event-related brain potentials in man , 349 - 411 .

17. M. Kutas and S. A. Hillyard , “ Reading senseless sentences: Brain potentials reflect semantic incongruity ,” Science, vol. 207 , no. 4427 , pp. 203 - 205 , 1980 .

18. Duncan , C. C. , Barry , R. J. , Connolly , J. F. , Fischer , C. , Michie , P. T. , Näätänen , R. & Van Petten , C. ( 2009 ). Event-related potentials in clinical research: guidelines for eliciting, recording, and quantifying mismatch negativity, P300, and N400 . Clinical Neurophysiology , 120 ( 11 ), 1883 - 1908 .

19. Kutas , M. , & Federmeier , K. D. ( 2011 ). Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP) . Annual review of psychology , 62 , 621 - 647 .

20. Connolly JF , Phillips NA , Stewart SH , Brake WG . Event-related potential sensitivity and semantic properties of terminal words in sentences . Brain Lang 1992 ; 43 : 1 - 18 .

21. Boddy , J. , & Weinberg , H. ( 1981 ). Brain potentials, perceptual mechanisms and semantic categorization . Biological Psychology , 12 ( 1 ), 43 - 61 .

22. Rugg , M. D. ( 1985 ). The Effects of Semantic Priming and Word Repetition on Event‐Related Potentials . Psychophysiology, 22 ( 6 ), 642 - 647 .

23. Neville HJ , Mills DL , Lawson D. Fractionating language: different neural systems with different sensitive periods . Cereb Cortex 1992 ; 2 : 244 - 58 .

24. Van Petten , C. , & Luka , B. J. ( 2006 ). Neural localization of semantic context effects in electromagnetic and hemodynamic studies . Brain and language , 97 ( 3 ), 279 - 293 .

25. Laszlo , S. , & Federmeier , K. D. ( 2011 ). The N400 as a snapshot of interactive processing: Evidence from regression analyses of orthographic neighbor and lexical associate effects . Psychophysiology , 48 ( 2 ), 176 - 186 .

26. Holcomb , P. J. , Grainger , J. , & O'rourke , T. ( 2002 ). An electrophysiological study of the effects of orthographic neighborhood size on printed word perception . Journal of Cognitive Neuroscience , 14 ( 6 ), 938 - 950 .

27. Nelson , D. L. , Schreiber , T. A. , & McEvoy , C. L. ( 1992 ). Processing implicit and explicit representations . Psychological review , 99 ( 2 ), 322 .

28. Müller , O. , Duñabeitia , J. A. , & Carreiras , M. ( 2010 ). Orthographic and associative neighborhood density effects: What is shared, what is different? . Psychophysiology , 47 ( 3 ), 455 - 466 .

29. Lau , E. F. , Phillips , C. , & Poeppel , D. ( 2008 ). A cortical network for semantics:(de) constructing the N400 . Nature Reviews Neuroscience , 9 ( 12 ), 920 - 933 .

30. Müller , O. , Duñabeitia , J. A. , & Carreiras , M. ( 2010 ). Orthographic and associative neighborhood density effects: What is shared, what is different? . Psychophysiology , 47 ( 3 ), 455 - 466 .

31. Tan , D. , & Nijholt , A. ( 2010 ). Brain-computer interfaces and human-computer interaction . In Brain-Computer Interfaces (pp. 3 - 19 ). Springer London.

32. Suchman , L. A. ( 1987 ). Plans and situated actions: The problem of humanmachine communication . Cambridge university press.

33. Benyon , D. ( 1993 ). Adaptive systems: a solution to usability problems. User modeling and User-adapted Interaction , 3 ( 1 ), 65 - 87 .

34. Curran , T. , Tucker , D. M. , Kutas , M. , & Posner , M. I. ( 1993 ). Topography of the N400: brain electrical activity reflecting semantic expectancy . Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section , 88 ( 3 ), 188 - 209 .

35. Wenzel, M. A. , Bogojeski , M. , & Blankertz , B. ( 2017 ). Real-time inference of word relevance from electroencephalogram and eye gaze . Journal of Neural Engineering.

36. Block , C. K. , & Baldwin , C. L. ( 2010 ). Cloze probability and completion norms for 498 sentences: Behavioral and neural validation using event-related potentials . Behavior research methods , 42 ( 3 ), 665 - 670 .

37. Nelson , D. L. , McEvoy , C. L. , & Schreiber , T. A. ( 2004 ). The University of South Florida free association, rhyme, and word fragment norms . Behavior Research Methods , 36 ( 3 ), 402 - 407 .

38. J H Friedman . Regularized discriminant analysis . Journal of the American Statistical Association , 4 ( 405 ): 165 , 1989 . doi: 10 .2307/2289860

39. Kutas , M. , & Hillyard , S. A. ( 1984 ). Brain potentials during reading reflect word expectancy and semantic association . Nature , 307 ( 5947 ), 161 - 163 .

40. Laszlo , S. , & Federmeier , K. D. ( 2011 ). The N400 as a snapshot of interactive processing: Evidence from regression analyses of orthographic neighbor and lexical associate effects . Psychophysiology , 48 ( 2 ), 176 - 186 .

41. W. Liu and S. F. Chang , BRobust multi-class transductive learning with graphs , [ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit ., 2009 , pp. 381 - 388 .

42. D. Goldberg , D. Nichols , B. M. Oki , and D. Terry , Busting collaborative filtering to weave an information apestry , [ Commun. ACM , vol. 35 , no. 12 , pp. 61 - 70 , 1992 .

43. Gamberini , L. , Spagnolli , A. , Blankertz , B. , Kaski , S. , Freeman , J. , Acqualagna , L. & Ferrari , E. ( 2015 ). Developing a symbiotic system for scientific information seeking: the MindSee Project . In Symbiotic Interaction (pp. 68 - 80 ). Springer, Cham.