In interpersonal relations, interlocutor's emotions may be easily detected by facial and oculomotoric expressions. Several books are devoted to recognition so called body language expressions (e.g. [18]) several others - to avoiding of uncontrolled disclosure of spontaneous affects (e.g. [14]). Since modern information societies come to replace direct relations by computer-mediated communication and virtually all computers capture the face of the user, the idea of automatic emotion recognition becomes one of most important issue of man-machine interaction. Several papers focusing on facial expression were published as far (e.g. [19]), however our interest focuses rather on emotions expressed in the eyetrack [8], [12]. The approach is motivated by two facts: - the eye globe motion response cannot be trained as easily as face movements in order to hide emotions and - the eye motion can be elicited by the displayed content.
Four main principles were employed as far to capture the instantaneous
position of the eye: electric, magnetic, optical and visual [23], [
Rysunek 1. A volunteer observer wearing Ober-2 eyetracking goggles. exceptions, designed for open eye recordings. They are immune to interferences of electrophysiological origin, but since the measurement is reliant on ambient lighting condition, eyetracking systems either assumme stable conditions either employ algorithms to compensate for variations. These methods are widely used for studies of perception, interaction or visual conspicuity of the scene [16].
While reflection-based eyetrackers are uniquely mounted on the observer’s head, most of visual eyetrackers use stand-alone cameras allowing for distant and even consciousless recording. This has two practical consequences: – wearable eyetrackers determine the eyeglobe position in head-related (i.e. relative) coordinates, allow for mobility during visual tasks, however transforming data to environment-related coordinates (i.e. absolute) needs continuous tracking of the head with a seach coil, visual system or navigation, – stationary eyetrackers determine the eyeglobe position in absolute coordinates but work in limited space, higher grade systems employ face or marker tracking algorithm to compensate for head motion either with an auxiliary camera either using a high resolution sensor.
The other aspect is the eyetracking speed. In most perceptual studies 50-200 position samples per second are sufficient for accurate eyetracking, what justifies suitability of consumer-grade cameras to these tasks. Otherwise, studies of fast eye motion like microsaccades, or interactive applications requiring real time reaction of the system would benefit from information rate up to 2 ksps. Detection of affects, described later in this paper will also need a high-speed recording equipment, since degradation of perception under influence of emotion may be related to alternate patterns of ocular muscles control. 3
Various eyetracking techniques were implemented in our laboratory for last two
decades. We employed electrical, visual and infrared reflection-based methods
accordingly to the examination purpose and expected performance. Several
attempts to eyetracking were focused on:
– human equilibrium assessment - computer-assisted processing of
electronystagmographic response to Hallpike caloric test [
These works revealed a particular usability of the eyetracking in assessment of human preferences and performance in completing visual tasks. Some tasks were performed in real live (driving), but most of them were implemented as computer simulation through displayed images (ECG signal, OCT image or a text message to read). This setup facilitates standardization of measurement conditions and stabilization of the observer making the results repeatable and reliable. 4
Human performance degrades in certain circumstances due to internal or external influences. Using the eyetracking as a reliable tool for instantaneous and objective assessment of performance we also can estimate the degradation extent and correlate it with the intensity of disturbing factors. This research is expected not only to shed light on the dependence of human errors and efficiency in various stress conditions but also to quantitatively measure the influence of disturbing factor.
One of the most interesting internal factor influencing human performance are affects, probably due to their hardly controlled occurrence. Nevertheless, recognition of affects combined with knowledge of human response is expected to help in adaptation of intelligent computer interface to maintain the most effective communication. To this point we aim at preparing visual tasks for measurement of performance in neutral conditions as a reference and then its degradation with increasing level of a specific emotion. This research is expected to reveal external factors that evoke emotions and possibly influence human performance in certain tasks.
Other interesting issue is modeling human performance in presence of external influencing factor of given intensity. Such model may be useful in adaptation of computer communication interface to actual preferences of the operating human. Planned experiments are expected to apply various information forms (i.e. displays) to a computer operator being in gradually rising emotions in order to rank these forms accordingly to their robustness. Having such personal profiles we could predict and employ the most appropriate communication form depending on actually detected emotional state. 5
Industrial defects are very costly, whereas machinery are usually very complex (e. g. Boeing 373 contains 367000 parts and Airbus A380 has an approximate 4 million parts [20]). Skilled engineers and technicians are subjects of a perpetual training and examining process to maintain best possible working excellence. Despite these efforts, little is known on how much stress conditions influence their skills at the particular moment when an unexpected repair work is performed. Since searching among real-life object, their representation in documentation and comparing measurement readouts are purely visual tasks, we postulate predicting the skills of each individual in wide range of unfavorable conditions in order to apply the personalized interface providing the most proper information in a most acceptable form.
Entertainment selection belongs to typical tasks of personal digital advisors, however the information based on age, gender or history of the spectator or client is not sufficient to yield suitable suggestions. The particular purpose of watching movies is rather to get involved emotionally than to know details of another story. In addition, emotions are hardly explainable in a conscious way. Therefore the objective detection of spectator’s emotions and preferences, which can be easily captured by an eyetracker, is postulated to be used in context of automatically derived semantic description of the footage. The scanpath indicating the preferred character and meta-description of this character are helpful in research for other movies containing characters of the same type. 6
Scanpath analysis is a promising tool for objective assessment of degradation of human performance under various emotional conditions. The method is, however, limited to visual tasks. Future works will be directed to establishing appropriate models of scanpath response to standardized affective stimuli and to optimizing detection methods towards their use as a background real-time service in any digital communication equipment.
Studies show that building of a cheap high speed eyetracker is still an
interesting scientific challenge [
This scientific work is supported by the AGH University of Science and Technology in year 2016 as a research project No. 11.11.120.612.
12. de Lemos J., Sadeghnia G. R., Ólafsdóttir I., Jensen O.: Measuring emotions using eye tracking, Proceedings of Measuring Behavior 2008, 266 13. Lewandowski T., Augustyniak P.: The system of a touchfree personal computer navigation by using the information on the human eye movements. Proc. 3-rd international conference on Human System Interaction (HSI 2010), 2010, 674-677 14. Meyer Y.: Managing Your Emotions: Instead of Your Emotions Managing You,
FaithWords; Warner Book’s Edition 2002 15. Mikrut Z., Augustyniak P.: Estimation of Execution Time for Tasks of Objects
Counting and Localization Using the Ober2 Device, Proc. IFMBE 2, 2002, 144-145 16. Morimoto C., Mimica M. (2005), Eye gaze tracking techniques for interactive applications, Computer Vision and Image Understanding, 98(1), 4-24, DOI: 10.1016/j.cviu.2004.07.010 17. Ober J. J., Hajda J., Loska J., Jamicki M.: Application of eye movement measuring system Ober2 to medicine and technology, Proc. of SPIE textbf3061(1), 1997, 327-332 18. Pease A., Pease B.: Body Language The Definitive Book, Pease International,
Buderim, Australia 2004, 19. Przybyło J.: Vision Based Facial Action Recognition System for People with Disabilities. Proc. 8-th international conference on Human System Interactions (HSI2015), 2015, 244-248 20. https://www.quora.com 21. Rached T.S., Perkusich A.: Emotion Recognition Based on Brain-Computer Interface Systems, in: Brain-Computer Interface Systems - Recent Progress and Future Prospects, InTech, 2013, 253-270 22. SMI I-VIEW Hi-Speed http://www.smivision.com/en/gaze-and-eye-trackingsystems/products/iview-x-hi-speed.html 23. Teiwes W.: Video-okulografie - Registrierung von Augenbewegungen in drei Freiheittsgraden zur Erforscherung und Medizinischen Diagnostik des Gleichgewichtssystems (in German), dissertation TU Berlin 1991