=Paper=
{{Paper
|id=Vol-1794/afcai16-paper7
|storemode=property
|title=Quantifying Attention in Computer-based Tasks
|pdfUrl=https://ceur-ws.org/Vol-1794/afcai16-paper7.pdf
|volume=Vol-1794
|authors=Davide Carneiro,Dalila Durães,Paulo Novais
|dblpUrl=https://dblp.org/rec/conf/afcai/CarneiroDN16
}}
==Quantifying Attention in Computer-based Tasks==
https://ceur-ws.org/Vol-1794/afcai16-paper7.pdf
Quantifying Attention in Computer-based Tasks
Davide Carneiro1,2 , Dalila Durães3 , Javier Bajo3 , and Paulo Novais2
CIICESI, ESTG, Polytechnic Institute of Porto
1
Felgueiras, Portugal
2
Algoritmi Center/Department of Informatics, Minho University
Braga, Portugal
{dcarneiro,pjon}@di.uminho.pt
3
Department of Arti cial Intelligence, Technical University of Madrid, Madrid, Spain
{d.alves,jbajo}@fi.upm.es
Abstract. Attention-to-task is one of the most important Human cogni-
tive abilities, allowing an individual to selectively focus on a speci c issue
(among many possible sources) and eectively carry out a task. With-
out this ability to focus, the individual would constantly switch between
stimuli, hardly concluding any task. While attention can be in uenced
by many internal and external factors, the purpose of this paper is not
to analyse them but rather to propose an approach to monitor the atten-
tional behaviour of computer users. The proposed approach may improve
the individual's self-awareness as well as the team manager's knowledge
about the state of the workforce. It may thus improve the de nition of
better attention-management strategies, with expected improvements in
variables such as on-task behaviour, productivity or work quality.
Keywords: Human-Computer Interaction, Keystroke Dynamics, Mouse
Dynamics, Attention
1 Introduction
Nowadays, working, as well as many other activities (e.g. education) take place
wholly or partially at the computer. This represents a major shift that took place
in the span of a few years, but that de nitely changed our relationship with the
workplace or the classroom.
This new way of working or studying has, therefore, new characteristics that
can represent, at the same time, challenges and opportunities. On the one hand,
working for long hours at the computer may have a negative impact in the indi-
vidual's health, namely due to the lack of physical exercise. For the same reason,
these jobs also tend to be considered as more boring when compared to tradi-
tional more physical and active ones [1]. Asides from other aspects, this means
that these jobs may cause a certain lethargy, fatigue or sleepiness, especially
when carried out for long hours [2].
One of the common ways of countering these negative eects is to advise
people to make breaks and pauses at regular intervals, taking small walks or
simply exiting the working space for a while, as a way to activate the mind and
the body. However, this is done either at rigid intervals (e.g. every hour) or left
at the responsibility of the individual.
The rst approach has the drawback that we are not all equal: while one in-
dividual may maintain performance even after working for some hours straight,
another may feel bored after an hour. Moreover, the same individual may be-
have dierently in dierent days, depending on factors such as motivation, task
characteristics, time of the day, and many other internal and external factors.
The latter approach may also have drawbacks. Namely, the notion of atten-
tion is a rather subjective one and dependent on the individual's interpretation.
Moreover, despite the individual's state, she/he may continue working because
there are deadlines to meet.
In all these situations, the main problem is the lack of an eective mea-
sure of attention [3], that can objectively quantify this concept. The de nition
and quanti cation of such a measure is the key element in this paper, provided
through real-time analytics [4]. It is especially suited for milieus where people
work with the computer for long periods of time (e.g. software houses, banking,
call centres, academia). It is designed with the purpose to provide team man-
agers or human-resources personnel with important knowledge about how each
individual behaves throughout the day or how each individual reacts to events
(e.g. increased/decreased workload).
2 Approach
The measure of user attention proposed in this paper is based on task char-
acteristics, keystroke and mouse dynamics, activity level and application usage
tracking.
Task characteristics are de ned by the team manager or responsible and
include, among others, the names of the applications that are related to the task.
These names are de ned using an approach close to natural language, which is
then translated to regular expressions. As an example, the team manager can
provide rules such as "Contains 'Microsoft'" or "Starts with 'Adobe'". These
rules are then used by the system to determine which applications are work-
related, allowing to quantify the time spent interacting with them versus with
others, non-work-related applications.
Keystroke and mouse dynamics include ways to characterize the user's in-
teraction with the computer through the mouse and the keyboard. The system
extracts 16 features that include mouse velocity, typing rhythm, distance trav-
elled by the mouse, writing latency, among others, that fully characterize the
user's interaction patterns. In previous work we have explored, to a large extent,
how these features relate to worker performance or how they are in uenced by
stress, namely in the academic context [5, 6]. In this work, these features are
revisited to de ne a way of quantifying the level of activity of the user.
Indeed, the level of activity is a very important factor to consider when
analysing attention as it provides the necessary context to interpret the other-
wise poor information regarding attention (i.e. application usage). For instance,
a given user may have a work-related application in foreground but leave the
computer and come back after an hour. This does not mean that the user was
devoted to the task for that hour since the application was focused but there
was no interaction going on. This approach allows to point out these situations
as well as to provide a level of activity. All in all, it improves the accuracy of the
quanti cation of attention.
Finally, application usage tracking considers the analysis of how users spend
their time at the computer. The developed system tracks changes in the active
application, recording its name and the time at which the change took place.
This allows to quantify the amount spent interacting with each application, in
a given period of time. The knowledge about task characteristics allows thus
to infer how much time was spent interacting with work-related applications.
Moreover, it also allows to infer other behaviours such as listening to music while
working or to identify moments in which the worker is no longer producing or
has completely abandoned the task.
3 Architecture
In order to implement this approach, a system was developed with an archi-
tecture as described in Figure 1. It is divided in three major components. The
raw data is generated in the devices, then pre-processed (by redundancy elimina-
tion) and stored locally whenever possible (as in personal computers) in a SQLite
database. Then data is synchronized with the web servers in the cloud. The tar-
get database is MongoDB (object-oriented DB). MongoDB4 is a database that
is half way between relational and non-relational systems. It provides indexes on
collections, it is lockless and provides a query mechanism. MongoDB provides
atomic operations on elds like relational systems MongoDB supports automatic
sharding by distributing the load across many nodes with automatic failover and
load balancing, on the other hand CouchDB achieves scalability through asyn-
chronous replication. MongoDB supports replication with automatic failover and
recovery. The data is stored in a binary JSON-like format called BSON that sup-
ports boolean, integer, oat, date, string and binary types. The communication
is made over a socket connection (in CouchDB it is made over an HTTP REST
interface).
One of the most interesting applications of this system is to monitor student's
attention in the classroom. In fact, this system has been in the past year in
the Caldas das Taipas Higher School, located in northern Portugal. Dierent
classes, with dierent characteristics, are being continuously followed throughout
the year. This vast data-collection will allow teachers to assess the in uence on
attention of aspects such as the time of the day, breaks, classes' contents, classes'
objectives, and learning styles. The aim is that with this knowledge, teachers
are able to improve their teaching strategies, adapting them to improve student
attention.
4
https://www.mongodb.com
Fig. 1. Architecture of the attention monitoring system.
As an example, we brie y analyse the data collected for two dierent classes:
a bells-letters class (12F) and a vocational class (12I). Both classes took place
at the same time but in dierent rooms and the contents and aims of each class
were the same: an application to teach algorithmic concepts. The teacher de ned
the applications that were necessary to carry out the task as those containing
the strings "code.org" and "Microsoft Word".
The Visualization layer provides intuitive ways for the team manager (in this
case the teacher) to assess the behavioural dierences between these two classes.
Figure 2 shows the evolution of the general attention of each class, calculated
through the running average (bells-letters class (a) and vocational class (b)). It
is possible to conclude that attention in the bells-letters class is generally higher
but with a tendency to decrease while in the vocational class it starts relatively
lower but with a tendency to increase over time. The teacher may have access
to this information in real time or after the class takes place. In either case, the
teacher may interpret this knowledge in its proper context (e.g. characteristics
of the group of students, aims of each course, etc.) and look for ways to improve
student attention.
The system also provides easy access to information regarding the level of
activity of the users, which quanti es the degree of interaction between the
user and the computer. For example, Figure 3, which considers the same data
described before, shows that the general level of activity is higher in the bells-
letters class (12F) than in the vocational class (12I), as evidenced by a faster
use of the mouse.
With this kind of information, the teacher is able to better understand stu-
dents, and perceive not only which students or groups of students are more
Fig. 2. Evolution of attention in both classes.
Fig. 3. Comparison of Mouse Velocity and Mouse Acceleration in both classes.
attentive to tasks but also, and perhaps more important, which tasks are more
motivating and engaging, allowing to better steer classes in the future.
4 Conclusions
Concluding, the proposed system is able to quantify the level of attention of a
group of people, in real time. Moreover, it also measures the level of activity
using keystroke and mouse dynamics, through features such as mouse velocity,
keyboard typing speed, among others. This, together with information about
the task (e.g. required applications) allows to quantify the actual attention of
each computer user to the current task, in real-time. Such information can sig-
ni cantly improve existing approaches, by providing an accurate measure based
on which better decisions can be taken to manage attention.
When compared to existing approaches, the main advantage of this system
is that it is not based on productivity measures, i.e., it is not based on how
much the worker is producing but rather on how the worker is producing. This
is an advantage as productivity-based initiatives often have a negative impact
on productivity that stems from the added pressure on the worker. Moreover,
user privacy is protected by masking sensitive information such as the keys
pressed or the speci c applications used. That is, the team manager has only
access to the compiled data and not to the raw data. For these reasons, we
believe that this approach may constitute an eective and interesting approach
to implement attention monitoring initiatives, in milieus such as the workplace
or the classroom.
Acknowledgement
This work has been supported by COMPETE: POCI-01-0145-FEDER-007043
and FCT Fundação para a Ciência e Tecnologia within the Project Scope:
UID/CEC/00319/2013.
References
1. Kuppermann, M., Lubeck, D.P., Mazonson, P.D., Patrick, D.L., Stewart, A.L.,
Buesching, D.P., Filer, S.K.: Sleep problems and their correlates in a working pop-
ulation. Journal of General Internal Medicine 10(1) (1995) 2532
2. Black, C.D., O'Connor, P.J., McCully, K.K.: Increased daily physical activity and
fatigue symptoms in chronic fatigue syndrome. Dynamic Medicine 4(1) (2005) 1
3. Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cognitive
psychology 12(1) (1980) 97136
4. Kejariwal, A., Kulkarni, S., Ramasamy, K.: Real time analytics: algorithms and
systems. Proceedings of the VLDB Endowment 8(12) (2015) 20402041
5. Carneiro, D., Castillo, J.C., Novais, P., Fernández-Caballero, A., Neves, J.: Multi-
modal behavioral analysis for non-invasive stress detection. Expert Systems with
Applications 39(18) (2012) 1337613389
6. Pimenta, A., Carneiro, D., Novais, P., Neves, J.: Monitoring mental fatigue through
the analysis of keyboard and mouse interaction patterns. In: Hybrid Arti cial In-
telligent Systems. Springer (2013) 222231