Behavior Change Support Systems (BCSS) benefit from understanding the user: a user profile can help select the right way to formulate an argument, selecting the right tone, format and content. Part of such a profile is an adequate representation of the computer literacy of a user. Unfortunately, computer literacy is commonly measured by asking the user to fill in a questionnaire. This an obstacle to the adoption of a BCSS, and as such is not a good way to build a model of a user's computer literacy. In this paper we describe the setup of a series of experiments intended to identify indicators that can be measured automatically and that correlate well with a relevant concept of computer literacy.
The concept of using the computer as a means to persuade users originated from the
90s, with Captology, or Computers as Persuasive Technologies coined as a term in a
special interest group meeting in CHI '97 [
A persuasive system with the intent to change attitudes or behaviour conveys
messages designed to influence the user in some way. Whether the intent is to change an
attitude, behaviour, or just to garner short-term compliance, these messages use
persuasive techniques designed to influence users, explicitly or implicitly, to affect a
change in their view or behaviour. To do so, it is important to understand the user:
certain persuasive techniques can backfire when applied to the wrong person (for
example, using Cialdini's Authority principle [
We propose a system for measuring theoretical construct of computer literacy in an automated fashion, based on published definitions of these measurements in existing literature. Computer literacy influences a user’s perception of technology, as well as the user’s comprehension of any computer related message. This can influence the persuasiveness of a system: communicating on the wrong technical level, or with the wrong medium (such as written text, or a virtual agent) can cause an adverse reaction. In the context of the Personal Information Security Assistant (PISA) project, we aim to implement a system to measure computer literacy, with the goal of using these measurements to tailor interactions with the user, which should improve the persuasive power of the system. In this workshop paper, we restrict ourselves to the description of experiments to correlate automated measurements of Computer Literacy to more traditional measurement by questionnaire. These experiments aim to answer the following research questions: • RQ1: How can we translate mental constructs to automated measurements based on user observation? • RQ2: How can we compare such automated measurements to existing measurement methods? Answering these research question allows us to answer the following research question as a part of future work: • RQ3: Can we use these measured mental constructs to construct a persuasive profile, improving the persuasive power of a system? 1.2
A comprehensive overview of literacies including a discussion about computer
literacy has been published by Bawden [
Computer literacy contains several dimensions within literature. Throughout our
literature review, we find the five most important dimensions of computer literacy to
be the following, based on their frequency of occurrence within literature:
• Skills: The skills associated with computers. This includes basic computer
operation skills such as knowing how to use a keyboard and mouse, but can also contain
more advanced concepts such as programming skills.
• Knowledge: Knowledge of the characteristics, capabilities and context of the
computer. This includes general computer terminology and software concepts, but
also the social and ethical context in with computers reside.
• Attitude: The collection of Cognitive, Behavioural and Affective attitudes that a
person can have towards computers. This includes well covered concepts such as
computer anxiety, but also computer interest and beliefs about computers.
• Experience: The measure of time and frequency a person uses a computer. It is
assumed that time spent using the computer leads to associated knowledge and
skills.
• Information: Argued by some to be "distinct but interrelated" [
There are no elements common to all definitions and/or measurements of computer literacy that we reviewed. However, the Computer Skills and Computer Knowledge dimensions are present in most of the consulted documents, and we regard these as the central elements of computer literacy. 2
The PISA is personal assistant designed as a BCSS for Privacy and Security, aiming to change behaviours and attitudes relating to personal information management and personal security. In this context, measuring Computer Literacy serves a dual purpose: it a) allows for communication with the user on a technically appropriate level, and b) helps in the assessment of the risks a user is subject to such as phishing and viruses. We have implemented a system that can collect measurements related to computer literacy described in section 1.2. These measurements include elements such as browsing behaviour, typing speed and installation activity. Our goal is now to investigate if systematic relations exist between these measurements on the one hand, and CL as measured by a questionnaire, on the other. To the extent that these relations exist, we can replace the measurement of computer literacy by questionnaire with the measurement of computer literacy by our software.
Of the five dimensions of computer literacy listed in section 1.2, elements of the Computer Skills, Experience and Information dimensions are measured by our system. It is unsurprising that Skills, Experience and Information lend themselves to automated observation, as these dimensions come the closest to observable actions. Computer Knowledge and Computer Attitude, on the on the other hand, are theoretical constructs within the mind of the user rather than actions observable by the computer, and we expect these to correlate less well with our automated measurements.
Figure 1 shows a fragment of the larger diagram which describes how computer
literacy is measured. This fragment contains all of the elements of computer literacy
that our system can collect. This diagram uses the same functional decomposition
approach to describe how the concepts are related. The colours indicate whether the
concept is a Theoretical Construct (shades of grey), or a Variable containing an
operationalized measurement. The numbers next to the diagram indicate how often the
concept occurs within the 189 documents investigated over the course of the
preceding literature review [
The experiment detailed below aims to investigate how well such a subset of related concepts measures computer literacy compared to the existing practice of using questionnaires.
Computer Literacy 74
Internet Surfing Skill
Search Engines
Internet Activity Type 41 8 5 Availability of IT facilities
6 26 12 7 4
Gaming Frequency Keyboard
Skills Privacy Skills 4 7 4
Availability of Computers
9 Computer 12 at Home
CaotmWpourtker 5 16 36 Computer Experience
Computer Use
Frequency Internet Surfing
Frequency 17 9 Computer
Skills
Application Specific Skills Basic Computer Operation Skills Digital Security
Skills
Computer
Maintenance Skills To validate the system, a two week test is planned. Before this time, subjects will install the experimental system on the computers that they use for their daily work and private life, capturing information about the elements shown in Figure 1. Internet browsing information is captured using browser plugins that cooperate with the experimental system. The system itself measures typing speed, active and focused applications and installed software. All gathered data will be anonymized and sent to a central server using a pre-generated pseudonym as identifier. Alongside this, a questionnaire will be handed to the subjects that will incorporate questions taken from existing, validated computer literacy questionnaires. This questionnaire will contain questions clustered along the same dimensions as were found in existing literature, with additional questions for the dimensions that are measured through the experimental system. This questionnaire will be validated in a separate experiment. The questionnaires are sent and stored using the same pseudonym, linking the experimental system and the questionnaire for comparison purposes while preserving anonymity.
The data gathered by the experimental systems will then be compared to the data gathered by the questionnaires distributed at the start of the experiment. We are currently planning to then perform a cluster analysis along the dimensions of computer literacy. Since this comparison is a non-trivial task however, the details of this analysis are currently still under consideration and beyond the scope of this article.
Should this approach prove effective, we speculate that a similar literature analysis could then be performed to measure other theoretical concepts. This would allow for the usage of cognitive models in persuasive profiles in a novel, real-time manner, improving the detail of persuasion profiles and, by extension, the persuasive power of a system.
The PISA project is sponsored by NWO and KPN under contract 628.001.001.