=Paper=
{{Paper
|id=Vol-2327/UIoT6
|storemode=property
|title=Impact of Voice-based Interaction on Learning Practices and Behavior of Children
|pdfUrl=https://ceur-ws.org/Vol-2327/IUI19WS-IUIoT-6.pdf
|volume=Vol-2327
|authors=Subhasree Sengupta,Radhika Garg
|dblpUrl=https://dblp.org/rec/conf/iui/SenguptaG19
}}
==Impact of Voice-based Interaction on Learning Practices and Behavior of Children==
https://ceur-ws.org/Vol-2327/IUI19WS-IUIoT-6.pdf
Impact of Voice-based Interaction on Learning Practices and
Behavior of Children
Subhasree Sengupta Radhika Garg
School of Information Studies School of Information Studies
Syracuse University Syracuse University
Syracuse, NY, USA Syracuse, NY, USA
susengup@syr.edu rgarg01@syr.edu
ABSTRACT 1 INTRODUCTION
Smart devices have become an integral part of the everyday lives Today, various types of smart devices are deeply integrated in our
of children. Today, children can even use voice-based interactions day to day lives. The use of technology has increased not only for
to interact with devices for a wide range of activities. Previous adults but also for children, be it as a source of entertainment or as
research has shown that voice-driven interfaces have a potential a learning aid. So much so, that the exposure and use of technology
to offer a potent new mechanism for teaching, engaging, and sup- has been considered as a crucial influence on the process of learn-
porting children in daily life. Our paper, therefore, argues that it is ing and development of children [5]. Bower and Sturman demon-
critical not only to investigate how children use voice-based inter- strated that wearable devices offer a range of pedagogical uses
actions to communicate with devices (e.g., smart speakers) but also (in-situ contextual information, recording, simulation, communica-
the nature of relationships that children form with these devices, tion, first-person view, in-situ guidance, feedback, distribution and
the influence such use has on children’s learning and behavior, gamification), afford benefits to educational quality (engagement,
and the role that parents or guardians play in deciding the norms efficiency, and presence), and provide logistical advantages (hands-
of use for children. We also propose to explicitly and intricately free access and free up space) in a class room setting [2]. More
investigate complexities in use and its impact relative to entangled recently, smart devices (e.g., smart phones, tablets, smart speakers)
identities (conveyed through overlapping attributes of gender, eth- have started to offer conversational assistants (e.g., Amazon Alexa,
nicity, race, class) and larger social systems. To this end, we propose Siri, and Google Now) that lend flexible means of interacting with
to use Social Learning Theory to understand how children learn the device. Due to the presence of such voice assistants, children no
through observing and interacting with smart devices, specifically longer need to read or write to be able to interact with the devices
using voice-based commands. Methodologically, we will conduct [9]. As the amount of background information a child needs to use
participatory design sessions and follow-up interviews to get a these devices has reduced, it can have an impact in the information
nuanced understanding of how children mentally contextualize seeking, behavioral (e.g., children might imitate and emulate certain
voice-enabled smart devices and how social influence (e.g., parental characteristics of these devices), and learning practices pursued by
expectation/norms), social function of identification (e.g., children’s children and the factors that affect these practices. Hence, in this pa-
emotional connection with technology), and learning goals impact per we argue that it is critical to investigate how and why children
their usage patterns. are using these devices (e.g., voice-connected speakers), and the
influence voice-based interactions with devices has on children’s
CCS CONCEPTS behavior and learning practices.
We propose to investigate this issue through the lens of Ban-
• Human-centered computing → User studies; Empirical stud-
dura’s Social Learning Theory (SLT). It explains ‘observational
ies in HCI; Scenario-based design; Participatory design.
learning’ in terms of how people learn through observing othersfi
behavior, attitudes, and the outcomes (penalty or reward) one might
KEYWORDS incur due to such a behavior [1]. However, SLT is a complex and
Voice-based interactions; Children’s behavior and learning prac- subjective concept with many different facets to it, exploring all
tices; Social learning theory; Parasocial relationships of which is beyond the scope of this study. Therefore, to under-
stand ‘observational learning’ our study centers around the three
ACM Reference Format: social factors provided by Over et al [10]. These three factors are:
Subhasree Sengupta and Radhika Garg. 2019. Impact of Voice-based Interac- social function of identification (parasocial relationships), the type
tion on Learning Practices and Behavior of Children. In Joint Proceedings of of role that children associate with voice assistants; learning goals,
the ACM IUI 2019 Workshops, Los Angeles, USA, March 20, 2019. ACM, New the type of learning tasks that children use voice assistants for;
York, NY, USA, 3 pages.
social norms and customs, particularly focussing on the role that
parents/guardians play in regulating children’s use of technology.
Therefore, we will investigate three fundamental research questions
in this paper:
IUI Workshops’19, March 20, 2019, Los Angeles, USA
Copyright © 2019 for the individual papers by the papers’ authors. Copying permitted
for private and academic purposes. This volume is published and copyrighted by its
editors. • RQ1: What are the type of parasocial relationships that child
form with the voice-enabled smart devices?
IUI Workshops’19, March 20, 2019, Los Angeles, USA Sengupta and Garg, et al.
• RQ2: What are the type of learning objectives or tasks that learning goals of children. For example, if a child does not know
children are interested to use voice-enabled smart devices how to do something and wants to learn about it, he/she may ask the
for and how parasocial relationships may or may not impact device for help and use the information gained from these devices
those? to perform that task. Preliminary work by Lovato et al. highlighted
• RQ3: How do social norms and customs (especially those that children in general either ‘explore’ voice assistants such as
instilled/followed by parents and guardians) affect the way Siri and Google Now or use them to ‘seek new information’ [9]. In
children use voice-connected smart devices? ‘exploration’ children use the voice assistants for fun and are even
able to develop a bond with the voice assistants. In ‘information
To answer these questions we aim to conduct Participatory De-
seeking’ children use the voice assistants to find facts and develop
sign (PD) sessions and set of interviews with design groups consist-
knowledge base. Both these forms of use have impact on children’s
ing of children from different age groups: 7- 9 years, 10-12 years
development. However, the main source of data for their study
and 13-17 years. This will enable us to investigate the role age and
were Youtube videos of children’s activity, which might not be
gender of children within the context of our research questions. For
representative of children’s actual usage patterns. We aim to add
example, we will explore if the nature of parasocial relations/role
to this work, by investigating actual logs of voice commands to
that younger children associate with voice-enabled smart devices
understand the categories of use. Further, usage patterns can greatly
differ from those that older children associate with these devices.
differ by age and gender, thus while annotating the voice commands
we will also explore how usage differs based on age and gender. In
2 BACKGROUND AND RELATED WORK order to understand this better we will conduct a design session
In this section we first discuss the framework by Over et al [10] to investigate the kind of devices children would like to use for
that acts as a foundation of our proposed study. Then, we discuss varying learning goals that they might have the role of parasocial
prior studies that have looked into what type of activities children relationships in this process.
are using voice-enabled devices for, parental role in use of voice- Role of parents: The final influencing factor that Over et al. [10]
enabled devices by children, and type and influence of parasocial put forth is ‘social influence’, which comprises of social customs,
relationships that children develop with such devices. expectations, and norms that might affect the way children use or
Oven et al. highlighted three crucial factors that impact the selec- communicate with a device. Particularly we will focus on the role
tivity in ’observational learning’ (i.e the fact that people selectively parents or guardians play a key in role in introducing a smart device
choose to imitate or emulate certain behavior they observe) [10]. to children and establishing norms of use and the extent of use, and
These factors were used to operationalize SLT to be able to under- children’s use of devices might also be influenced by the way their
stand children’s use, how they relate with these devices, and how parents or other older members of the family use these devices.
it impacts their learning practices. Cheng et al. presented four roles parents play in helping children
Parasocial relationships: The first factor that Over et al. [10] communicate with voice-controlled devices [4]. Parents may also
stated is ‘social function of identification’ that explains that chil- help establish boundaries for device usage by children as illustrated
dren emulate and establish an emotional connection with those by related work (e.g., [8]). They may control the amount of time,
they feel they resemble or want to be like. For example, they may the type of content and the nature of interactions children may
find the voice of a device relatable or they may find a character in have with these devices thereby influencing the learning practices
a game that they play on a device relatable and start to personate of children. Hiniker et al. demonstrated that parents are vital in
the characteristics of those. More importantly, children sometimes scaffolding children’s use of a novel/relatively newer technology
assert roles (such as friend, mentor, pet) to these devices thereby [8]. Therefore, we propose to expand on prior work by including
personifying them and forming a relationship with these devices. parents in PD sessions to identify how they support or regulate
These sort of connections have also been termed as parasocial re- children’s use of voice-connected devices for learning, and how
lationships (i.e., one-sided, emotionally driven relationships that this differs by age and gender of children.
children develop with media characters) [3, 7]. Druga et al. demon-
strated that the ability to have voice-based interactions, elements 3 PROPOSED METHOD
of social realism, and human-like characteristics makes these smart
Our study will be conducted in following two stages:
devices more relatable and easy to use for children [6]. Brunick et
al. [3] highlighted how parasocial relationships can be useful for
developing educational tools for children by embedding in intel- 3.1 Historical Log Analysis
ligence agents ability to generate parasocial interactions, such as In this stage we will deploy a survey on Amazon Mechanical Turk
conversational timing and response personalization. Gray et al [7] (AMT) to collect children’s voice history logs comprising of their
emphasized that factors such as social realism and personification interactions with smart speakers. The survey will also consist of
should be considered when designing an intelligent agent for chil- questions on on family structure, number of children in families,
dren. Therefore, we propose to explore the type of relationships types of smart speakers. The primary goal of the survey is to an-
children form particularly with voice-enabled smart devices and notate voice history logs to get an understanding of the type of
how that impacts the type of learning goals that children use these activities that children use smart speakers for and get an estimate
devices for. of the percentage of those used for learning. Further, we will also
Learning goals: The second factor that Over et al. [10] put forth analyze how the usage patterns, learning tasks differ by age and
is ‘learning goals’, which includes self-established or self-motivated gender of children.
Impact of Voice-based Interaction on Learning Practices and Behavior of Children IUI Workshops’19, March 20, 2019, Los Angeles, USA
3.2 Participatory Design Sessions devices and how social influence (e.g., parental expectation/norms),
In second stage we will employ Cooperative Inquiry [11] for con- social function of identification (e.g., children’s emotional connec-
ducting PD sessions that will focus on co-designing devices/technology tion with technology), and learning goals impact their usage pat-
with children that they might like to use for learning. Each of these terns. To this end, we propose to employ both historical log analysis
sessions will be divided in two parts, the first part called circle time and participatory design sessions.
will be used to help the participant better contextualize the task
they are about to do in the session and the second part will be the REFERENCES
[1] Albert Bandura. 1977. Social learning theory. Prentice Hall.
actual design prompt based on which the participants will perform [2] Matt Bower and Daniel Sturman. 2015. What are the educational affordances of
a design activity. wearable technologies? Computers & Education 88 (2015), 343–353.
Design Session 1 (DS1) [3] Kaitlin L Brunick, Marisa M Putnam, Lauren E McGarry, Melissa N Richards, and
Sandra L Calvert. 2016. Children’s future parasocial relationships with media
Prior research has shown that children perceive interactive media characters: the age of intelligent characters. Journal of Children and Media 10, 2
characters as enjoyable companions and develop different paraso- (2016), 181–190.
cial relationship with them. Therefore, the design session will begin [4] Yi Cheng, Kate Yen, Yeqi Chen, Sijin Chen, and Alexis Hiniker. 2018. Why Doesn't
It Work? Voice-Driven Interfaces and Young Children's Communication Repair
with the circle time (15 minutes), where we will ask participants to Strategies. In Proceedings of the 17th ACM Conference on Interaction Design and
share with us their favorite cartoon or media character. The aim of Children. ACM, 337–348.
[5] Cynthia Chiong and Carly Shuler. 2010. Learning: Is there an app for that. In
circle time is to ask “question of the day” to get adults and children Investigations of young children's usage and learning with mobile devices and apps.
started. After that we will ask them to design an interactive tech- New York: The Joan Ganz Cooney Center at Sesame Workshop. 13–20.
nology/device to identify kind of roles children would like the such [6] Stefania Druga, Randi Williams, Cynthia Breazeal, and Mitchel Resnick. 2017.
Hey Google is it OK if I eat you?: Initial Explorations in Child-Agent Interaction.
devices to take, specifically as they use them for various learning In Proceedings of the 2017 Conference on Interaction Design and Children. ACM,
tasks (e.g explore unknown facts, improve their language skills, or 595–600.
to help them to hone their deductive reasoning). We will utilize [7] James H Gray, Emily Reardon, and Jennifer A Kotler. 2017. Designing for Paraso-
cial Relationships and Learning: Linear Video, Interactive Media, and Artificial
Bags-of-stuff, Big Paper and Layered Elaboration PD techniques [11]. Intelligence. In Proceedings of the 2017 Conference on Interaction Design and Chil-
Through such activities we will investigate two things: 1) The type dren. ACM, 227–237.
[8] Alexis Hiniker, Bongshin Lee, Kiley Sobel, and Eun Kyoung Choe. 2017. Plan &
of parasocial roles that children see these smart devices to take, play: supporting intentional media use in early childhood. In Proceedings of the
and 2) the connection between the parasocial role and the type of 2017 Conference on Interaction Design and Children. ACM, 85–95.
learning task. [9] Silvia Lovato and Anne Marie Piper. 2015. Siri, is this you?: Understanding
young children’s interactions with voice input systems. In Proceedings of the 14th
Design Session 2 (DS2) International Conference on Interaction Design and Children. ACM, 335–338.
In this design session we will elicit information regarding how [10] Harriet Over and Malinda Carpenter. 2012. Putting the social into social learning:
children think about using different speech agents (e.g, Amazon explaining both selectivity and fidelity in children’s copying behavior. Journal of
Comparative Psychology 126, 2 (2012), 182.
Alexa, Google Now) for various learning tasks, using Stickies PD [11] Greg Walsh, Elizabeth Foss, Jason Yip, and Allison Druin. 2013. FACIT PD: a
[11] technique. The circle time prompt for this session will ask framework for analysis and creation of intergenerational techniques for par-
ticipatory design. In proceedings of the SIGCHI Conference on Human Factors in
participants how they (would) use a device/technology to explore Computing Systems. ACM, 2893–2902.
a fact they wish to learn about. The design prompt ask participants
to note what they liked, disliked or would like to improve based on
the current devices, using the design technique of Stickies.
Design Session 3 (DS3)
In the final session we will use Stickies [11], Layered Elaboration
[11] as design methods to have children and parent revisit design
ideas that children built in DS1 and build them with the help of
parents/guardians. One of the member of the research team will
present the ideas generated during DS1 through storyboards. The
circle time prompt will have participants think about how parents
influence children’s use of technology. For the design prompt, the
parents and children will be then asked to explain their likes, dis-
likes, and further design ideas. In such a way parents and children
would work together to make the storyboards better based on each
other’s ideas and opinions. For example, parents might want to
include the possibility of regulating children’s use (e.g., permitted
duration, tone of the device) in the designs proposed by children.
4 CONCLUSION
Our position paper proposes to investigate how voice-based interac-
tions with smart devices are affecting or can affect learning practices
of children. Particularly, we use the three factors by Oven et al. to
operationalize the use of SLT as a tool to answer our research ques-
tions on how children mentally contextualize voice-enabled smart