=Paper=
{{Paper
|id=Vol-2068/symcollab6
|storemode=property
|title=Social Imaging and Human Technology for Empowering People
|pdfUrl=https://ceur-ws.org/Vol-2068/symcollab6.pdf
|volume=Vol-2068
|authors=Kenji Suzuki
|dblpUrl=https://dblp.org/rec/conf/iui/Suzuki18
}}
==Social Imaging and Human Technology for Empowering People==
https://ceur-ws.org/Vol-2068/symcollab6.pdf
Social Imaging and Human Technology
for Empowering People
Kenji Suzuki
University of Tsukuba
Tsukuba, Japan
kenji@ieee.org
ABSTRACT for people to live in harmony with others as well as
Social imaging is regarded as a technology to identify and intelligence and motor functions.
represent social behaviors. By using wearable devices and
mixed reality technology, we aim at supporting people who These technologies include: wearable technology to measure
have difficulty making facial expressions and interacting implicit and explicit social behavior, robotics technology that
with other people, to express feelings and to act among supports actions and modification through interactive
people. In this paper, we introduce the overview of social feedback to sensory organs, biosignal processing technology
imaging technologies and its application to empower people. that captures human intention and emotional response. We
We define the Social Imaging as the technology to identify develop these technologies, called human technology,
and represent social behavior, and also define Human centered on advanced wearable technology that works with
Technology as the technology for understanding and shaping people to innovative science and technology fields for
behavior, which brings out latent human capabilities and understanding and shaping behavior.
potential abilities of people. HUMAN TECHNOLOGY
Author Keywords One of the common global social challenges is the functional
Wearable Device, Affective Computing, Computational rehabilitation/regeneration of people who have difficulties
Behavioral Science, Autism Spectrum Disorders and disabilities in movement. In addition to physically
impaired persons, children with developmental disorders
ACM Classification Keywords such as Autism Spectrum Disorder or Learning Disabilities
H.5.2 [User Interfaces]: User-centered design. are increasing around the world. Therefore, research on
INTRODUCTION human-machine systems using robots and wearable devices
We challenge to realize a future society where individuals in the field of medical care, nursing care, healthcare, medical
with their own initiative will benefit from social interaction treatment and education support, has been popular in recent
with others with the aid of technologies, which brings out years. Examples in the United States are Cyber-Human
latent human capabilities and potential abilities of people. Systems (CHS) that aims to make a human-machine system
The technology can support the understanding and shaping where human and machines coordinate with each other. Also
social behaviors and the assisted lifestyles will then become in Europe, in Horizon 2020, robot and ICT advancements
widely available. made for societal implementation is an important issue and
has a considerable amount of attention. These kinds of
The Social Imaging is defined as the technology to identify research on the innovation of human-machine systems
and represent social behavior. We consider that actions for (Shaping the human-technology frontier) is dramatically
interaction with other people according to one’s own increasing importance around the world, and it is positioned
motivation and initiative are regarded as social behavior, and as one of the six main strategic targets of NSF (National
will establish a methodology to support the function and Science Foundation, US) along with data science, quantum
capabilities such as movement and functions of the mind and mechanics, and astrophysics. Assistive technology is a term
body, activities of daily lives, and social participation where used for devices for people with disabilities, and it
people play a role at work, family and in the region, which supplements the reduced physical and sensory functions. The
are specified in the ICF (International Classification of assistive technology aims to allow people with disabilities to
Functioning, Disability and Health [1]). Sociality or social participate more fully in all aspects of life (home, school, and
skill, which can be represented by understanding facial community) and increases their opportunities for education,
expressions, talking to someone eye-to-eye, understanding social interactions, and potential for meaningful employment.
others’ stances and feelings, cooperating with each other, On the other hand, the technology for human augmentation
expressing what you want to do, adjusting to the environment or enhancement have already become widely available. With
by reducing your own desire, is an important characteristic the advancement of biomedical sciences and technologies
progress, new ethical, legal and social implications should be
© 2018. Copyright for the individual papers remains with the authors. considered.
Copying permitted for private and academic purposes.
SymCollab '18, March 11, Tokyo, Japan.
We develop four primary categories of abilities underlying
the realization of social behavior (e.g., independent abilities)
for the major outcome measures related to the movement and
understanding for individuals with disabilities. The
theoretical model is presented in Figure 1. There are typical
impairment or disabilities as also shown in the Figure. These
factors, in combination and alone, determine individuals
with disabilities integration into society; an individual’s type
and severity of impairment. In the past, shaping behavior or
training focused on the voluntary actions rather than the
internal mechanism underlying social behavior. As
mentioned before, difficulties of children with ASD or
developmental disorders are regarded as the problems in the
involuntary movement. We develop a methodology to let
them notice the involuntary movement as the voluntary
Figure 1. Human movement, cognitive characteristics and
movement. The design methodology of these assistance for disabilities: Motion-perception / voluntary-involuntary axes
the deficits in sensory motor function may not suppress the
cause but may lead the brain to find new solutions. This is a
triggering or signaling based assistance for shaping differences and then we need help their brain to perceive
behaviors. Involuntary movement are represented by the differences.
voluntary movement, and then they understand the behavior
Voluntary Understanding: Actions speak louder than
as the subjective experience.
pictures when it comes to understanding what others are
Voluntary Movement: Action is the process of acting or doing and feeling [5]. The representative ability of voluntary
doing something with his/her own intention. “Voluntary” understanding is considered as awareness, which is the state
here means that the human agent initiated the motion of her or level of consciousness where sense data can be confirmed
body based on her (spontaneous) decision and regardless by an observer. Here we consider that it is regarded as the
whether there has been a previous external stimulus process to understand self and other’s behaviors in a
provoking a reflex or any internal constraint. The voluntary voluntary manner based on the subjective experience.
(active) approach to motor learning is more effective than a However, current science, psychology, philosophy and
passive approach because the active approach entails neuroscience, has limited explanations for subjective
activation of the entire processes related to the intended experience. The mechanisms as well as influences of
movement. It is also irrelevant whether motion actually knowledge, attention, and intention on sensory awareness,
occurs, concerning the release of a controlling neural signal including perceived timing of events are investigated in this
(motor program) which may or may not result in bodily field. This also include to assess the functions of information
motion (ex. Paralysis). Therefore, persons with difficulty in perceived without awareness in determining what is
the voluntary movement are mainly physically impaired. a perceived with awareness.
disability that limits a person's physical capacity to move and
Involuntary Understanding: In the previous researches on
coordinate actions although their intention is available.
perception without awareness in cognitive psychology [6],
Involuntary Movement: Persons with difficulty in the stimuli are perceived even when observers are unaware of
involuntary movement includes patients with the stimuli. The involuntary cognitions in everyday life
neurodegenerative diseases, such as the tremor, chorea, or without a pre-determined focus is [7] Especially, an
myoclonus by Alzheimer's disease, ALS or Parkinson's involuntary response to an unexpected and sudden stimulus
disease. In the framework of social behavior, we assume that is closely linked to affective processing [8]. Therefore, we
persons with Autism Spectrum Disorder focus on the emotion in this domain. In recent years, there
(Neurodevelopmental disorder) are main target group with has been an increased interest in detecting and supporting
special needs. One of the main characteristics of person with human physical and psychological wellbeing. A potential
ASD is related to social problems that include difficulty source of information can be the experiences of pleasure
communicating and interacting with others (DSM-5, expressed though facial expressions, which can be used to
Diagnostic and Statistical Manual of Mental Disorders) as infer a person’s emotions and give insights into their internal
well as restricted interests and repetitive behaviors [2]. We state and happiness.
follow the approach that ASD starts as a movement disorder SOCIAL IMAGING
[3]). Children with ASD often exhibit clear movement Social imaging is regarded as a technology to identify and
developmental delays and disorganization, labeled and as represent social behaviors. By using wearable devices and
clumsiness. The function of the brain is to organize mixed reality technology, we aim at supporting people who
movement, combined with attention to the feeling of self [4]. have difficulty making facial expressions and interacting
We consider that this is due to the ability to perceive
Figure 2. Devices and technologies related to Social Imaging: (a)(b) Wearable devices, (c) Mixed Reality based FUTUREGYM with
a large scale floor projection system and (d) Paired Device
with other people, to express feelings and to act among autism spectrum disorders (ASD). It is known that children
people. with ASD have difficulty in social interactions involving
understanding and using facial expressions. Therefore,
Technologies that visualize biological functions and
support of social interaction at the early stage of children’s
behaviors that cannot be seen from the outside are called
lives has drawn considerable attention in the field of child
“imaging”. For example, “brain imaging” is regarded as the
development. However, even though there have been several
visualization of the functions of the brain, which reveals the
studies about motivations for facial expressions and
structural and functional characteristics of each person’s
communication towards people, it is difficult to carry out
brain. In recent years, several different sensing technologies
objective evaluations since it is not easy to make quantitative
have been developed about measurement and analysis of
measurements in daily lives and at school. In order to
human behavior. In this domain, “behavior imaging” has
understand these kinds of behaviors, it is indispensable to
been proposed in computational behavior science including
consider “social imaging”, whose aim is to understand
the measurement of a personal viewpoint, which tries to
people’s social behaviors.
understand each person’s behavior.
DISCUSSION AND CONCLUSION
The uses of technology in interventions for children with Our goal is to provide long-term support for children with
autism have been widely reported (e.g., [9]). Previous studies special needs, who have social impairments, and to evaluate
have shown effectiveness of technology-based intervention the effects of this support. Throughout long-term feasibility
for teaching academic skills [10] or social skills [11] [12]. study, we have measured the smiles of children with ASD in
Our emerging data highlights the effects of the technology a quantitative manner and have reported that their smiles
use in measuring and promoting of positive social behaviors might facilitate social positive behaviors [18]. We are going
(emotion recognition, face-to-face, approaching to the peers, to develop an evidence-based developmental support system
and touching with peers) in children with ASD. especially for shaping social behaviors of children with ASD.
In this study, we use several wearable devices in order to Deeper insight into their behavior, and elucidating the
measure interactions as shown in Figure 2.; EnhancedTouch development of social interaction among people at the early
is a bracelet-type device to detect and enhance human-human stage of their lives, is very significant for learning social
physical touch [13]. Enhanced Reach is a bibs-type device to skills such as joint attention, imitation, language
understand group dynamics and facilitate interaction [14]. understanding, and understanding the development of
Facelooks is a head-band type device to detect and enhance communication skills. In this study, we aim to elucidate the
face-to-face behavior. FUTUREGYM is an interactive environmental conditions in which children with ASD can
school gymnasium with a large-scale interactive floor demonstrate their abilities to the fullest, and to the
projection system in a school setting in order to develop establishment of an inclusive developmental support system,
social interaction skills such as prosocial behaviors including where we support forming creativity and sociality including
helping and cooperating behaviors [15]. We design and cognitive functions and language functions.
implement an ergonomic wearable device, with high Empathic design is an effective approach for paying attention
reliability, for reading positive expressions from facial EMG to the user's feelings toward a solution in the practical site
signals. The physical and physiological signals of the such as schools, after-school playroom or home. We need
emotions such as smiles caused by these interactions deep understanding of the existing environment, in particular,
[16][17]. New technologies to identify and represent social the unique goals and players of the organization. We need
interactions are proposed as “social imaging.” The measured define the problem and potential areas of impact as well as
social behaviors are stored in a database as personal data, the allocated resources and operational process. Then, we
which includes physiological and behavioral characteristics. could align the current technology state to the target
There is a high demand in society for the establishment of an environment, and implement solutions into it together with
effective developmental support method for children with players. Individuals with disabilities will be involved in each
activity, as joint investigators of this project.
Helping children with developmental disorders (ASD, LD) 9. Joe McCleery. 2015. Comment on Technology-Based
to interact with their peers or caregivers with given visual Intervention Research for Individuals on the Autism
aids. This helps these children to be aware of social cues, a Spectrum, J Autism Dev Disord, 45, 12: 3832-5.
behavior that is considered critical for the development of 10. Victoria Knight, Bethany R. McKissick, Alicia
social interaction. Providing them with social signals by Saunders. 2013. A review of technology-based
using visual or auditory feedback with wearable technologies. interventions to teach academic skills to students with
In addition, providing an automatic, objective measure of autism spectrum disorder, J Autism Dev Disord, 43, 11:
wellbeing could be used to support therapists, patients, 2628-48.
engineers and doctors in order to improve therapeutic
activities and devices, and also to understand spontaneous 11. Florence D. DiGennaro Reed, Sarah R. Hyman, Jason
laughter and smiles. The developed wearable device for M. Hirst. 2011. Applications of Technology to Teach
reading positive expressions are used. Based on the further Social Skills to Children with Autism, Research in
investigations together with verbal information, providing Autism Spectrum Disorders, 5, 3: 1003-1010.
quantitative measure of laughter and smiles recorded during 12. Anne M. Donnellan, David A. Hill, Martha R. Leary.
long-term therapeutic interventions to quantify and infer the 2012. Rethinking autism: implications of sensory and
user’s affective state in order to support psychological or movement differences for understanding and support,
medical professionals. Front Integr Neurosci., 6:124.
ACKNOWLEDGMENTS 13. Kenji Suzuki, Taku Hachisu, Kazuki Iida, 2016,
This research was supported by JST-CREST, Japan (No. EnhancedTouch: A Smart Bracelet for Enhancing
JPMJCR14E2). The authors would like to thank all Human-Human Physical Touch, In Proceedings of
colleagues, in particular, teachers and students of special SIGCHI Conference on Human Factors in Computing
needs school at Otsuka for the FUTUREGYM project. Systems (CHI’16), 1282-1293.
REFERENCES 14. Asaki Miura, Takashi Isezaki and Kenji Suzuki. 2013.
1. World Health Organization. 2001. International Social Playware with an Enhanced Reach for
Classication of Functioning, Disability and Health Facilitating Group Interaction, In Proceedings of CHI
(ICF), Geneva, World Health Organization. '13 Extended Abstracts on Human Factors in
Computing Systems, 1155-1160.
2. Dimitris Bolis, Leonhard Schilbach. 2017. Observing
and Participating in Social Interactions: Action 15. Issey Takahashi, Mika Oki, Baptiste Bourreau, Itaru
Perception and Action Control Across the Autistic Kitahara, Kenji Suzuki. 2017. FUTUREGYM: A
Spectrum, Dev Cogn Neurosci, gymnasium with interactive floor projection for
doi:10.1016/j.dcn.2017.01.009. children with special needs, Int J Child Comput
Interact, (in press)
3. Victoria L. Chester, Matthew Calhoun 2012. Gait
symmetry in children with autism, Autism Research 16. Anna Gruebler and Kenji Suzuki. 2014. Design of a
and Treatment, ID:576478. Wearable Device for Reading Positive Expressions
from Facial EMG Signals, IEEE Trans Affect Comput,
4. David Franklin, Daniel Wolpert. 2011. Computational
mechanisms of sensorimotor control, Neuron, 72, 5, 3: 227-237.
3 :425-442. 17. Monica Perusquia-Hernandez, Masakazu Hirokawa,
Kenji Suzuki. 2017. A Wearable Device for Fast and
5. Charles Darwin. 1872. The expression of the emotions
Subtle Spontaneous Smile Recognition, IEEE Trans
in man and animals, London, John Murray.
Affect Comput, 8, 4: 522-533.
6. Philip M. Merikle, Daniel Smilek, John D. Eastwood.
2001. Perception without awareness: perspectives from 18. Atsushi Funahashi, Anna Gruebler, Takashi Aoki,
Hideki Kadone and Kenji Suzuki. 2014. The Smiles of
cognitive psychology, Cognition, 79, 1-2: 115-134.
a Child with Autism Spectrum Disorder During an
7. Julie Krans, June de Bree, Michelle L. Moulds. 2015. Animal-assisted Activity May Facilitate Social Positive
Involuntary Cognitions in Everyday Life: Exploration Behaviors - Quantitative Analysis with Smile-detecting
of Type, Quality, Content, and Function, Front Interface, J Autism Dev Disord, 44, 3: 685-693.
Psychiatry, 6:7.
8. Peter J. Lang, Margaret M. Bradley, Bruce N.
Cuthbert. 1990. Emotion, attention, and the startle
reflex. Psychol. Rev., 97: 377-395.