Social imaging is regarded as a technology to identify and represent social behaviors. By using wearable devices and mixed reality technology, we aim at supporting people who have difficulty making facial expressions and interacting with other people, to express feelings and to act among people. In this paper, we introduce the overview of social imaging technologies and its application to empower people. We define the Social Imaging as the technology to identify and represent social behavior, and also define Human Technology as the technology for understanding and shaping behavior, which brings out latent human capabilities and potential abilities of people.
The Social Imaging is defined as the technology to identify
and represent social behavior. We consider that actions for
interaction with other people according to one’s own
motivation and initiative are regarded as social behavior, and
will establish a methodology to support the function and
capabilities such as movement and functions of the mind and
body, activities of daily lives, and social participation where
people play a role at work, family and in the region, which
are specified in the ICF (International Classification of
Functioning, Disability and Health [
SymCollab '18, March 11, Tokyo, Japan. for people to live in harmony with others as well as intelligence and motor functions.
These technologies include: wearable technology to measure implicit and explicit social behavior, robotics technology that supports actions and modification through interactive feedback to sensory organs, biosignal processing technology that captures human intention and emotional response. We develop these technologies, called human technology, centered on advanced wearable technology that works with people to innovative science and technology fields for understanding and shaping behavior.
HUMAN TECHNOLOGY One of the common global social challenges is the functional rehabilitation/regeneration of people who have difficulties and disabilities in movement. In addition to physically impaired persons, children with developmental disorders such as Autism Spectrum Disorder or Learning Disabilities are increasing around the world. Therefore, research on human-machine systems using robots and wearable devices in the field of medical care, nursing care, healthcare, medical treatment and education support, has been popular in recent years. Examples in the United States are Cyber-Human Systems (CHS) that aims to make a human-machine system where human and machines coordinate with each other. Also in Europe, in Horizon 2020, robot and ICT advancements made for societal implementation is an important issue and has a considerable amount of attention. These kinds of research on the innovation of human-machine systems (Shaping the human-technology frontier) is dramatically increasing importance around the world, and it is positioned as one of the six main strategic targets of NSF (National Science Foundation, US) along with data science, quantum mechanics, and astrophysics. Assistive technology is a term used for devices for people with disabilities, and it supplements the reduced physical and sensory functions. The assistive technology aims to allow people with disabilities to participate more fully in all aspects of life (home, school, and community) and increases their opportunities for education, social interactions, and potential for meaningful employment. On the other hand, the technology for human augmentation or enhancement have already become widely available. With the advancement of biomedical sciences and technologies progress, new ethical, legal and social implications should be considered.
We develop four primary categories of abilities underlying the realization of social behavior (e.g., independent abilities) for the major outcome measures related to the movement and understanding for individuals with disabilities. The theoretical model is presented in Figure 1. There are typical impairment or disabilities as also shown in the Figure. These factors, in combination and alone, determine individuals with disabilities integration into society; an individual’s type and severity of impairment. In the past, shaping behavior or training focused on the voluntary actions rather than the internal mechanism underlying social behavior. As mentioned before, difficulties of children with ASD or developmental disorders are regarded as the problems in the involuntary movement. We develop a methodology to let them notice the involuntary movement as the voluntary movement. The design methodology of these assistance for the deficits in sensory motor function may not suppress the cause but may lead the brain to find new solutions. This is a triggering or signaling based assistance for shaping behaviors. Involuntary movement are represented by the voluntary movement, and then they understand the behavior as the subjective experience.
Voluntary Movement: Action is the process of acting or doing something with his/her own intention. “Voluntary” here means that the human agent initiated the motion of her body based on her (spontaneous) decision and regardless whether there has been a previous external stimulus provoking a reflex or any internal constraint. The voluntary (active) approach to motor learning is more effective than a passive approach because the active approach entails activation of the entire processes related to the intended movement. It is also irrelevant whether motion actually occurs, concerning the release of a controlling neural signal (motor program) which may or may not result in bodily motion (ex. Paralysis). Therefore, persons with difficulty in the voluntary movement are mainly physically impaired. a disability that limits a person's physical capacity to move and coordinate actions although their intention is available. Involuntary Movement: Persons with difficulty in the involuntary movement includes patients with neurodegenerative diseases, such as the tremor, chorea, or myoclonus by Alzheimer's disease, ALS or Parkinson's disease. In the framework of social behavior, we assume that persons with Autism Spectrum Disorder (Neurodevelopmental disorder) are main target group with special needs. One of the main characteristics of person with ASD is related to social problems that include difficulty communicating and interacting with others (DSM-5, Diagnostic and Statistical Manual of Mental Disorders) as well as restricted interests and repetitive behaviors [2]. We follow the approach that ASD starts as a movement disorder [3]). Children with ASD often exhibit clear movement developmental delays and disorganization, labeled and as clumsiness. The function of the brain is to organize movement, combined with attention to the feeling of self [4]. We consider that this is due to the ability to perceive differences and then we need help their brain to perceive differences.
Voluntary Understanding: Actions speak louder than pictures when it comes to understanding what others are doing and feeling [5]. The representative ability of voluntary understanding is considered as awareness, which is the state or level of consciousness where sense data can be confirmed by an observer. Here we consider that it is regarded as the process to understand self and other’s behaviors in a voluntary manner based on the subjective experience. However, current science, psychology, philosophy and neuroscience, has limited explanations for subjective experience. The mechanisms as well as influences of knowledge, attention, and intention on sensory awareness, including perceived timing of events are investigated in this field. This also include to assess the functions of information perceived without awareness in determining what is perceived with awareness.
Involuntary Understanding: In the previous researches on
perception without awareness in cognitive psychology [
SOCIAL IMAGING Social imaging is regarded as a technology to identify and represent social behaviors. By using wearable devices and mixed reality technology, we aim at supporting people who have difficulty making facial expressions and interacting with other people, to express feelings and to act among people.
Technologies that visualize biological functions and behaviors that cannot be seen from the outside are called “imaging”. For example, “brain imaging” is regarded as the visualization of the functions of the brain, which reveals the structural and functional characteristics of each person’s brain. In recent years, several different sensing technologies have been developed about measurement and analysis of human behavior. In this domain, “behavior imaging” has been proposed in computational behavior science including the measurement of a personal viewpoint, which tries to understand each person’s behavior.
The uses of technology in interventions for children with
autism have been widely reported (e.g., [
In this study, we use several wearable devices in order to
measure interactions as shown in Figure 2.; EnhancedTouch
is a bracelet-type device to detect and enhance human-human
physical touch [
DISCUSSION AND CONCLUSION
Our goal is to provide long-term support for children with
special needs, who have social impairments, and to evaluate
the effects of this support. Throughout long-term feasibility
study, we have measured the smiles of children with ASD in
a quantitative manner and have reported that their smiles
might facilitate social positive behaviors [
Empathic design is an effective approach for paying attention to the user's feelings toward a solution in the practical site such as schools, after-school playroom or home. We need deep understanding of the existing environment, in particular, the unique goals and players of the organization. We need define the problem and potential areas of impact as well as the allocated resources and operational process. Then, we could align the current technology state to the target environment, and implement solutions into it together with players. Individuals with disabilities will be involved in each activity, as joint investigators of this project.
Helping children with developmental disorders (ASD, LD) to interact with their peers or caregivers with given visual aids. This helps these children to be aware of social cues, a behavior that is considered critical for the development of social interaction. Providing them with social signals by using visual or auditory feedback with wearable technologies. In addition, providing an automatic, objective measure of wellbeing could be used to support therapists, patients, engineers and doctors in order to improve therapeutic activities and devices, and also to understand spontaneous laughter and smiles. The developed wearable device for reading positive expressions are used. Based on the further investigations together with verbal information, providing quantitative measure of laughter and smiles recorded during long-term therapeutic interventions to quantify and infer the user’s affective state in order to support psychological or medical professionals.
ACKNOWLEDGMENTS This research was supported by JST-CREST, Japan (No. JPMJCR14E2). The authors would like to thank all colleagues, in particular, teachers and students of special needs school at Otsuka for the FUTUREGYM project.