This study sheds light on J. J. Gibson's theory of “affordances” by presenting a competent visualization of how infants perceive and act in their everyday home environments, with the aim of preventing accidents and promoting development. Gibson described affordance as a unification of an environment's action-relevant properties and an animal's way of life. To visualize infants' affordances in their natural settings, this study adopted Augmented Reality (AR) technology, which enhances a user's perceptions of and interactions with his/her real environment. In this study, we developed an AR application for iPads and iPhones. We conducted longitudinal observations of two infants from 4 to 12 months of age at their homes in Tokyo, Japan. The findings of these observations revealed 10 typical objects that appeared most frequently among the everyday items found in people's homes. Those objects were assigned to vision-based markers in order of the actor's age and were incorporated into the application. This AR application prototype system was implemented in two ways: (a) Objective-C and OpenCV and (b) openFrameworks and Vuforia. Our informal user study showed that (a) was more suitable regarding recognition time. However, (b) was much broader in terms of the interface design.
Affordance is a fact in the environment as well as a fact embodied by the act. The ecological psychologist James Gibson proposed the theory of affordance referring to the mutuality of animal and environment. The most familiar living environment is a "home," which includes various affordances. The process of perceiving affordances is not the same for all people sharing the same space and objects as others’ bodily features differ from ours. Still more, an infant differs from an adult.
Most studies have examined the affordances for infants
outside of their homes, either outdoors or in a laboratory
setting
Empirical research concerning individual affordance has
been advanced based on a series of pioneering studies by
In this study, we explored the suitable method for an
everyday home environment in order to better visualize
infants’ developmental resources. Technology that enables
the visualization of our environments has remarkable
current real-world applications. First developed over 40
years ago, Augmented Reality (AR) is one such technology
that enables the seamless merging of virtual content with the
real world
Therefore, the purpose of this study was twofold: 1) to present an analysis of the affordances of infants’ home environments based on longitudinal observations during the first year of life and 2) to propose a new way of visualizing infants’ affordances by creating an AR mobile application prototype for iPhones and iPads to provide parents and surrounding adults with better understanding.
The longitudinal observations were conducted with two healthy male Japanese babies, who were observed in their homes during the first 18 months of their lives. Each family used a digital video camera to record their baby’s actions at their home on a weekly basis. This application focused on data collected between 4 to 12 months of age (the total recorded time was 75 h).
(1) Paper prototypes. (2) Designing the app using Adobe Illustrator CS6, Photoshop CS6, and Shade. (3) Designing the marker recognition using OpenCV and Vuforia. (4) Writing an iOS application using Objective-C. GUI and user interactions were developed using Objective-C.
Augmented Reality Movies (1) All Augmented Reality movies were based on the longitudinal observations of two babies. (2) Hand line drawings that were created based on the recordings were converted to digital data using Photoshop and Lightroom. They were saved as 448 px × 336 px images. The sequences were animated in Photoshop using the Animation Timeline and were exported as QuickTime Movies. Figure 1 shows the example of an infant interacting with a cabinet at age 11 months, 17 days.
This study proposed an AR application prototype for iPhones and iPads capable of visualizing the affordances of infants’ indoor environments during the first year of life. On the basis of these observations, we extracted objects. According to Gibson’s classification of objects, we divided them into two patterns: detached and attached objects related to the home’s room layouts (Table 1).
In this prototype, 10 objects—a bathtub, cabinet, chair, cushion, door, futon, sofa, table, threshold, and wall—were extracted as the most frequently found everyday things in the infants’ homes. Based on the longitudinal observation, infants’ perceptions and actions toward these 10 objects were observed and included not only positive developmental possibilities but also risks of troubles— positive and negative affordances for infants in the home. Concrete examples of those affordances are described in Table 2. These descriptions were embedded in the application as reference for the app’s users. The users were able to learn about a home’s contents according to three categories: plan of room layout, attached objects, and detached objects. Figure 2 is an example of a detached object. The user can tap to select a category and items, and pop-up windows show the properties of the object for infants. The user can then tap the window’s close button or swipe icons to see the next items.
Regarding marker recognition, two types of patternmarker codes (Figures 3 (a) and 3 (b)) were created. Figure 3 (a) was made by combining dots and objects’ icons using OpenCV. Figure 3 (b) was made using textures and objects’ icons using OpenCV. The textures reflected the objects’ features.
The AR system recognizes these markers’ codes from the captured camera images. Both could be attached to realworld objects at virtually no cost since the marker codes can be printed using laser or inkjet printers.
Figure 4 introduces the overall information flow of the system. The user begins by setting an arbitrary place marker on the real objects in the real situation. After starting the AR application, the user can capture the marker with his/her device camera. As soon as the camera recognizes the marker, the user can select the age of the actor. Then, the AR movie starts to play (Figure 5).
Each marker includes several AR movies based on 4–12month-old babies’ actions toward the object. Each movie is color-coded according to the actor’s age. By selecting both object and age, the user is able to discern the differences in the infants’ developmental processes. The application enables the user to simulate infants’ affordances toward various objects in their own environments.
Informal user observations formed the basis for this prototype. These observations indicated that marker recognition time is one of the most effective points to use in the prototype development. We measured the velocities of recognition time to compare two types of AR markers under two lighting conditions: (A) fluorescent light and (B) flat surface fluorescent light.
Tables 3 (A) and 3 (B) show the rate of change of velocity related to the illuminance of the surface (vertical and horizontal directions) and the distance between marker and device, in addition to the verified accuracy of marker recognition via an illuminance meter. The findings revealed differences between (A) and (B) of more than 30 cm. In the case of the flat surface fluorescent light, the differences depended on the surface direction (Table 3 (B)). On the whole, (B) shows relatively unstable properties and more effects of the surroundings than does (A).
With this AR application prototype, users could put markers on real objects anywhere there is adequate space to paste them. This would make it possible to expand user experience by increasing understanding of infants’ actions.
The first years of life are filled with positive and negative affordances that enable infants to develop. These affordances could be changed according to age, motor skills, or object layout. This AR application prototype provides an opportunity to examine the relationship between infants and their environments and to learn from it. Additionally, it allows a more precise look at what may lead to accidents within the app users’ own home environments. Moreover, the prototype displays resources that may aid development in the long run.
The target users of this application are not limited to young parents. Its popularity is expected to spread into the educational, design, and cognitive science fields from the context of ecological psychology.
From a usability perspective, 1) the application’s UI/UX aimed simple graphic design and standard gestures (tap and swipe) functionality. The way of providing notification of risk information and tips for development continues to improve. 2) A marker-based system would currently be more suitable for users in a variety of situations. It will also be necessary to gather more data in the near future concerning children’s actions by sharing this prototype. The goal is to create an AR application global composition system that will allow users worldwide to capture augmented information about real, everyday objects (furniture, door, steps, etc.) in their surroundings without any markers.
This research was supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI (26870473), Grant-in-Aid for Young Scientists (B).