Gamified Model to Support Shopping in Closed Spaces Aimed at Blind People: A Systematic Literature Review

Abstract

In Colombia, building and implementing accessible spaces to include the needs of people with disabilities has always been a challenge for governmental entities. In particular, the visually impaired (VI) community presents difficulties in multiple sectors of society. During the world summit of local and regional leaders[1] addressing these challenges was included in the Agenda for Sustainable Development 2030. Adapting malls and supermarkets infrastructures to include people with disabilities is specially difficult, making tasks such as navigating and shopping for VI people, without external aids, almost impossible. Using a gamified model could help to create a system that allows VI people to shop autonomously while enjoying the process (user experience). A systematic literature review was conducted to find what has been done to allow blind people to shop independently in closed spaces. A total of 876 studies were found in 7 databases. The results of this study show that is possible to use technological developments to assist people during the different steps of the shopping process in closed spaces.

Keywords

• Shopping Closed Spaces
• Blind People
• Systematic Review
• Gamification

-

The disabled people currently represent 15% [ 2 ] of the world population in which 1.3 billion people have some type of visual impairment. In Colombia, it is estimated that the population with some kind of disability is approximately 6.4% [ 3 ] of the residents. Included in this percentage is the VI community, which represents the 43.5% [ 3 ] of this population, this will be considered as the relevant market for this study. To interact with the environment, people rely on their senses (touch, sight, hearing, smell, and taste). This applies to the Shopping Process as well, so when people enter a supermarket is possible for them to get the products they need. However, for a visually impaired person, entering a supermarket is a challenging experience because they cannot interpret the cognitive signs, which usually are visual. This issue could be solved by changing the infrastructure of the supermarkets and transforming them into accessible places for VI people or by introducing innovative solutions that allow VI people to navigate and find products within the current framework [ 4 ]. Colombia is one of the countries that do not have accessible supermarkets for blind people. Therefore, the shopping process for the visually impaired is unpleasant due to the lack privacy and autonomy. This study aims to research what emerging technology-based tools have been implemented to support the shopping process in closed spaces for blind people around the world.

This paper has 5 sections. Section 2 presents an overview of the shopping process in closed spaces for blind people. Section 3 describes the methodology used for the systematic review. Section 4 details the results obtained after the review. Finally, section 5 concludes the study and defines the future work. 2 2.1

Background

The shopping process for blind people

Adam Crosier and Alison Handford in their article ”Customer Journey Mapping as an Advocacy Tool for Disabled People” [ 5 ] created a Customer Journey Map that described a case study involving between five to eight people diagnosed with different conditions of vision loss. For each person, a follow-up was carried out from their home to different points in the city such as supermarkets and warehouses. Based on these actions, the authors defined the phases of the Customer Journey Map and the emotions that the subjects were feeling throughout the process. For this study, the Customer Journey Map when the disabled person entered the supermarket was the main focus of interest, as shown in the Fig. 1. Taking this findings into account, the Customer Journey Map confirms the critical points and situations where blind people have difficulties shopping in a store.

Gamification for the blind people

Gamification is a method to apply different elements of game playing to other activities with the purpose of engaging a product or service to a specific user. Nowadays, this is a technique widely used to create accessible technology. In the article ” Interactive Gamification Learning Media Application For Blind Children Using Android Smartphone in Indonesia ” [ 6 ], it is noticed that through gamification strategies, processes that sometimes are repetitive and tedious can be turned into something attractive improving user experience. In the article, the authors stated that:”Gamification creates entirely new engagement models, targeting new communities of people and motivating them to achieve goals they may not even know they have [ 7 ].”. The results of the article showed that through gamification, children who were visually impaired were able to learn in a more enjoyable manner. Proving that gamification is often used on people with visual disabilities, making this a succesful strategy to improve the way VI people learn to use new technology. 3

Research Method

A systematic review of the literature was carried out following the guidelines proposed by Kitchenham and Charters [ 8 ]. This process helped to find and organize the information that exists on a particular topic, below is the review. The objective of the Systematic Review is to find the technologies proposed by the academy to aid blind people in the physical shopping process. Since most of the supermarkets in Colombia are closed spaces, another goal is to determine whether navigation in these spaces is done through inclusive technologies; and if gamification has been implemented as an alternative to improve the user experience in emerging accessible blind technologies.To answer these unknowns, the following research questions have been posed: 1. What are the technological developments that blind people can use to make purchases physically? 2. What technologies have been used to perform navigation in closed spaces for blind people? 3. Are there technological tools to allow blind people to play video games? The following databases were used to carry out the review, each search considered documents that have been published in the last 5 years (2015-2020) and the different filters that the databases allowed.

Searches conducted in English: – IEEE Xplore (http://ieeexplore.ieee.org) – ACM Digital library (http://dl.acm.org) – Ebsco (https://www.ebsco.com/) – SCOPUS (https://www.scopus.com/home.uri)

Searches conducted in Spanish: 3

– Redalyc (http://redalyc.org) Phrases such as: ”indoor navigation blind”,”blind assistive shopping” , among others, were used to perform the search. However, the documents found were not specific enough. For example, in IEEE Xplore 1000 documents were obtained on the subject, which is why the databases’ advanced search tools were used, as shown below: – IEEE Xplore: The search was performed in Metadata and Full Text, adding the following keywords and logical operators, where a 149 documents were found. ((((“assistive technology”) OR (“indoor navigation shopping”)) AND (“blind”))) – ACM Digital library: The search was performed with the command shown below only taking into consideration the title, the abstract and the keywords. Where a 289 documents were found. [[All: ”assistive technology”] OR [All: ”indoor navigation shopping”]] AND [All: blind] AND [Publication Date: (01/01/2015 TO

12/31/2020)] – SCOPUS: The search was performed by title, keywords and summary where 64 documents were found with the following search command:

(TITLE-ABS-KEY (”assistive technology” AND ”blind”)OR TITLE-ABS-KEY (”indoor navigation shopping” AND”blind” )) – EBSCO: Initially, no documents were found taking into account only the title, abstract and keywords, for this reason we decided to perform the search on the entire document. As a result, 342 documents were obtained.

(assistive technology OR indoor navigation shopping) AND blind – Redalyc: 37 relevant documents were found in this database. Redalyc does not allow advanced searches with logical operators, so the Google tool is used with the following command: (“tecnolog´ıa de asistencia” OR ”navegacio´n en espacios

cerrados”)AND ciegos site:redalyc.org filetype:pdf

3.1 Inclusion and exclusion criteria

After the search was made, the following criteria was defined to reduce the number of documents to include in the bibliography.

– EC1: Documents not related to technologies for the possible development of the research work. – EC2: Documents outside the 2015-2020 time range. The selected documents were screened against the guidelines established below: – IC1: Documents with models for mobilization in closed spaces for blind people. – IC2: Documents that present technological support so that blind people can participate in the video game environment. – IC3: Documents that support the physical purchasing process of a blind person.

Due to the large number of documents found, the guidelines established by S.Keshav in his article ”How to read a paper”[ 9 ] were applied. These determines a way to efficiently read an article through 3 steps: first The abstract, titles and conclusions are read, Second The document is read more thoroughly taking into account the figures, graphs, and possible references, Third The document is read in its entirety. Based on this strategy, the documents of each database are delimited as illustrated in the Table 1.

This systematic review was updated on July 30, 2020. A total of 876 documents were obtained to which the defined inclusion and exclusion criteria were applied, obtaining a total of 12 articles. Additionally, a search on grey literature was performed on national thesis repositories. However, there were no other documents that met the desired criteria. The chosen articles were sorted by title, authors, year of publication, place of publication, DOI, document ID, database, and keywords. To perform the data extraction an alphanumeric ID was assigned to each document starting with the letter D and followed by a number from [D1],[D2],[D3],...,to [D12].

The articles were reviewed and organized on groups as follows:

NAVIGATION ON CLOSED SPACES These articles presented different solutions to allow blind people to move in closed spaces, either by using a smartphone or an accessible hardware designed for this specific purpose. – Technologies based on smartphones: Most of the articles found posed solutions developed as mobile applications (6). As shown on a survey applied to 466 people with visual disabilities in the article ”Exploring the use of smartphones and tablets among people with visual impairments: Are mainstream devices replacing the use of traditional visual aids?”[ 10 ], Smartphones have become the main accessibility tool for blind people. This leads to highlighting the importance of developing design parameters for inclusive mobile applications; to aid blind people on common daily tasks such as shopping. The authors concluded that accessible technologies for visually impaired people are migrating to smartphone-based ones, this claim is supported by the results obtained in the survey: • 87.4% of the respondents agreed that conventional devices such as smartphones are displacing traditional solutions. • 69.6% of the respondents held that personal use of a smartphone is more important than a specialized device for daily activities. • Among the respondents who were smartphone users most of them 89.8% had been using their smartphone for over 3 years, 7.5% with 1–2 years of experience, and 2.7% with less than one year from experience. I. Doush et al. in their article [ 11 ] developed an integrated system that helps blind people to localize objects within a range of 10 cm. The study is based on 2 pilot cases that helped them define a series of recommendations to improve navigation systems for the blind. Furthermore, they developed an application taking advantage of the WIFI, Bluetooth, and RFID modules of the smartphone device and designed an accessible interface for blind people. Many other authors studied in this review focused more on navigating blind people to the entrance of establishments, omitting the process that takes place within the store. In [ 12 ] [ 13 ] S. Alghamdi and A. Meliones et al. respectively develop applications to know the relative position of the user in closed spaces, using technologies implemented in the store or supermarket such as WLAN, beacons, and Tactile Route Indicators integrated through an application in the user’s smartphone. Some of the developments were limited to the characteristics of the space where the user intended to use the mobile application. In [ 14 ] people can move autonomously using an application along with an open software platform like Smart-Space Real-Time Location System (RTLS) Ubisense, but the places where the system is implemented cannot change regularly, the authors recommend its use for places such as hospitals or supermarkets. In [ 15 ] a probabilistic algorithm is defined so that blind people can move with an application in multi-story places. [ 16 ] analyzes the requirements to develop navigation systems for blind people taking into account the challenges that emerge on existing ones, the different design parameters created to overcome the challenges are explained in detail. They define a complete software structure divided into smaller modules that handle different phases of the navigation process.

In recent years, researchers have managed to integrate many available technologies into the development of mobile applications. The table 2 summarizes the technologies used on inclusive mobile applications for blind people, found on the articles of this section. As shown, the technologies used for the development of inclusive applications are mostly integrated into the hardware of current smartphones. The findings show that WIFI is the most used technology on emerging inclusive mobile applications for the VI. This represents the tendency of developers to take advantage of the network infrastructure that already exists to guide blind people in closed spaces. – Technologies based on hardware different from smartphones: The solutions proposed for hardware different to smartphones are mostly wearable devices so that VI people can navigate through spaces with their limbs free of any restrain. We found 3 fascinating prototypes that aim to help blind people perform tasks related to the shopping process. In [ 17 ] The authors proposed a blind-sight navigator (BSN), which is a wearable device in the form of a backpack and a helmet. The device is integrated with PICS as microcontrollers to perform image recognition of the place where the user is moving. Also, there is a device that transmits vibratory signals to the user to alert them to possible obstacles in the way.

The second prototype [ 18 ] is a vest that blind people would wear to walk. The vest is integrated with a camera, an IMU, a laser sensor, and a headset to give commands to the user. The implemented logic modules allow navigation in closed spaces and the recognition of objects in space, is an integrated solution for closed environments with complex structures.

In the third prototype, [ 19 ] the main objective of the authors was to create a low-cost and open-source prototype to help blind people solve the primary task of navigation, orientation, and obstacle detection. The device consists of a bracelet made in 3D printing that integrates an ultrasound sensor and an IMU that are low-cost sensors and easy to acquire on the market. USE CASES The following documents present case studies using two mobile applications ”MagNav and Ucap”.[ 20 ] presents an application use case: MagNav, an accessible navigation app. It is a system that guides users employing auditory signals through a closed space, in this case, the test was performed in a shopping center in the United States. However, MagNav does not have the function for object identification. The other case found was also on a mobile application named UCap [ 21 ], a crowdsourcing application for the visually impaired persons on an android smartphone. The application uses the camera to allow customers to take pictures of the product they aim to identify, it should be noted that the application is exclusively responsible for identifying products.

GAMIFICATION There was 1 article that show some research done regarding gamification strategies for blind people. However, not much information surfaced about gamification processes applied to blind people. The article [ 22 ] attempts to verify the accessibility of blind people to the gaming environment in mobile applications. They discovered that most games that offer accessibility do not allow interaction with other players, a feature that seeks to be improved by initiatives such as ”Game2Senses”; which is an innovative project for blind people to participate in a multiplayer game using only the touchscreen and gyroscope of the device. 4

CONCLUSIONS AND FUTURE

WORK

The Systematic Literature Review found that 12 out of 876 papers met the criteria of inclusion defined in this review. There is a lot of research done regarding accessible technology for blind people. However, there was not much literature on a integral system to allow blind people to shop autonomously covering all the challenges that the process imposes. Most of the studies analyzed the navigation of closed unfamiliar spaces , rather than object recognition; and none of the studies found, proposed a gamified model for the shopping process. Since smartphones are slowly replacing traditional accessibility tools; a great majority of the articles used a mobile application as a tool for transforming inadequate closed spaces to inclusive places for blind people. Regarding hardware different to smartphones, the solutions are wearable devices that interact with the environment the way eyes do for sighted people. There were 3 promising prototypes found: a backpack with a helmet, a bracelet, and a vest. All of the solutions offered basic navigation functionalities such as obstacle avoidance and image recognition. In terms of existing mobile applications ”Mag-Nav” helps blind people navigate unknown closed spaces and ”U-Cap” allows visually impaired people (low vision) to identify an object by taking a picture of it. Mobile games and applications mostly are not accessible for blind people however some recommendations on the subject were presented.

Future work: Without a doubt, having no solution to aid the visually impaired shopping process has inconceivable effects both physically and emotionally on the blind community. As a result, there should be more research aimed at an integrated system that solves all the challenges that affect the visually impaired community on the shopping process. Based on the results obtained in this review and following the functional recommendations done in the analyzed documents, the development of a Gamified Model to Support Shopping in Closed Spaces Aimed at Blind People using emerging technological developments will be the main objective of future research.

References

1. Durban2019: Ciudades inclusivas y accesibles . [Online]. Available on: ”https://www.uclg.org/sites/default/files/ciudades\_inclusivas_y_ accesibles_documento_de_politica.pdf [Last Access: 2020 -01-20]” ( 2019 )

2. OMS: 10 datos sobre la discapacidad . [Online]. Available on: ”https://www.who. int/features/factfiles/disability/es/\[Last Access: 2020 -01-15]” ( 2017 )

3. Dane: Censo general 2005 . [Online]. Available on: ”https://www.dane.gov.co/ files/censo2005/discapacidad.pdf [Last Access: 2020 -01-23]” ( 2005 )

4. of Justice, U.D.: Maintaining accessible features in retail establishments . [Online].Available on: ”https://www.ada.gov/business/retail_access.htm [Last Access: 2020 -01-23]” ( 2009 )

5. Crosier , A. , Handford , A. : Customer journey mapping as an advocacy tool for disabled people . Social Marketing Quarterly 18 ( 1 ) ( March 2012 ) 67 - 76 . https://doi.org/10.1177/1524500411435483 https://doi.org/10.1177/ 1524500411435483

6. Sari , A.C. , Fadillah , A.M. , Jonathan , J. , David Prabowo, M.R. : Interactive gamification learning media application for blind children using android smartphone in indonesia . Procedia Computer Science 157 ( 2019 ) 589 - 595 . https://doi.org/https://doi.org/10.1016/j.procs. 2019 . 09 .018 http://www. sciencedirect.com/science/article/pii/S1877050919311767 The 4th International Conference on Computer Science and Computational Intelligence (ICCSCI 2019 ).

7. Burke , B. : Gamify : how gamification motivates people to do extraordinary things . Bibliomotion, Inc, Brookline, MA ( 2014 )

8. BA , K. , Charters , S. : Guidelines for performing systematic literature reviews in software engineering . 2 ( 01 2007 )

9. S. Keshav , David R. Cheriton School of Computer Science, U.o.W.: How to read a paper . 1 ( 01 1997 )

10. Martiniello , N. , Eisenbarth , W. , Lehane , C. , Johnson , A. , Wittich , W. : Exploring the use of smartphones and tablets among people with visual impairments: Are mainstream devices replacing the use of traditional visual aids? Assistive Technology 0(0) ( 2019 ) 1 - 12 . https://doi.org/10.1080/10400435. 2019 . 1682084 https://doi.org/10.1080/10400435. 2019 .1682084 PMID: 31697612 .

11. Doush , I.A. , Alshatnawi , S. , Al-Tamimi , A.K. , Alhasan , B. , Hamasha , S.: ISAB: Integrated indoor navigation system for the blind . Interacting with Computers (June 2016 ). https://doi.org/10.1093/iwc/iww016 https://doi.org/10.1093/ iwc/iww016

12. Alghamdi , S. : Shopping and tourism for blind people using RFID as an application of IoT . In: 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS), IEEE (May 2019 ). https://doi.org/10.1109/cais. 2019 . 8769581 https://doi.org/10.1109/ cais. 2019 .8769581

13. Meliones , A. , Sampson , D. : Indoor blind navigator . In: Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, ACM ( June 2017 ). https://doi.org/10.1145/3056540.3064959 https: //doi.org/10.1145/3056540.3064959

14. Alnafessah , A. , Al-Ammar , M. , Al-Hadhrami , S. , Al-Salman , A. , Al-Khalifa , H.: Developing an ultra wideband indoor navigation system for visually impaired people . International Journal of Distributed Sensor Networks 12 ( 7 ) ( July 2016 ) 6152342 . https://doi.org/10.1177/155014776152342 https://doi.org/10.1177/ 155014776152342

15. Murata , M. , Ahmetovic , D. , Sato , D. , Takagi , H. , Kitani , K.M. , Asakawa , C. : Smartphone-based indoor localization for blind navigation across building complexes . In: 2018 IEEE(PerCom) , IEEE (March 2018 ). https://doi.org/10.1109/percom. 2018 . 8444593 https://doi.org/10.1109/ percom. 2018 .8444593

16. Arvai , L. : Mobile phone based indoor navigation system for blind and visually impaired people: VUK - visionless supporting frameworK . In: 2018 19th International Carpathian Control Conference (ICCC) , IEEE (May 2018 ). https://doi.org/10.1109/carpathiancc. 2018 . 8399660 https://doi.org/10. 1109/carpathiancc. 2018 .8399660

17. Boudreault , A. , Bouchard , B. , Gaboury , S. , Bouchard , J.: Blind sight navigator . In: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments-PETRA 16, ACM Press ( 2016 ). https://doi.org/10.1145/2910674.2910709 https://doi.org/10.1145/ 2910674.2910709

18. Mekhalfi , M.L. , Melgani , F. , Zeggada , A. , Natale , F.G.D. , Salem , M.A.M. , Khamis , A. : Recovering the sight to blind people in indoor environments with smart technologies . Expert S. with A. 46 (March 2016 ) 129 - 138 . https://doi.org/10.1016/j.eswa. 2015 . 09 .054 https://doi.org/10.1016/j. eswa. 2015 . 09 .054

19. Petsiuk , Pearce: Low-cost open source ultrasound-sensing based navigational support for the visually impaired . Sensors 19 ( 17 ) ( August 2019 ) 3783 . https://doi.org/10.3390/s19173783 https://doi.org/10.3390/s19173783

20. Giudice , N.A. , Whalen , W.E. , Riehle , T.H. , Anderson , S.M. , Doore , S.A. : Evaluation of an accessible, real-time, and infrastructure-free indoor navigation system by users who are blind in the mall of america . Journal of Visual Impairment & Blindness 113 ( 2 ) ( March 2019 ) 140 - 155 . https://doi.org/10.1177/0145482x19840918 https://doi.org/10.1177/0145482x19840918

21. Hoonlor , A. , Ayudhya , S.P.N. , Harnmetta , S. , Kitpanon , S. , Khlaprasit , K. : UCap: A crowdsourcing application for the visually impaired and blind persons on android smartphone . In: 2015 International Computer Science and Engineering Conference (ICSEC), IEEE (November 2015 ). https://doi.org/10.1109/icsec. 2015 . 7401406 https://doi.org/10.1109/icsec. 2015 .7401406

22. Smith , K. , Abrams , S.S. : Gamification and accessibility . International Journal of Information and Learning Technology 36 ( 2 ) ( March 2019 ) 104 - 123 . https://doi.org/10.1108/ijilt-06-2018-0061 https://doi.org/10.1108/ ijilt-06 -2018-0061