Visual impairment is a condition that presents significant challenges in the daily lives of those who experience it. In the city of Arequipa [1], as in many other regions, people with visual disabilities face additional difficulties when trying to identify banknotes and objects in their everyday environment. This project aims to address this issue by developing an economical assistance device based on computer vision and IoT. This device will enable visually impaired individuals to identify banknotes and objects, providing them with greater autonomy and independence in their daily lives. In this context, the design, implementation, and evaluation of this innovative device are presented, which has the potential to improve the quality of life for visually impaired people in Arequipa and serve as a model for similar solutions worldwide [2].
In the current context, technology plays a fundamental role in improving the quality of life for people with visual disabilities. One of the key challenges faced by these people in their daily lives is the identification and management of different denominations of banknotes, a crucial task in financial transactions and daily activities. In response to this need, various global and national research efforts have been carried out to develop banknote recognition systems using advanced technologies such as machine learning, computer vision, and image processing.
Globally, several research studies have focused on the development of banknote recognition systems to assist visually impaired individuals in identifying different denominations of currency.
In [3], they proposed developing a Malaysian banknote recognition system to assist visually impaired people. The study aimed to analyze the impact of region and orientation on the performance of Machine Learning and Deep Learning approaches. The results revealed that SVM and BC algorithms achieved 100% accuracy, while kNN and DTC achieved 99.7%. It was also concluded that orientation influences the performance of the AlexNet model, showing better execution with similar orientation data.
On the other hand, in [4], the accurate classification of Honduran banknotes was the focus, including a new L200 banknote, with an emphasis on adapting to the incorporation of new banknotes in circulation. Two high-performance methods were presented. The first relied on advanced local descriptors such as ORB or SIFT, generating feature vectors for algorithms like SVM and Random Forests. The second introduced the LempiraNet CNN, which used transfer learning to address data limitations. The results demonstrated outstanding accuracy of 98% or higher, with LempiraNet being significantly faster than the other method.
Conversely, [5] proposed an innovative approach based on quaternionic wavelet transform (QWT) and a deep convolutional neural network for banknote classification. This methodology leveraged the multiscale structure and directional sensitivity of QWT. The results highlighted superior performance compared to other state-of-the-art banknote classification algorithms, as well as meeting real-time requirements for banknote classification systems.
In [6], the focus was on creating an Iraqi banknote classification system based on Deep Learning and computer vision technology. The central objective was to develop a multiclass classification model capable of distinguishing between different denominations of Iraqi banknotes and providing equivalent voice commands to inform visually impaired people about the value of the banknotes. The system achieved an impressive accuracy of 98.6%, demonstrating its viability and potential to enhance the financial independence of this user group.
In the research [7], they focused on the recognition of Ethiopian banknotes using a convolutional neural network (CNN). The research included comprehensive evaluations of different CNN architectures and optimization techniques. The highlighted architecture, MobileNetV2, implemented with RMSProp optimization, achieved outstanding accuracy of 96.4%. Additionally, the model was implemented on an integrated platform using Raspberry Pi, with potential applications in automatic monetary transactions.
In [8], they addressed the recognition of Colombian banknotes by visually impaired people. They developed a classification system for eleven denominations of banknotes using image processing techniques and MLP neural networks. The system achieved 95% accuracy after manipulating samples and expanding the dataset, underscoring its effectiveness in the autonomy of visually impaired individuals.
Finally, in [9], they introduced an AI-backed mobile application for the recognition of banknotes from the United Arab Emirates, aimed at visually impaired people. The application used a pre-trained convolutional neural network to detect and classify banknotes, in addition to providing auditory signals. Although the average accuracy reached 70% in tests and 88% in fivefold cross-validation, the application represents a step toward independence in daily financial transactions.
On a national level, in Peru, research has focused on the design and development of low-cost ergonomic tools to improve the mobility of people with visual disabilities.
In Lima, [10] proposed the creation of an ergonomic GPS-enabled cane for blind people with the aim of increasing their autonomy. The methodology was based on systems engineering for monitoring and software development. The results indicated that the ergonomic cane improved the mobility of people with visual disabilities and allowed tracking by their family members. The importance of considering aspects such as the shape, size, and weight of the cane to achieve the desired ergonomics was highlighted.
On the other hand, in Chiclayo, [11] focused on the development of an Intelligent Geolocating Sensor Cane to support blind people in their mobility. The methodology used was based on the Rational Unified Process (RUP) and embedded systems. The results demonstrated that this sensor cane could make life more dynamic and secure for people with visual disabilities. The fusion of the RUP and embedded systems methodologies contributed to the efficient design of the prototype.
Finally, in Arequipa, [12], the focus was on improving the quality of daily mobility for people with visual disabilities through an Electronic Cane with ultrasonic sensors. The methodology included a descriptive-explanatory study and the experimental design of sensors. The results highlighted that this Electronic Cane could reduce accidents in the daily mobility of people with visual disabilities.
In summary, notable advances have been made worldwide and nationally in creating tools and systems to improve the lives of people with visual disabilities, using technologies such as machine learning, computer vision, and ultrasonic sensors. These advances demonstrate the potential of technology to increase the independence and safety of people with visual disabilities. The research conducted in Peru emphasizes the importance of considering the specific needs of this group and the utility of combining various methodologies to create effective solutions. Together, these developments offer a promising outlook in which technology will continue to play a fundamental role in improving the quality of life for people with visual disabilities, with possibilities for application elsewhere and a focus on inclusion and autonomy.
The study employed an iterative, action-oriented methodology and followed the five-step process of the Design Thinking approach (i.e., empathize, define, ideate, prototype, and evaluate) [16].
This phase was rigorously executed, encompassing a comprehensive literature review of available assistive tools and technologies. Requirements gathering was done through interviews and analysis of local statistics, the evaluation of Artificial Vision algorithms, a detailed comparison of IoT devices, and the meticulous selection of Computer Vision techniques. This ensured that the device would be designed with a closer understanding of the needs and conditions of visually impaired individuals from the Association of the Blind in Arequipa. Best practices and accessible technologies were leveraged to provide an effective solution tailored to their primary needs, which revolve around the recognition of banknotes and nearby everyday objects.
After gaining an empathetic understanding of the main needs of visually impaired individuals, the define phase involved categorizing the results obtained from the evaluation of techniques, Artificial Vision algorithms, and IoT devices. This classification identified and selected the most appropriate and effective approaches to address the needs of visually impaired individuals in the study region. Additionally, a comprehensive compilation of results related to the current situation of these individuals and available assistive tools in the context of Arequipa was conducted. These findings provided a solid foundation for decision-making in the design of the device. It is worth mentioning that all these results were thoroughly documented in the thesis report, thus establishing a solid and well-founded basis for the subsequent ideation and prototyping phases.
With a defined understanding of the problem, a creative flow of ideas is generated. In this phase, a plan is designed and developed to achieve the goal of creating a product that improves or solves the identified problem. To accomplish this, a set of actions was followed for creating the dataset of banknote images. This included setting up the necessary image capture equipment, involving the selection of appropriate cameras and sensors. A meticulous procedure for image capture was developed, considering a variety of relevant scenarios and situations reflecting the diversity of environments in which visually impaired individuals might use the device. Image capture was carried out extensively, covering banknotes representative of the local reality. Subsequently, labeling of these images was performed, preparing them for use in training the Artificial Vision module. The results obtained in this stage provide a solid dataset of banknote images, crucial for the development of the product. These achievements have been thoroughly documented, establishing a clear and well-founded direction for the subsequent prototyping and evaluation stages.
During this stage, the implementation of the vision module designed for real-time image capture and processing, a fundamental element of the proposal, has taken place. This achievement was not only crucial for the device's operation but also highlights the robustness of the strategy in combining Internet of Things (IoT) technologies with advanced Computer Vision techniques to accomplish the task of recognizing banknotes and everyday objects near the user. Additionally, exhaustive tests have been conducted, and the components of the vision module have been debugged. This has allowed for the effective identification and addressing of any obstacles or potential deficiencies, ensuring optimal system performance. Furthermore, acceptable communication has been achieved between the vision module and the IoT device, thereby reinforcing the reliability and effectiveness of the proposed solution. The design of the first prototype is presented below:
In the evaluation phase, technical and functional tests have been conducted under controlled conditions to validate the integration of the system, measure effectiveness in banknote recognition, and collect technical feedback. Although end-users have not yet been included in these evaluations, crucial data has been gathered to identify areas for improvement in the design and operation of the device. These data and observations will serve as a foundation for future tests with end-users and contribute to the overall success of the project.
One of the main outcomes in the development of the project proposal is:
1. The elaboration of a systematic review of assistive tools for visually impaired individuals, conducted during the empathize stage, allowed for the identification of the main limitations and contributions in the last 5 years of research in the field. These are related to advances in assistive technologies, the Internet of Things, and Computer Vision. In figures 4 and 5, the main findings of the systematic review are presented, emphasizing the key limitations and contributions. Access link: https://ieee-dataport.org/documents/dataset-peruvian-banknotes 3. The construction of the first prototype of the proposed assistive device is a significant outcome of the project. With this, the overall objective of providing assistance in the recognition of Peruvian banknotes and nearby objects to the user would be achieved. As a key contribution within the construction of the first prototype, the effective use of pre-trained deep learning models for banknotes and models pre-trained on an ImageNet dataset for objects, the characterization of processing complexity, the design of an economical assistive system, and the empirical evaluation of accuracy in real-world conditions have been highlighted. Thus, achieving the design of the first prototype, the optimal processing of Peruvian banknote and object images, and the empirical evaluation of the accuracy of pre-trained deep learning algorithms with 68% for banknote recognition and 73% for object recognition with Fine-Tuning.
This study has focused on addressing the challenges faced by visually impaired individuals in identifying banknotes in the city of Arequipa and the development of an assistive device based on artificial vision and IoT to mitigate these challenges. From the research and work conducted, the following conclusions can be drawn:
• The need for technological solutions for visually impaired individuals is critical. The identification of banknotes and objects is essential in daily life, making the creation of economical and effective assistive devices imperative to improve their quality of life.
The exhaustive review of the state of the art in global and national banknote recognition systems highlights that technology, particularly machine learning, computer vision, and sensors, can be a valuable ally for visually impaired individuals, providing them with independence and security.
The Design Thinking approach, encompassing the stages of empathy, definition, ideation, prototyping, and evaluation, has proven effective for the development of an assistive device that caters to the specific needs and conditions of visually impaired individuals in Arequipa.
The creation of a dataset of banknote images and the development of the real-time vision module are fundamental achievements supporting the viability and effectiveness of the proposed assistive device.
Although tests with end-users are still pending, the technical data collected so far will serve as a basis for future evaluations and refinements of the device. This project is anticipated not only to enhance the quality of life for visually impaired individuals in Arequipa but also to potentially serve as a model for similar solutions worldwide.