Navigating the Web is one of important missions in the field of computer accessibility. Many specialized techniques for VIP (Visually Impaired People) succeeded to extract the information displayed on digital screens and succeeded to transform this information in a linear way either into a written format on special Braille devices, or into a vocal output using text to speech synthesizers. However, although this success, screen readers failed to transform the 2-dimentional structure of the navigated web page; despite many researches confirm that perception the structure enhances web navigation and memorization. In this paper, we propose a new technique aimed to enhance the VIP ability to navigate the Web by affording a “first glance” web page overview. This technique focuses on improving non-visual vibrotactile access to web pages on touch-screen devices, based on extraction and re-organization the structure of texts and graphical elements for web pages, reformatting and converting automatically these visual structures and textual information into vibrating pages using a graphical vibro-tactile language.
Gaël Dias 1
[H.5.2] [User Interfaces] Graphical user interfaces (GUI), Haptic I/O, User-centered design, User interface management systems.
In October 2013, the world health organization estimated that the number of VIP in the world is 285 million, 39 million of them are blind, and 246 million of them have low vision. The organization defined four levels of visual functions depending on the international classification of diseases, which are: normal vision, moderate visual impairment, severe visual impairment, and blindness*.
VIP depend on screen readers in order to deal with computer
operating systems and computational programs. One of most
important and desired targets by VIP is navigating the Web,
considering the increased importance and expansion of web-based
computational programs. Screen readers present some solutions to
navigate the Web, either by transforming a web page into a
written Braille, or into a vocal output. Some screen readers
installed on touch devices transform a web page into a
vocaltactile output. But there are some drawbacks for these proposed
solutions: on the one hand, the Braille techniques are costly, and
only few number of VIP have learned Braille (in France, there are
about 77 000 visually impaired people and only 15 000 of them
have learned Braille -from statistics published in September
2011) **. On the other hand, transforming the information of a web
page into a vocal format might not be suitable in public and noisy
environments. Finally most of Braille solutions are not suitable
for mobile devices [
Current products for VIP such as screen readers depend mainly on
speech synthesis or Braille solutions, such as ChromeVox [
A tactile web browser for hypertext documents has been proposed
by [
In figure1, we notice many symbols; each one represents an HTML element. For example, the symbol represents a menu of items. And the symbols and represent texts with cold and hot colors.
Tactos is a perceptual interaction system suggested by [11], it
consists of three elements: 1- tactile simulators (two Braille cells
with 8 pins) represent a tactile feedback system, 2- a graphics
tablet with a stylus (represents an input device), 3- computer [
First glance could be defined as the ability -in a blink of an eye- to
understand the document layout and its structural semantics [
The first phase in our model is to extract visual structures in the navigated web page, and to convert these visual blocks to zones (segments) for facilitating the navigation in later phases. We achieve this phase depending on a hybrid segmentation method. Then the system will represent on the tablet screen the extracted visual elements as symbols using a graphical language (this language is under-development). The third phase is to browse these graphical symbols depending on size of the used touchedscreen device, and then in the fourth phase, our system provides a vibro-tactile feedback when the blind user touches the tablet. The intensity and the frequency of the vibration depend mainly on gray level under the finger. A tablet (Asus Model TF101 with Android operating system) has being used for our tests. In this paper, we focus only on the fourth phase which specializes in giving the user a vibro-tactile feedback by transforming light contrasts of touch-screen devices into low-frequencies tactile vibrations.
To achieve the desired system, we have designed an electronic circuit which controls two micro-vibrators placed on the hands. A Bluetooth connection with an android tablet allows controlling the vibration intensity (Amplitude) of vibrators. An Android dedicated program on the tablet displays an image on the screen and detects where the user touches the tablet screen. The gray level of touched points is transmitted to the embedded device in order to control the vibration intensity. At the moment, only one micro-vibrator was used for pre-tests described in this paper.
Our objective of the designed protocol is enhancing the ability of
VIP to recognize the 2-D structure of a web page. In order to test
the prototype mentioned in section 3, we designed some images
contain different structures (detailed in section 4.2), and we tested
the prototype firstly on 15 sighted persons (their eyes were closed)
[
Each experiment (either for sighted or blind persons) consists of 4 ordered phases of training (learning task), and four ordered phases of evaluation (evaluation task). All the experiments were filmed, and the designed program stocked many parameters in log files (coordinates X, Y, pressure on the screen, and the time at each touch). Figure 4 presents the 4 images of training phases, and figure 5 presents the 4 images of evaluation phases.
In the training task, each user discovered firstly graphical elements in each image presented in figure 4 (images a, b (NT2), c (NT3), d (NTG)), and users were informed about names of graphical elements. The name of each image NT2, NT3, NTG, indicates how many transitions are necessary to access the square center, for example NT2 proposes 2 transitions to access the center of the square.
The evaluation task consists also of 4 phases, the first one allows to discover the image 5.a and to name each square inside it, then next phases are about images 5.b, 5.c, and 5.d, where we asked users to discover contents of each image, then to describe these contents, and to redraw discovered elements inside each image. We chose these images depending on following considerations: - Image 5.a contains all squares on which users have trained in the training task, so it could test the ability to memorize and to distinguish the shapes. - Image 5.b contains 3 rectangles with matched sizes and with vertical order, and the image 5.c contains 3 rectangles with different sizes and many relations of directions, so testing images 5.b, and 5.c could test the ability of distinguishing sizes, and distinguishing relations of directions. - Image 5.d contains different shapes (a rectangle and a polygon), so it could test the ability to distinguish different shapes in the same image. - The tested images contain examples of expected results of the segmentation process, so success of distinguishing these shapes by blind users could be an indicator of their ability to distinguish results of segmenting web pages.
The results of pretests with sighted persons were already
published in [
The test performed with each one of the 5 blind persons consisted of following: personal and technical questions, explanations of the test objective, a training task, and finally an evaluation task. The approximated average time for the test for each person is about 1 hour.
Before starting the tests with the 5 blind persons, we asked them to support us with some information about their age and date of their blindness. Table 2 summarizes answers of personal questions. We also asked users to provide us with some technical information about their experience in dealing with operating systems, screen readers, and what are the main problems when they navigate the Web. Table 3 shows a summary of answers for these technical questions. The first two columns indicate the number of operating systems (either Windows or Linux) used either on fixed or portable computers. The third, fourth, and fifth columns indicate the number of users who use JAWS (Job Access With Speech), NVDA (NonVisual Desktop Access), and ORCA, either on fixed or portable computers. No one of the five blind persons uses a tablet, and the screen readers used with cellular phones are Talks and MobileSpeak with Nokia, and Voiceover with IPhone. Only one of the 5 persons uses a telephone to access the Web (access via IPhone). The main problems of accessing the Web via fixed or portable computers, or via IPhone telephone were: problems of access to Flash files, problems of AJAX technologies, and no ability to know the global structure of web sites. (These problems have been reported to us by the 5 users).
To give the blind persons a good idea about the test, we explained in details what are the objectives and the phases of each tasks, and described contents of the embedded system; we also explained the final objective of the project, and why we concentrate on vibrotactile technique regardless of other techniques.
This phase was important to initiate users for accepting kindly the test and for doing their best to interact with next steps as correctly as possible.
In this training task, the user discovers the graphical elements in each image presented in figure 4 (images a, b (NT2), c (NT3), and d (NTG)), and the users were informed of each shape name. This task was very important for users to test the system before the evaluation task, and to know exactly how the system transforms different the grey level under the touched points on the tablet screen to a vibration mode. It is also very useful for users to control their speed of mapping the screen either to discover either the borders or the contents. During this task, the program recorded the touching information in log files (X, Y coordinates, Pressure, and Time).
Table 4 indicates training times in minutes for each user, and for each image in figure4. We notice from table 4 that discovering the first image takes more time, and it is normal because it is the first experiment for blind users on this prototype. We notice also that there is a significant decrease in time between discovering the first and the last image in the training task. This could be an indicator that training users could decrease the time for discovering graphical elements. We can also notice the significant difference between different tested persons, for example user with ID4 needed 7.43 minutes to scan the images (A, NT2, NT3, and NTG), but the user with ID3 needed 19.34 minutes to scan the same images.
In this task, firstly we asked each user to discover the image 5.a and to find how many squares inside it and to name each founded square, then we asked them to discover images 5.b (IDP1), 5.c (IDP2), 5.d (IDP3), and to describe them to us, and to redraw discovered shapes. Table 5 illustrates an evaluation of answers for the first question to name squares in image 5.a (The blind users have been asked to name the shapes in image 5.a, and for each shape, we evaluated the number of correct and incorrect answers). (In tables 5 and 6, the symbol represents a correct answer for the touched shape, and the symbol X represents an incorrect answer or inability to select the name of the touched shape). We notice from table 5 that the lowest number of errors is assigned to image NT2, and it is the same result which we obtained during tests with sighted persons.
Results of answers for other questions related to images IDP1, IDP2, and IDP3 are summarized in table 6. We notice from data in tables 4, 5, and 6, that the best performance is for the user with ID4, and this may be because that this female user is the youngest between others, and it could be because that she was the only one that has already used touched devices (an IPhone in her case working with VoiceOver). After answering questions about each image of images (IDP1, IDP2, IDP3), we asked each user to redraw the graphical elements founded in each touched image. Figure 6 views the redrawing results of the user ID4 (ID4 is the female user who gave best answers).
User ID / Image IDP1 IDP2 IDP3 Total
When comparing results of redrawing (Figure 6) with images IDP1, IDP2, and IDP3, we find that the results are interesting, and we can conclude the following:
because relations of directions (vertical order, left to, right to,…) between redrawn shapes is nearly equal to relations of directions between real shapes.
The average of times in minutes consumed for each evaluation question is summarized in table 7.
To get an idea about the most touched and the least touched areas on the screen during learning and evaluation tasks, we divided the touched-screen into 16 areas (as in figure 7, r00...r03, r10...r13, r20...r23, r30...r33), and calculated the average of touches in each area for all users.
We have founded that the most touched areas are r12, r11, r22, r21, and the least touched areas are r30, r33, r32, r00 as described in figure 8. This information could be useful in next phases of our research in completing the graphical vibro-tactile language by putting the important information in the most touched areas. During analysis the results, we have noticed that there are a lot of differences between the pressure values for all users (Pressure value depends on the used tablet; in these experiments we have used Asus Model TF101 with Android operating system). To analyze pressure values, we calculated the max pressure value between all users, it was 3.19, and the average was 1.73, then we redrew the touched points for each user with considering that points with pressure values equal to the max value have been drawn in red color, points with pressure values less than the max and greater than the average have been drawn in blue color, and points with pressure values less than the average have been drawn in green color. Figure 7 represents an example of these points in different colors (for IDP3 evaluation task of the user who has id 4). After analyzing all the images drawn for all users, we have noticed that majority of red points (max pressure) are in images for which users gave right answers. This notice may be useful in designing our graphical vibro-tactile language, since we can observe when the user decreases or increases his touch pressure. The increasing of pressure may indicate that the user touches graphical elements interesting for him, and the decreasing may indicate that the user touches graphical elements non-interesting for him.
During the tests we observed also that users try sometimes to scan the screen very quickly, it might be because they try to get a lot of information in a short time.
In this paper, we summarized our current work which aims to design an approach for non-visual access to web pages on touchscreen devices. The designed vibro-tactile protocol transforms the information viewed on the screen and touched by users to vibration by transforming light contrasts of touched pixels into low-frequencies tactile vibrations.
The obtained results are interesting, since we used in these experiments only one vibration motor of low level quality (phone vibrator), and the learning period was very short, so there are many enhancements to be achieved in next versions either on the hardware/software level or on the level of learning phase (increasing the number and quality of micro-vibrators, making more control on frequencies and amplitudes sent to microvibrators, adding vocal abilities to the current approach, integrating automatic intelligent methods for segmenting web pages,…).
In the same way that the environment enables a blind person to move in space with sidewalks and textures which will be explored by his/her white cane, we hope giving the blind user an ability to navigate documents depending on "textual sidewalks" and "graphical paths" which will be discovered by his/her finger. Next steps in this research will be 1) Adding elements to the graphical vibro-tactile language in order to represent more HTML elements such links, buttons, input fields, and other elements, 2) Making the program more interactive to guarantee vibrating in real time, and without any delay. 3) Including the results obtained in these pre-tests, for example focusing on areas which are most touched by the users, focusing on the variation of pressure for expecting the model of navigation, 4) adding more vibration motors to the designed circuit; current version includes 2 vibration motors, but we tested only one to know exactly how blind users navigate the screen using one finger; next tests may be on more vibration motors to discover how the blind users navigate the screen using more than one finger, 5) We plan also to add thermic actuators for translating the notion of colors. This may be very useful and hopeful for blind users to transfer information about colors, 6) after adding all mentioned desired changes to the circuit and to the program; we should test integrating the hybrid segmentation algorithm of web pages with the adapted version of the designed circuit to generate automatically the graphical elements of the navigated web page.
Acknowledgment This works is founded by the national agency of research ANR (Agence Nationale de Recherche http://www.agence-nationalerecherche.fr/ ) as a part of ART-ADN project ANR-12-SOIN0003-02 (Accès par Retour Tactilo-oral Aux Documents Numériques) in GREYC laboratory (www.greyc.fr) of the University of Caen Basse-Normandie (www.unicaen.fr) and the National Superior Engineering School of Caen (www.ensicaen.fr). Thanks to Rabeb BEN SASSI, and Abdelmajid TOUNSI for their participation in designing the system.