The accessibility of web pages from public institutions is important in ensuring equal access to information and services for all individuals, including those with disabilities. In this project, we aim to improve the accessibility of the web page from the Government of La Rioja. In particular, we are focused on three aspects that involve applying diferent natural language processing techniques. First, we will automatically caption all the images from the web page; second, we will provide transcriptions of all the videos from the Government of La Rioja; and, finally, we will improve the readability of the contents of the web page. In summary, this project is a first step towards making web pages from public institutions adaptable to the needs of each particular user that visits them.
the information of web portals with relevant information for citizens, for instance the web site of a public administration, this project is divided into three parts: © 2023 Copyright for this paper by its authors. Use permitted under Creative CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g CCoEmmUoRns LWiceonsrekAstthribouptionP4r.0oIncteerenadtiionnagl s(CC(CBYE4U.0)R.-WS.org)
2Ley de Accesibilidad Universal de La Rioja (10L/PL-0017) • Part I: Image accessibility. • Part II: Video accessibility.
• Part III: Easy read and Plain Language.
Usually, the description of images is a task conducted by a human annotator responsible for describing the Figure 1: A herd of sheep (description of the model) - Sheep image’s content and obtaining the appropriate metadata on a cattle track (description given). to facilitate accessibility to users. However, in public portals of the administration, it is common to find images that are already published but, due to their high number, processing [9]. For this purpose, the Keras library is used, it is unfeasible to process them individually by a human accessible through the reference [10]. More recently, annotator. models such as BLIP-2 [11] have emerged, which allow
The generation of the description of an image is a information to be extracted from images.
problem that can be carried out by Artificial Intelligence In previous works of the group, that are in a proof of
methods from two fields: on the one hand, computer concept stage, we have explored Deep Learning
mechavision methods to acquire the image content; and on the nisms to label authentic images extracted from an active
other hand, language models to convert the image con- portal, obtaining encouraging results and partial
suctent into meaningful words in a correct order [
Learning techniques applied to images, has already been put into practice, in particular, by this work team and, in general, by the research group within the University of 3.2. Video accessibility La Rioja [4] and [5]. The action of extracting text from audio is known as Au
In order to be able to apply this type of method, it tomatic Speech Recognition (ASR), the process by which is necessary to carry out a first step of analysis of the a computer system identifies spoken words [ 12]. We use initial data. For web portals under investigation, the set this process in our daily lives, as it is used in many arof images is usually large and exceeds 200,000 records. eas, such as voice assistants or dictation systems. For This large volume of images requires a preprocessing this purpose, we work with Recurrent Neural Networks step to filter the images and, for example, remove those (RNN) [13], which are neural networks with memory that are repeated or those that are thumbnails of larger that work with sequential information. Each neuron that images. forms it represents a temporal moment and can pass the
Some of the techniques to tackle this problem are based collected information to the next one. on using the model known as VirTex [6]. This model is There are diferent ASR classes depending on the spoaccessible at [7] and is trained with images obtained from ken text, like single words, connected words, continuous the web site htps://redcaps.xyz/, which collects images speech, and spontaneous speech. The class that is of most with descriptions. This model was created to add captions interest for this activity is continuous speech. In addition, to images automatically; see Figure 1. there are two types of models: acoustic models, which
Other known methods that may be useful in the de- extract information such as the speaker’s gender and velopment of this activity are those based on the use dialect, and language models, which deal with what conof Convolutional Neural Network (CNN) [8] and Trans- stitutes a possible word, which words may go together, formers, a method commonly used for natural language and in which sequence.
split into four parts: • Separating the audio from the image. • Transcribing the audio to text. • Aligning the transcribed text with the audio. • Reassembling the video (image + audio + text).
The second step requires studying known models for creating transcribed text. Some of these models, such as Wav2Vec or PocketSphinx, are pre-trained models for both English and Spanish, but there are others that have only been pre-trained for English. The latter requires dealing with the transition between English and Spanish since these models do not have a direct application in videos with Spanish audio. The study and subsequent evaluation of the diferent models will result in selecting a model to be used with the diferent videos to obtain the text of their subtitles.
Once we have the text transcribed correctly, we must proceed to assemble it with the video — in this phase the problem of synchronizing the audio with the text appears. For this step, we must take into account that each person speaks at a diferent speed, so we cannot standardize the alignment. To solve this problem, we can use a neural network known as Connectionist Temporal Classification (CTC) [14], which is used to train Recurrent Neuronal Networks (RNNs), which, as mentioned above, are a very useful tool in problems such as speech recognition or handwriting. This kind of network allows us to take into account the context and not only the last step.
The technique described here are in constant evolution, and, in addition to the aforementioned methods, in September 2022, a new ASR system was published known as Whisper [15]. Using this model, in a previous work by the group, we have explored the feasibility of captioning all videos present on a live web portal. From this first proof of concept, it has been inferred that this is a computationally expensive endeavor, but is feasible once a suficiently eficient model has been generated.
be confused; so we define them as follows.
Easy read is a way of adapting information to make it easier to read and understand text for people with reading dificulties. It is a method of adaptation with simple and clear language. Easy read is aimed at a number of groups with certain reading comprehension dificulties. Some of them are the following: • People
dyslexia). • People with low literacy or little schooling. with learning dificulties (such as • Foreigners or immigrants who do not have a good understanding of the Spanish language. • Children who need reading reinforcement. • Deaf people with comprehension dificulties. • Elderly people with mental disorders. • People with hyperactivity and attention deficit disorders. • People with intellectual or developmental disabilities (such as autism, aphasia, and so on).
a number of guidelines must be followed. One of the first documents on how to produce Easy-to-Read text was published by IFLA (International Federation of Library Associations and Institutions) [16]. There is a second document, which was produced by several Europe organizations, under the title “Information for All” [17]. In addition, there are some adaptations classified by Plena Inclusión [18].
Easy read does not have a fixed standard, but there are proposed diferent levels since it is impossible to adapt a text in the same way for all people with dificulties, as these are very diverse. This adaptation refers to both text and images or any other element that can be incorporated into a document (graphics, diagrams, and so on). IFLA establishes three levels, similar for both original Easy Read and adapted Easy Read works: • First level. In this simplest level, there are many images and little text. The text has a low syntactic dificulty. • Second level. In this intermediate level, less simple than the previous one, text is written using a vocabulary and expressions that are known to everyone, and it is easy to follow and understand.
Images are also used at this level. • Third level. In the most complex levels, text are longer, unusual words are used, and jumps in time and space appear. At this level, there are few images.
user to whom the adapted text is addressed.
On the other hand, we will use the definition given by the International Federation of Plain Language in relation to the concept of clear text or plain language: “A communication is written in plain language if its wording, structure, and design are so transparent that the readers to whom it is addressed can find what they need, understand what they find and use that information”.
Plain language benefits the general public, but it is not intended for people with comprehension dificulties, such as Easy read. Although there is no current regulation, an international ISO standard on plain text is being worked on, and it is supported by diferent organizations that promote it (Clarity International3, Plain Language4 • Sentence Boundary Detection. or the International Plain Language Federation5). Plain • Named Entity Recognition. language has become widespread, especially in the ad- • Entity Linking. ministrative and legal fields, which handle texts that are • Similarity. aimed to the general public, but with highly technical • Text Classification. language that is sometimes dificult to understand.
Many features of any text can be modified or trans- • Rule-based Matching. formed to make it more readable and understandable. These tasks will allow us to pre-process the text in The goal of adaptations to Easy read and/or Plain lan- order to generate a model using Deep Learning methods guage is to transform complex sentences into simpler with the objective of automating the creation of Easy ones [19]. Adaptations also include the way the text is read and Plain language. displayed.
Text simplification is usually based on the following four tasks: 4. Conclusions • Lexical simplification aims to replace dif- The presented project aims to study existing tools and ifcult words with easier words (synonyms) that propose new ones to address the improvement of accesare considered better to understand or read, as sibility in web portals that are of interest to the general long as the meaning is not altered. public, such as the web sites of public administrations. • Syntactic simplification aims to trans- This project consists of three parts, which can be conform long phrases or sentences containing syn- sidered separately, but also can be seen as interconnected. tactic figures that are unreadable or incomprehen- For instance, both image descriptions and video subtitles sible into simpler ones and in active form (passive should be understandable to anyone who reads them, and form should be avoided whenever possible). this implies the use of plain and easy to read text. • Eliminating information consists in reduc- The ultimate goal is to have a web page that adapts ing phrases or sentences, keeping the essential the accessibility of the provided information based on information, and eliminating unnecessary details the users visiting it, allowing them to choose the level of that do not add anything new to the idea to be accessibility thatthey prefer when accessing the informatransmitted. tion. • Adding information provides extra knowledge that may help the reader to understand and learn the meaning of one or more unknown terms.
These four simplification modes are related, and some- Science group of University of La Rioja. In particular, times a mixture of them is needed to maintain coherence several members of this team provide their experience and obtain the final text. in the areas of Computer Vision and Natural Language
There is a wide variety of software tools for Natural Processing that are necessary to conduct this project. Language Processing (NLP) in diferent languages, such as NLTK, spaCy, or Scikit-learn. In this project, we will study these tools to apply them to Easy read and Plain Acknowledgments language.
These libraries provide objects that help to represent The work was partially supported by the project text elements, such as sentences and words. They are PID2020-115225RB-I00 funded by MCIN/AEI/ used to extract information, as natural language under- 10.13039/501100011033 and PID2020-116641GBstanding systems or for pre-processing text before apply- I00 funded by MCIN/AEI/ 10.13039/501100011033 as well ing Deep Learning techniques. as by transfer projects OTCA211018 and OTCA221110.
The main tasks to be performed are as follows: • Tokenization. • Part-of-speech (POS) Tagging. • Dependency Parsing.
• Lemmatization.
[3] A. Roy, A guide to image captioning., 2020. materiales de lectura fácil, 2012. URL: URL: https://towardsdatascience.com/a-guide-to- https://www.ifla .org/files/assets/hq/publications/ image-captioning-e9fd5517f350. professional-report/120-es.pdf . [4] D. Lacalle, H. A. Castro-Abril, T. Randelovic, [17] Inclusión Europa, Información para todos., C. Domínguez, J. Heras, E. Mata, G. Mata, Y. Mén- 2016. URL: https://www.plenainclusion.org/ dez, V. Pascual, I. Ochoa, SpheroidJ: An open-source publicaciones/buscador/informacion-para-todosset of tools for spheroid segmentation, Computer pautas-europeas-de-la-lectura-facil/. Methods and Programs in Biomedicine 200 (2021) [18] O. Garcia-Muñoz, Lectura Fácil - Méto105837. dos de redacción y evaluación, 2012. URL: [5] A. Inés, C. Domínguez, J. Heras, E. Mata, V. Pascual, https://www.plenainclusion.org/publicaciones/ Biomedical image classification made easier thanks buscador/lectura-facil-metodos-de-redaccion-yto transfer and semi-supervised learning, Computer evaluacion/.
Methods and Programs in Biomedicine 198 (2021) [19] X. Wan, Automatic Text Simplification, Computa105782. tional Linguistics 44 (2018) 659–661. [6] K. Desai, G. Kaul, Z. Aysola, J. Johnson, Redcaps: web-curated image-text data created by the people, for the people, 2021. [7] U. Vision, Image captioning with virtex model trained on redcaps, 2022. URL: https://huggingface.co/spaces/umichVision/ virtex-redcaps. [8] K. O’Shea, R. Nash, An introduction to convolu
tional neural networks, 2015. [9] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, A. Rush, Transformers: Stateof-the-art natural language processing, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Association for Computational Linguistics, Online, 2020, pp. 38–45. [10] M. Hodosh, P. Young, J. Hockenmaier", Framing image description as a ranking task: Data, models and evaluation metrics, Journal of Artificial Intelligence
Research 47 (2013) 853–899. [11] J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, arXiv preprint arXiv:2301.12597 (2023). [12] K. Doshi, Audio deep learning made simple: Automatic Speech Recognition (ASR), How It Works., 2021. URL: https://towardsdatascience.com/audiodeep-learning-made-simple-automatic-speechrecognition-asr-how-it-works-716cfce4c706. [13] S. Hochreiter, J. Schmidhuber, Long short-term
memory, Neural computation 9 (1997) 1735–1780. [14] A. Hannun, Sequence modeling with CTC., 2017.
URL: https://distill.pub/2017/ctc/. [15] A. Radford, J. W. Kim, T. Xu, G. Brockman,
C. McLeavey, I. Sutskever, Robust speech recognition via large-scale weak supervision, 2022. [16] International Federation of Library Associations and Institutions, Directrices para