=Paper=
{{Paper
|id=Vol-415/paper-16
|storemode=property
|title=Speech and Braille tools to improve access to university scientific courses
|pdfUrl=https://ceur-ws.org/Vol-415/paper15.pdf
|volume=Vol-415
|dblpUrl=https://dblp.org/rec/conf/cvhi/Bernareggi07
}}
==Speech and Braille tools to improve access to university scientific courses==
https://ceur-ws.org/Vol-415/paper15.pdf
Conference & Workshop on Assistive Technologies for People with Vision & Hearing Impairments
Assistive Technology for All Ages
CVHI 2007, M.A. Hersh (ed.)
SPEECH AND BRAILLE TOOLS TO IMPROVE ACCESS TO UNIVERSITY SCIENTIFIC
COURSES
Cristian Bernareggi
Università degli Studi di Milano, Biblioteca di Informatica
via Comelico 39, 20122 Milano (Italy)
Phone: +39 02–50316409 , Email: cristian.bernareggi@unimi.it
Abstract: Visually impaired students meet problems in attending scientific university courses. The
main barriers concern reading scientific resources, which are usually available in formats not
accessible through Braille devices and speech synthesizers, processing mathematical expressions
through assistive tools and understanding and manipulating graphical representations. This paper
illustrates the aims of the @Science thematic network which was founded to share best solutions
about science accessibility and some of the emerging tools to process mathematical formulae
experimented by @Science members.
Keywords: Mathematics, accessibility, science, visually impaired, blind, LAMBDA system,
MathPlayer, thematic network
1. Introduction
Visually impaired students who attend university scientific courses usually come across with difficulties
concerning mathematical expressions and technical drawings. Writing, reading, understanding and
processing mathematical expressions are tasks particularly difficult for blind students.
Actually, the main barriers do not come from the complexity of mathematical concepts, but from the
lack of suitable tools to write and process mathematical expressions effectively, efficiently and
satisfactorily through speech and Braille output. Moreover, technical drawings are generally employed
in scientific courses both by teachers to illustrate concepts and by students for homework and tests.
Therefore, visually impaired students should be able both to explore graphical descriptions and to
generate and manipulate drawings.
According to the sight impairment and to the tactual skills, there exist techniques and assistive tools
which can be employed to improve non-visual access to technical drawings. In order to share best
experiences about non-visual access to university scientific studies and to assess and contribute to
widespread emerging technologies to support visually impaired students to study university courses,
the thematic network @Science was established.
The founding members are universities and associations for visually impaired all over Europe and it
aims to involve more and more institutions. In this context, some emerging tools were experimented.
The following sections introduce the objectives of the @Science thematic network and some of the
tools which have been experimented up to now.
C. Bernareggi
2. @Science Thematic Network
2.1 Network aims
In the framework of the European Union eContentPlus Programme, the thematic network @Science
was founded. This network is co-ordinated by the Università degli Studi di Milano in Italy. The project
started in October 2006 and has a duration of 2 years. The network aims to facilitate visually impaired
students to access university scientific courses. Digitally available scientific resources are of crucial
importance for visually impaired students. So, the network members will collect and share practices
about how to use scientific digital resources through speech and tactile assistive tools. At the moment,
there exist many differences among European countries as for science communication to visually
impaired students.
Some of the national peculiarities may constitute a communication problem (e.g. language dependent
educational resources or differences in Braille codes), whereas some techniques can be shared all
over countries (e.g. techniques to prepare tactile representations, programs to explore tactile
representations in specific knowledge domains such as function diagrams, web-based scientific
resources, and more). By supporting communication and knowledge transfer among universities,
digital content providers, institutions for visually impaired and libraries about these specific
accessibility issues, the @Science network aims to prepare guidelines to produce digital scientific
documentation accessible by visually impaired students in university courses. The founding
consortium consists of the co-ordinating Università degli Studi di Milano (Italy), the University of Linz,
Institute Integriert Studieren (Austria), the Katholieke University Leuven (Belgium), the Comenius
University, faculty of Mathematics, physics and informatics (Slovakia), the Union of the Blind in Verona
(Italy) and the Pierre et Marie Curie Université (France).
2.2 Methodology
In order to achieve the aims above mentioned, @Science members will undertake mainly information
and collaboration actions. Information actions will be undertaken to widespread both general
background information about the problem of non-visual access to science and about specific
solutions (e.g. emerging technologies, success experiences in some learning tasks such as learning
from diagrams, etc.). Information activities will be achieved especially through contributing to
conferences and by organizing thematic events. Collaboration actions mean to involve individuals,
institutions (e.g. universities, schools, associations for visually impaired, libraries) and companies (e.g.
assistive technology manufacturers, publishers, online content providers) to discuss and share
experiences about access to digital scientific resources and select and share the best solutions. Some
of these activities are set through the @Science web portal.
The @Science website is the access point to the thematic network @Science on Internet. It can be
browsed at the address: http://www.ascience.eu. This website collects pages concerning the aims of
the thematic network, how institutions can join the network, how to contact network members and how
to subscribe to the services being delivered by the network (e.g. the newsletter). It is divided up into
four sections: events, projects, articles and tools. The events section collects information about up-to-
date events (conferences, workshops, etc.) about science accessibility issues. @Science members
contribute to many of these events and prepare reports on the emerging opportunities to be read on
the website.
The projects section collects reports about the latest ongoing projects on specific issues. Both
international and local projects are analysed. The articles section contains both informative papers on
specific issues developed by the network members and abstracts of useful papers available from other
content providers. The tools section stores demo releases or free programs which can help visually
impaired students overcome certain barriers in scientific studies. Moreover, the web portal provides
access to a forum and a mailing list about science accessibility. Individual studying and teaching
experiences, assistive strategies, technical solutions are some of the topics discussed in the forum
and in the mailing list.
2
C. Bernareggi
3. Tools To Process Mathematical Expressions
3.1 Introduction
This section focuses on some of the emerging tools which promise to improve access to mathematics
for visually impaired students. At first, the LAMBDA system is illustrated. It is a system which aims at
facilitating blind students to process mathematical expressions in an educational context.
Subsequently, MathPlayer, a tool to enable exploration of mathematical expressions in web pages, is
presented.
3.2 The LAMBDA system
The LAMBDA system (Schweikhardt et al., 2006) is the result of the European LAMBDA project
(acronym for Linear Access to Mathematics for Braille Device and Audio-synthesis) in the framework
of the IST program (further information can be found at www.lambdaproject.org). The LAMBDA
system aims to facilitate reading, writing and processing of mathematical expressions through Braille
and speech output, especially taking into account the needs of visually impaired students who share
their learning experience with sighted students in an educational context (e.g. in a classroom,
attending a lesson, taking an exam, etc.). In order to achieve high modularity and extensibility, the
LAMBDA system is made up of many functional components, which interact to perform the required
tasks. The main components are: the LAMBDA code, the mathematical editor, the LAMBDA font and
the converters.
The LAMBDA system is designed to process text and mathematical expressions mainly through Braille
and speech output, without necessarily embossing on paper. So, the LAMBDA consortium chose to
use 8-dots Braille combinations to represent mathematical symbols. Eight-dot Braille codes are often
incomplete and not officially approved by national Braille authorities. Therefore, in order to overcome
these national differences, a set of rules was defined to uniformly describe Braille mathematical
expressions in a linear form. Hence, a common markup language to linearly represent mathematical
expressions so that they can be easily read and understood by means of eight dots combinations was
developed. It means that dots combinations representing letters and mathematical symbols are
different from country to country, but they comply with the same set of linearization rules. The resulting
LAMBDA code follows some leading principles:
• whenever possible the meaning of the linearly represented notation is made explicit;
• it aims to achieve compact linear representations;
• eight dots configurations are used;
• it aims to preserve national dots configuration at least for the most common mathematical
symbols;
• intuitive rules and dots combinations are designed. For example, prefixes are employed to
change the meaning of a group of symbols;
• rules which exploit the blank character are not used;
• it aims to be extensible. New notations can be described by adding symbols which comply
with the markup rules.
The mathematical editor is the functional component which implements the strategies devised in order
to make easy to read, write and manipulate text and mathematical expressions by means of speech
output and Braille display, in an educational setting. The main assistive features concern:
• the possibility to move the focus wherever in the editing area. It means that the focus can be
moved directly to a specific position also after the end of line and the end of file. This operative
mode was designed to be used when a certain two-dimensional layout has to be preserved
(e.g. to arrange columnar layouts to solve arithmetic operations);
• the possibility to input the mathematical symbols by name from a list;
• mnemonic short-cut keys to input mathematical symbols. Short-cut keys can be customized;
• input based on the context. When a symbol which opens a structure is input, by pressing a
key combination it is possible to close the last input symbol;
• exploration of mathematical expressions by looking through their building block structure;
• selection, cut and paste operations on meaningful portions of the expression (e.g. selection of
the numerator, denominator, argument of a function, etc.);
3
C. Bernareggi
• a working paradigm to perform calculations with matrices [C. Bernareggi, 2006].
All of the assistive features embedded in the editor can be extended or customized through a Python-
based scripting language.
In order to make easy understanding by sight what the visually impaired student is writing, a special
font is available in the LAMBDA system. It maps Braille specific symbols (e.g. open fraction, close
fraction, etc.) to shapes which can be understood by sight.
At present the LAMBDA system embeds a converter from MathML to the LAMBDA document and
viceversa. It is still not totally complete. Nevertheless many expressions concerning many
mathematical areas can be successfully converted.
3.3 Using the LAMBDA system in university scientific courses
The LAMBDA system is one of the latest attempts to create a working environment which can be
adapted to national needs. Anyway, because of the choices concerning the eight dots mathematical
Braille code, it is innovative with respect to many local traditions. Therefore, its real impact on blind
students will strongly be related to how much the mathematical Braille code will be appreciated and
used. Some features are of special interest for university courses. At first, the possibility to move the
focus wherever in the working window enables Braille display users to perform particular tasks based
on a two-dimensional layout. For example, digits arranged in a tree can be rather quickly written and
processed (e.g. that helps understand algorithm complexity on trees). The Python-based scripting
language can be successfully employed by expert university students to automate advanced tasks
(e.g. to explore a graph represented by its adjacency matrix in the document). The possibility to
extend the Braille code is particularly useful for new notations which are usually introduced in
advanced courses (e.g. bra-ket notation in quantum mechanics). Anyway, at the moment, an easy to
use user interface to extend the underlying mathematical Braille code is still not available. Processing
matrices is a very hard activity for visually impaired students. So, the paradigm to work with multiple
matrices implemented in the LAMBDA system can be of great help especially in linear algebra courses
as well as to compare data sets (e.g. in statistics).
3.4 MathPlayer
MathPlayer (Soifer, 2005) is a plug-in fwhich extends the Microsoft Internet Explorer web browser. It
was designed to render MathML in web pages. MathML is not an image format, so MathPlayer is able
to interpret the markup language and produce the output. In particular, it is able to dynamically display
a mathematical expression according to its font and the color set, and it applies rules to generate
speech output. Hence, users can choose the most suitable font or color scheme for their reading
needs. For example, visually impaired readers are likely to set a large font and high color contrast.
MathPlayer properly works with some of the most known screen readers such as JAWS, Window-
Eyes and HAL. At present, MathPlayer does not use prosody to render mathematical expressions by
speech output. It generates text strings made up of the names of mathematical symbols and commas
and periods to set pauses. The text messages produced are read by speech synthesizers. At the
moment, screen readers use an off-screen model, which enables users to navigate through the
spoken verbal description of the math without any structural navigation strategy. Design Science is
working to embed in MathPlayer customized navigation paradigms. MathPlayer also allows web
browsers users to copy a MathML expression and paste it in a MathML-aware program. This is
particularly useful for computation, but might also be useful when used in conjunction with other
software aimed at making math accessible (e.g. the LAMBDA system) or with mainstream applications
used to process scientific documents (e.g. MathType or Scientific Notebook).
3.5 Using MathPlayer in university scientific courses
MathPlayer opened up many opportunities to visually impaired students who need access to digital
scientific resources. Embedding MathML expressions into web pages will make possible reading of
scientific web resources which are at present available only as images. So, for example, scientific web
sites (e.g. MathWorld) could soon make available content as MAthML. That would increase the
4
C. Bernareggi
amount of scientific digital resources available. In university courses, visually impaired students
seldom have scientific books in accessible formats. So, web resources could be of great help in the
learning process. Furthermore, MathML-aware mathematical editors could be used by professors to
prepare lecture notes which could be accessible through MathPlayer. Nonetheless, at the moment
MathPlayer presents some disadvantages concerning speech internationalization and braille output. In
particular, only the English language is available and no MathML into national Braille codes converter
jointly works with MathPlayer. So, the Braille reader reads on the Braille line the verbal description of
the mathematical expression. Due to the necessity to enrich the verbal description with commas,
periods and letters to generate speech output, the braille reader will not be able to read the
mathematical expression correctly on the Braille line. An example follows:
Screen display:
Figure 1 Screen display of the formula in traditional mathematical notation
Speech output:
square root of ay squared plus bee squared, end root. equals sine of x
Braille display output:
square root of ay squared plus bee squared, end root. equals sine of x
4. Conclusions
The tools introduced show a development direction based on the use of MathML markup language in
conjunction with assistive technologies to generate speech and Braille output. Following this line,
further software evaluation will be achieved in the context of the @Science thematic network and the
best tools will be documented. Up to now, the @Science network has undertaken qualitative
evaluation of some tools about access to mathematical expressions. Further work will concern also
access to technical drawings and tools to produce accessible scientific documentation.
References
Schweikhardt, W., C. Bernareggi, N. Jessel, B. Encelle and M. Gut, (2006). LAMBDA: A European
system to access mathematics with braille and audio synthesis, Proceedings of ICCHP06, LNCS,
vol. 4061, pp. 1223-1230, Springer
Bernareggi, C. (2006). Accessibility of two-dimensional structures in science learning, Proceedings of
the 2nd International Conference on Automated Production of Cross Media Content for Multi-
Channel Distribution, Firenze University Press
Soiffer, N. (2005). MathPlayer: web-based math accessibility, ASSETS 05, ACM Press, pp. 204-205
5