Recent advancements in handwritten text recognition (HTR) for historical documents have demonstrated high performance on cursive Arabic scripts, achieving accuracy comparable to Latin scripts. The initial RASAM dataset, focused on three Arabic Maghribi manuscripts, facilitated rapid coverage of new documents via fine-tuning. However, HTR application for Arabic scripts remains constrained due to the vast diversity in spellings, ambiguities, and languages. To overcome these challenges, we present RASAM 2, an extended dataset with 3,750 lines from 15 manuscripts in the BULAC library, showcasing various hands, layouts, and texts in Arabic Maghribi script. RASAM 2 aims to establish a new benchmark for HTR model training for both Maghribi and Oriental scripts, covering text recognition and layout analysis. Preliminary experiments using a word-based CRNN approach indicate significant model versatility, with a nearly 40% reduction in Character Error Rate (CER) across new in-domain and out-of-domain manuscripts.
other non-Latin scripts. The initial dataset, RASAM 1, included 300 images from three Bibiothèque des langues et civilisations (BULAC) manuscripts copied between 1734 and 1875, achieving promising results with an in-domain Character Error Rate (CER) of 4.8%.
However, the limited scope of RASAM 1 restricted its efectiveness in recognizing out-ofdomain manuscripts, even those with similar contemporary scripts and themes (see Table 1). To overcome these limitations, we introduce RASAM 2, an expanded dataset comprising 3,750 lines from fiteen manuscripts, encompassing a broader range of themes and handwriting styles. RASAM 2 aims to provide a comprehensive reference for training HTR models for Arabic scripts, enhancing their robustness and applicability across diverse Arabic Maghribi and Oriental texts. This paper presents the technical details of RASAM 2, its composition, and the initial results of using a new word-based Convolutional Recurrent Neural Network (CRNN) approach, which shows significant improvement in model versatility and a substantial reduction in CER for both in-domain and out-of-domain manuscripts.
©JJ.Ë@ ©J®Ë@ Commentary: The fā is confused with a bā (in both cases, a single point is subscribed).
Qå•Ag ñ“Ag Commentary: The rā is confused with a wāw (more or less open and long final).
XYªK. hYë Commentary: The pair of letters bā and ’ayn were confused with anhā (the subscript point of the bā was not spotted). The final dāl is confused with a ḥā, they may have a close ending.
àAÓQË@ àAÓñË@ Commentary: The rā of rummān (pomegranates) became a wāw, both often very close realisations - a possible example of a food word unknown by the model.
Që@ñm.Ì'@ Q唯@ñmÌ'@ Commentary: The first subscribed point is misunderstood and the ǧīm of ǧawāhir (jewels or gems) is confused with an ḥā. The unusually wide realisation of the hā is mistaken for a qāf (the dot on the line below is mistakenly equated with this line) followed by a ṣād. The rā is well understood.
The study of documents in Arabic constitutes a separate field within the handwritten text
recognition and document analysis questions more generally, owing to the great diversity and
variability they encompass, hence the workshops dedicated to this specific issue held at the last
ICDAR and ICFHR conferences. The latest developments in HTR for Arabic have however
demonstrated that the use of dedicated CRNN enables to overcome the issue of text
recognition for these scripts, with CER below 5%, even below 3% in specific cases, with few training
data [
In recent years, numerous datasets have emerged in an attempt to overcome these diferent
tasks. In the instance of non-historical documents, the IFN/ENIT dataset 1[
In the instance of historical documents, very specialized datasets exist, such as WAHD1[],
dedicated to writer identification, or KERTAS [
Together, these datasets are already covering a vast part of the production of documents in Arabic scripts (subject to their compatibility, see Table2). Although the proof of concept is successful for text recognition, the challenge today is to increase the versatility of existing models by providing a greater variety of fully annotated and transcribed documents.
Summary: RASAM 2 dataset comprises 250 images from 15 diferent manuscripts. 3,750 lines in total have been transcribed, 250 lines by manuscript on average, regardless of the type (main text or marginal notes). It entails 5,702 annotated lines in total and focuses on Arabic Maghribi manuscripts (see Table 5 in appendix for the complete list of manuscripts). Its purpose is to extend the variety of cases encountered in RASAM 1, in order to provide a robust training basis for documents in Arabic scripts.
• Dataset availability (v.1.0): https://github.com/calfa-co/rasam-dataset.
• License: Apache2.0
• Data format: pageXML with Text regions and lines
• Annotation tool: Calfa Vision3 [
The dataset holds 522,371 characters (divided in 54 classes) for a total of 93,855 words (divided in 22,027 classes). The ḍammatan and @ classes in particular are under-represented and are likely to be less encountered, and so less recognized in a character-based approach (see below Section 4). The words waw ( ð ), min/man ( á Ó ) and fī ( ú¯ ) are the most represented in the dataset, with 4,398; 2,246 and 2,189 occurrences respectively, a contrario the wordsal-akhdūd (XðYgB@), qaṭām (ÐA¢¯) and la‘ād (XAªË) are among the least represented (a single occurrence).
We retained four text regions and two annex regions for the semantic classification of contents: • MainZone: the main text region of the document. This region can appear several times within a single page, when the text is segmented or in case of a multiple column layout; • MainZone:title: text region located at the same level as the main text, for headings and stylized titles; • MarginTextZone: marginal text region regardless of its location in the page; • MarginTextZone:catchword: marginal text region corresponding to the catchwords, systematically under the main text region; • StampZone: stamps present on the page; • TableZone: region corresponding to a table.
A summary of the text regions distribution is given in Table3.
As outlined in the introduction, the aim of this new dataset is to enhance the versatility and
robustness of RASAM 1 by training it on a wider variety of manuscripts in order to expand the
base of its (1.) vocabulary, (2.) layouts and (3.) scripts. As a result, 15 manuscripts make up
this new dataset.
(1.) Of the fiteen new manuscripts, five (1/3 of the corpus) have themes and/or vocabulary
related to the first dataset. Like MS.ARA.1977 (RASAM 1), MS.ARA.1944 (RASAM 2) belongs
to the historical genre; and like MS.ARA.609 (RASAM 1), the manuscripts MS.ARA.1936, 1943,
1960 and 1983 deal with Islamic law – with the diference that, on the one hand, the legal issues
are not identical, which means that a new vocabulary has to be learned, and that, on the other
hand, MS.ARA.1936 also includes Berber written in Arabic. The other ten manuscripts of the
new dataset (the remaining 2/3 of the corpus) cover new themes, not yet dealt with by RASAM
1. In detail, MS.ARA.1947 is a classical Arabic literature text, MS.ARA.1926 a collection of
litanies, MS.ARA.23, 24, 45b and 1982 cover vocabulary related to Arabic grammar and linguistics.
MS.ARA.6, 9, 65 and 1946 consist of collections on various topics ranging from Islamic
jurisprudence to Arabic grammar, including private correspondence and exegesis of Qur’anic verses.
In addition, several manuscripts show significant variations in handwriting, particularly for
the latter collections.
(2.) From the layout perspective, the RASAM 1 dataset already covered complex layouts:
MS.ARA.609 integrated many tables within the body of the text and MS.ARA.1977 recorded
many lines of poetry which traditionally are ofset from the main text [
(3.) From a strictly palaeographic point of view, the RASAM 2 dataset intends to deal with
a broader variety of hands. The emphasis has been placed on three points in particular. (a.)
Firstly, particular interest has been given to the use of colors within these diferent manuscripts.
Some recent experiments conducted on the basis of RASAM 1 show that the use of colors largely
hinders the models’ good recognition of characters9[]. Therefore, many manuscripts in the
RASAM 2 corpus aim at providing the model with many color realizations (see MS.ARA.1926
and MS.ARA.6 supra, where blue, green, red and yellow are used in particular). (b.) Secondly,
RASAM 2 intends to be able to handle diferent text densities. RASAM 1 was indeed based
on only 3 manuscripts which, although diferent from the density aspect [
The latest developments in HTR for handwritten documents in Arabic scripts have shown that
operating a word-based CRNN (where every word is considered as a diferent class to identify)
outperforms a basic character-based CRNN (where each character is considered as a diferent
class to identify) on documents with a steady lexicon (both in learning time and CER)9[]. This
approach, despite being dependent on the targeted lexicon, relies on recognizing a word in
context, which appears a more robust approach for cursive Arabic scripts) 9[]. We hold onto
this approach, which is a variation of the one implemented for RASAM 1[
Lucas et al. have notably demonstrated that a fine-tuning strategy limited to 10 images (160
transcribed lines on average) for the Arabic Maghribi scripts, on the basis of a RASAM-trained
model is sufÏcient to reach a CER below 10% and to shorten the transcription work [
We are taking this fine-tuning approach from the RASAM model and testing it on two
samples: one in-domain sample, derived from RASAM 1 and RASAM 2, and one out-of-domain
sample derived from manuscripts from Lucas et al. [
In out-of-domain documents but belonging to the same family of scripts as RASAM 1 and 2, such as the Arabic Maghribi scripts, RASAM 2 demonstrates notable efÏciency, as evidenced in its application to TARIMA. Particularly noteworthy is its performance on Oriental scripts (RASM and Iskandar), where RASAM 2 not only outperforms RASAM 1 but also achieves significantly lower average CER scores (20.34 for RASM and 16.73 for Iskandar). These improved results not only enhance accuracy but also facilitate faster processing with minimal data requirements.
Besides the versatility of RASAM 2 model, Figure4 also shows its robustness with a very consistent CER per page and very little dispersion as in the case of RASAM 1. It is particularly visible on RASAM 2 dataset for which RASAM 1 model (out-of-domain test) reaches a CER between 11.67% (on the manuscript BULAC.MS.ARA.1982) and 48.80% (on the manuscript BULAC.MS.ARA.9).
A contrario, the CER of RASAM 2 model ranges between 1.71% and 28.47% in an in-domain instance, and between 7.26% and 26.88% in an out-of-domain instance. The extreme values are therefore practically twice as small as those for RASAM 1. Thus, there remain pages for which our new model does not immediately succeed in producing workable outcome, for these pages, it will then be necessary to adopt a fine-tuning strategy, which should be fast. 4 The median observed in Figure 5 is 27.97% for RASAM 1 for out-of-domain documents, and is reduced to 15.83% for RASAM 2, hence a 42% decrease in the error rate.
Figure 5 presents the average CER for each manuscript. In the in-domain instance, several manuscripts have a CER of less than 5%: this is the case for the manuscripts BULAC.MS.ARA.1943 (3.43%), BULAC MS ARA 1977 (4.91%), BULAC. MS.ARA.1982 (3.26%), BULAC.MS.ARA.1983 (3.58%), and BULAC MS ARA 45b (3.20%). The BULAC.MS.ARA.1936 and BULAC.MS.ARA.1947 manuscripts, even if they largely benefit from the new model, retain a high CER, higher than 15% and up to 16.25% for the BULAC.MS.ARA .1936 (compared with the 46.47% CER achieved with RASAM 1, but which is out-of-domain). 4In Lucas et al., a CER of 3.23% was reached with a diferent split and a slightly redesigned architecture, based on a meta-word-based approach (in the context of a specialized in-domain model). It also shows in particular that for the manuscript BULAC.MS.ARA.1957, the initial CER of 30.46% (RASAM 1) is reduced to 21.8% after a fine-tuning of only 20 lines. Applied to the same manuscript (see Figure5), RASAM 2 model obtains an initial CER of 25.5%9[].
In the out-of-domain instance, the gap between the results of RASAM 1 and RASAM 2 is narrower. If the manuscripts BULAC.MS.ARA.1922 (31.44% vs 26.38%) and BULAC.MS.ARA.1957 (35.95% vs 26.33%) retain a very high CER, the manuscripts BULAC.MS.ARA.1944 and BULAC. MS.ARA.1929 achieve a CER of 7.67% and 10.16%, better than the CER obtained in-domain for the manuscripts previously cited.
Despite the diversity of the TARIMA corpus, with both manuscripts and lithographs, the results remain very good. This is due to the proximity between the RASAM 1 & 2 dataset and the palaeographic characteristics of the TARIMA corpus, all of which are in Maghribi script. 4.2.2. Out-of-domain results (Oriental scripts) Out-of-domain results (Oriental scripts) RASAM 2 also demonstrates significantly enhanced efÏciency when applied to Oriental manuscripts, as illustrated by its performance with RASM and Iskandar. Its versatility is particularly evident in Iskandar, where the CER remains below 30%, with an average CER ranging between 8% and 20% (Fig. 4 and 5). Except for one manuscript (MS_Orient_A_02393), all the CER remain below 20% with RASAM 2. While RASM results exhibit some dispersion (albeit less than with RASAM 1), RASAM 2’s performance varies across the four manuscripts comprising the RASM dataset. Its highest result is observed in Dehli_Arabic_1901 (slightly above 16%), but none exceed 25%. The disparity in out-of-domain results between RASM and Iskandar likely arises from the diference in dataset adherence to RASAM guidelines. While Iskandar follows the RASAM guidelines, the RASM dataset diverges from them, which may explain the observed gap in CER results. For example, when the scribe omitted expected diacritics on certain letters, the transcriber left the letter without them, whereas the RASAM guidelines would have added the diacritics where necessary. This suggests that with minimal fine-tuning, RASAM 2 could readily adapt to various manuscripts, regardless of their script families.
RASAM 2 sets a new standard for the recognition of Arabic Maghribi scripts. Figure5 shows that it nevertheless produces many more errors than the average on four in-domain and outof-domain manuscripts, leading to an increase in the CER. Observation of the manuscripts (see Figure 6) reveals several situations where the CER decreases naturally.
Manuscript with vowel signs and numerous interlinear notes: This is the case of the
manuscripts BULAC.MS.ARA.1936 and BULAC.MS.ARA.1957 for which we observe an
important vocalization which is rarely present in these manuscripts. It leads, at this stage, to a greater
ambiguity of the forms to be recognized, but is however not insurmountable: a specialized
approach from RASAM shows for example that 160 lines are enough with a word-based approach
to reach a CER of 10.41% for the manuscript BULAC.MS.ARA.1957 [
Variation in line color: This is a phenomenon already observed in RASAM 1 1[
In conclusion, the RASAM 2 dataset ofers a high representativeness of Arabic Maghribi scripts. The word-based model trained on this dataset obtains very high in-domain and out-of-domain accuracies, achieving a 40-point CER reduction in all scenarios, which ensures an important coverage of Arabic Maghribi manuscript traditions. The dataset also demonstrates its versatility and can be easily fine-tuned on a new target, including Oriental scripts and new varieties of Arabic (Middle Arabic, Berber written in Arabic). In the future, we will study this transfer of RASAM models to other types of Arabic scripts, in particular Oriental ones. Additionally, we plan to conduct experiments using transformer-based models, as the critical mass of data for Arabic has now been reached, thanks to the RASAM team and all datasets produced within this scope. More generally, the datasets created in recent years around the RASAM team (TARIMA, Iskandar) have made it possible to create a set of open data decisive for the HTR of Arabic scripts.
This work was carried out within the framework of cooperation between the Research Consortium Middle-East and Muslim Worlds (GIS MOMM), the BULAC, and Calfa. It aligns with the scientific focus defined by the GIS MOMM, which prioritizes North African studies and digital humanities.
• RASAM 1 and 2 datasets: https://github.com/calfa-co/rasam-dataset • TARIMA dataset: https://github.com/calfa-co/tarima • Iskandar dataset: https://gitlab.huma-num.fr/lipa/iskandar
èYë ½ËX ÈA¯
MS.ARA.6 MS.ARA.9 MS.ARA.23 MS.ARA.24 MS.ARA.45b MS.ARA.65 MS.ARA.1926 MS.ARA.1936 MS.ARA.1943 MS.ARA.1944
Add.MS.7474 Add.MS.23494 Dehli.Arabic.1901 OR.3366
Btv1b10031486f Btv1b11003097s Btv1b110032696 Leeds.ms153
TOTAL 250 3,750 5,702 simple simple simple simple medium simple simple simple simple simple complex high complex high low low low low low low low low high high simple very low damaged Litany medium low damaged complex high damaged good good good good good good good good good good good good
Law Grammar