=Paper=
{{Paper
|id=Vol-1710/paper4
|storemode=property
|title=The Methodology of Automated Decryption of Znamenny Chants
|pdfUrl=https://ceur-ws.org/Vol-1710/paper4.pdf
|volume=Vol-1710
|authors=Marina Danshina,Andrey Philippovich,Irina Golubeva
|dblpUrl=https://dblp.org/rec/conf/aist/DanshinaPG16
}}
==The Methodology of Automated Decryption of Znamenny Chants==
https://ceur-ws.org/Vol-1710/paper4.pdf
The Methodology of Automated Decoding of Znamenny Chants
Andrey Philippovich1, Marina Danshina1, Irina Golubeva1
1
University of Mechanical Engineering, Moscow, Russia
{aphilippovich, mdanshina, igolubeva}@it-claim.ru
Abstract. The paper considers problems of applying information technology to
the processing of ancient music manuscripts. We propose to use automated
component methodology, which applies machine translation methods and
allows decoding Znamenny Chants into a linear notation. The present work
describes basic stages of the methodology. In order to implement the
methodology steps, the work develops and describes an automated system of
research studies - "Computer Semiography". The system consists of a module
of the chants' input into the database, review of the manuscripts in electronic
form, formation of a linguistic model of ancient musical chants, translation
models, and decoding of the chants. Music editor module allows users to play
the resulting melodies, while input of chants and translation rules makes it
possible for users to conduct researches without a reference to a particular
manuscript. As a result, these web services have been developed to assist
researchers, historians and musicians to process a wide variety of musical
manuscripts written in Znamenny notation.
Keywords: musical information technologies, decoding automatization,
statistical machine translation, linguistic model, semiography, ancient
manuscripts, visualization of Znamenny chants
1 Introduction
The modern system of linear notation of musical compositions provides us with
a method to describe the pitch and duration of notes with high accuracy.
However, in the XVII century, before the acceptance of this system, in ancient
Russia there was a widespread distribution of another method of tunes
recording - "Znamenny notation". In this method, melody was recorded with the
use of semiographic special characters - "znams" (Fig.1).
Fig. 1. Fragment of a znamenny manuscript
Musical manuscripts of the XVII century can be transferred to the modern
notation. There were special books at the turn of the transition to the new
records system– alphabets and dvoyeznamenniks, which contained translation
rules for certain znams and chants in two notations (znamenny and linear)
correspondingly. Unfortunately, earlier manuscripts (XI-XVI centuries) do not
contain special tips and the currently developed tips cannot be fully applied to
them. To offer a variant of translation of such manuscripts, it is necessary to
conduct thorough researches of the development of the Znamenny chant
system. Complexity of the translation is caused by a significant amount of
factors – not only lack of sufficient alphabets, but also existence of special
structures in chants (popevka, fita and others), which represent sequences of
znams with special interpretation. They can be decoded only by means of
special "instructions" (kokiznik, fitnik, etc.), like phraseological units in a
language.
Many famous scientists dealt with the study of Znamenny Chants. In the XVIII-
XX centuries D. V. Razumovsky (1818-1889) [1], S. V. Smolensky (1848-
1909) [2], M. V. Brazhnikov (1902-1973) [3] and other researchers worked on
drawing up specialized alphabets and the description of rules of znamenny
chants' execution. Some scientists were engaged in collecting the remained
musical manuscripts. Research of the znamenny chants is still a pressing
question. Modern scientists pay great attention to it: B. G. Smolyakov [4], B. P.
Kutuzov [5], E.G. Meshcherina [6], G. A. Pozhidayeva [7] and scientific
schools (projects), e.g. "Fund of znamenny chants" [8], "Dyache oko" [9], and
others [10]. Some researches are conducted with assistance of grants from
scientific funds in Russia (RGNF and others).
As a part of “Computer semiography” project [11], since 2000, scientists have
worked on solving problems of Znamenny Chants visualization [12]. They have
also conducted statistical researches [13], and investigated a possibility of
development of znams’ automated recognizer [14], aspects of musical
semiotics, and a structure of musical language [15, 16]. In 2011-2013 the
researches were supported by RGNF Grant No. 110412025v "The automated
system of scientific researches in the field of a computer semiography (ASNI
KS)".
Now researchers in the field of musical medieval studies tend to use modern
information technologies for accumulation of remote sources, their
systematization and automation of routine operations. However, these practices
are heterogeneous, demand generalization and development. This defines
relevance of the general component technique development of the Znamenny
Chants transfer into a linear notation. This would allow tracking transformation
of the manuscript from image to a melody, and combining the accumulated
knowledge of the studied manuscripts.
One of the most important requirements to a technique is a possibility of the
stage-by-stage and autonomous solution of separate tasks: the translation of
manuscripts in electronic form, drawing up electronic dictionaries, playing of
chants in Znamenny and musical notations.
2 General scheme of the technique
The proposed methodology of the automated decoding of Znamenny Chants
includes some components (stages), which can be realized both consecutively
and selectively, depending on the input data and research purposes. The
technique is schematically presented in Fig. 2, and below are its main stages:
1. Selection of basic data. Decoding of chants can be made depending
on the chosen chants.
2. Transfer of manuscripts into electronic form. Scanning and input of
chants, processing and saving of the information in the databases.
3. Development and setting up of the translated dictionary. Choice of
structuring methods and sources for creation of decoding rules;
carrying out necessary researches.
4. Transformation (decoding) of the manuscript by means of the
dictionary.
5. Preparation of the manuscript, materials and results of research for
electronic representation and playing in the web environment.
Fig. 2. The flowchart of the automated Znamenny Chants decoding technique
3 Selection of the source data
At this stage, it is necessary to define musical manuscripts and reference
materials, by means of which it will be possible to form translated dictionaries.
Ancient musical manuscripts are diverse, thus we need different materials to
evaluate and decode them. In simplified form, musical manuscripts with
Znamenny notation can be divided into the following groups:
A – alphabets (manuscripts, where certain znams correspond to certain notes
(one or several))
D – dvoyeznamennik (manuscripts with chants in two notations: znamenny
and linear)
K – kokiznik and fitnik (manuscripts, which represent a set of popevkas in
the Znamenny notation, grouped in chapters with the indication of each
popevka’s name)
Z – znamenny manuscripts (manuscripts, where chants are presented in the
Znamenny notation)
Sb – collections of popevkas (manuscripts, where popevkas are listed in the
Znamenny or linear notation)
Choice of the basic data usually depends on availability of sources and
possibility to use them for automated processing. In the future, it will be
possible to develop recommendations on how to choose certain manuscripts to
assess their completeness, reliability, and other parameters. Technically, the
problem of manuscripts selection for research (a set of IshRuks) can be
described with the following procedure𝜑! :
𝐼𝑠ℎ𝑅𝑢𝑘𝑠 = 𝜑! 𝑅𝑢𝑘𝑠 = 𝑟! , 𝑟! , … , 𝑟! , where
Ruks – all set of available manuscripts,
ri – certain manuscripts,
m – number of the chosen initial manuscripts.
In order to test the technique, we have chosen three trustworthy sources, which
represent different types of singing manuscripts:
Z: "A Circle of church ancient Znamenny chants in six parts" under D. V.
Razumovsky's edition. This manuscript is znamenny, which means that the
melody there is written down by means of znams, supplied with pometas. (1367
pages, 6 volumes).
A: Collection "Singing alphabets of Ancient Russia." Shabalin D.S. [18]
Sb: "Collection of popevkas of the Solovki meeting". This manuscript is a
collection of the popevkas, which are written down in the Znamenny notation.
Some of them are determined by words. The book is the fullest collection of
popevkas, classified by chapters (213 pages).
D: "Irmology" –dvoyeznamenny manuscript, where the melody is presented in
two notations: linear and znamenny (68 pages).
4 Technologies for manuscripts transfer into electronic form
Technologies for transferring musical manuscripts into electronic form could be
based on widespread approaches. To make chants visual in the web
environment, it is possible to scan, process and upload them on a web site in a
graphic format or PDF files. However, having a big amount of materials, there
will be tasks of creating an effective system of navigation and indexing of
graphic files.
Computer fonts. For research purposes, the manuscripts have to be entered in a
symbolic form, applicable for machine processing. At early stages of the
Computer semiografiy project (in 2000), it was decided to develop special
computer fonts, which would allow to set up chants in widespread text editors.
Within the following 5-10 years, there were also other znamenny fonts, the
main objective of which was visual representation of znamenny manuscripts.
A peculiarity of an AndrewSemio font is its orientation on maximum
functionality – convenience of input and the subsequent machine processing.
Fonts have undergone some stages of completions, from an experimental
version up to elaboration of an ergonomic component. In the final version,
similar znams settle down on one letter with a different tracing (normal, bold,
and italic). In the course of the researches, there have also been some
replenishments of znams – up to now scientists have revealed and presented
fonts of 202 znams.
Online editor of Znamenny Chants (IPSM) [20]. Initially, students entered
Znamenny Chants in a text editor of MS Word. However, special editor
programs were developed in the process of obtaining statistical data and further
expansion of a number of znams. At first, the programs were desktop, and then
they became web-oriented (IPSM).
Development of input technologies and updating of fonts required
improvements of data formats: in order to store the texts of chants users
switched from a simple *.doc format to, at first, Word tables and then to Excel
and XML. To realize the search queries or other resource-intensive operations,
it was also necessary to develop formats for data storage in Database
Management System (DBMS). The editor of Znammeny Chants (IPSM) was
developed to increase the efficiency of the input process and to continue
analyzing the entered manuscripts. This editor was made in the Python
language with the help of Django framework.
All znams in this editor are decoded by sequence of seven figures and grouped
according to their tracing similarity. At the first level, six main groups are
allocated, and each group has a maximum number of seven subgroups.
A special research was conducted before forming special groups [13, 15, 17,
18]. This research analyzed statistics of occurrence of znams in the previous
manuscripts. Figure 3 shows that the occurrence of znams is distributed
unevenly, therefore optimization of the web forms (Figure 4) allowed to
increase the speed and convenience of the input dramatically.
Fig. 3. Statistics of znams occurrence.
The developed application allows entering Znamenny Chants into the database
page-by-page. At the same time, the current page of the manuscript is entered in
the form of an image in a separate block on the page, where the current line of
the chant is largely displayed. If a user needs to upload a dvoyeznamennik, the
page will contain a special menu with the keyboard layout, which would allow
inputting the notes with the help of Odnoglas font [8].
Apart from the studied manuscripts, users can upload any chants (each record in
the database will correspond to one znam) and dictionaries (each record will
contain sequence of znams and notes corresponding to it in the database).
Fig. 5. Scheme example of the application
Fig. 4. Scheme example of the for inputting a translation dictionary.
application for inputting chants.
Checking of the entered records is carried out visually, comparing the initial
table with the electronic version. A special module is used to check that the
Znammeny Chants have been entered correctly. Based on the previously
conducted language module, this module checks probability of occurrence of
the entered znams sequences in the Znamenny manuscript [17].
Formally input procedure (𝜑! ) can be presented as a transfer of the manuscript
(r) to the sequence of znams (RZ) :
𝑅𝑍 = 𝑧! , 𝑧! , … , 𝑧! = 𝜑! 𝑟 , where
zi – manuscript znams,
𝑟 ∊ 𝐼𝑠ℎ𝑅𝑢𝑘𝑠 - the chosen manuscript,
m – quantity of znams in the r-manuscript.
5 Development and setting up of the translated dictionary
This stage assumes drawing up a dictionary, which would contain rules for the
translation of chants from Znamenny notation into a linear one. Several
methods can be used to form the dictionary:
• Creation of the dictionary on the basis of popevkas (productive
model),
• Creation of the dictionary on the basis of a dvoznamennik (n-gram
model),
• Combined method.
Decoding of chants is ambiguous, and all variants of the Znamenny Chants
transfer of the XI-XVI centuries are only assumptions. Therefore, it is important
for the researcher to receive, analyze and compare the transferred chants by
means of various methods.
Formation of the dictionary based on popevkas. In the first case, the
dictionary is based on popevkas. Previously prepared collection of popevkas
forms a set of rules. Each rule has a translation, based on a dvoznamennik or
alphabets.
𝑆𝑙 𝑟 = 𝜑! 𝑅𝑍, 𝑟 = 𝑝! = 𝑧! , 𝑧!!! , 𝑧!!! , 𝑛! , 𝑛!!! , 𝑛!!! , 𝑝
In order to provide independence of the translation from the initial sound and to
check a hypothesis of translation similarity for popevkas in different chapters, it
is necessary to keep interval sequence for each popevka in the database. This
code is not a designation of a note, but amount of half tones (intervals), based
on which the current sound differs from the previous one. Transformation of the
musical dictionary into the interval one can be presented as follows:
𝐼𝑆𝑙 = 𝜑! 𝑆𝐿 = 𝑧! , 𝑧!!! , 𝑧!!! , 𝐼𝑛𝑡𝑟! , 𝐼𝑛𝑡𝑟!!! , 𝐼𝑛𝑡𝑟!!! , 𝑝
During the automatic formation of the dictionary, it is offered to set a rule
priority, which would correspond to the quantity of znams in the rule. The
reason for this rule is that it is required to use the longest rules first. In the
course of the research, an expert can edit values of priorities, changing them to
what he considers to be more correct.
Formation of a dictionary based on a dvoznamennik. In the second case, a
dictionary is based on a dvoznamennik, which represents a case of parallel
chants (written down in two notations). It provides a chance to create N-gram
model of the translation. As a result of the dvoyeznamennik analysis, the
dictionary becomes similar to the one, which was built on the basis of
popevkas, but in this case the dictionary will consist of n-grams (n=1,2,3,4).
Probability of such rules can be calculated as a product probability of the
included n-grams.
The dimension of n-grams, which equals three, is chosen within the basic
functionality of the developed tools. Each trigram corresponds to a set of notes
and has a probability of trigram translation by these notes (Figure 6).
Fig. 6. Fragment of n-gram model.
The probability of each rule is calculated according to the formation rules of the
model translation in statistical machine translation: we calculate the probability
of P (n|z) for each pair , where z – sequence of znams, and n –translation
of this sequence. This probability can be made according to the following
formula:
!(!,!)
𝑃 𝑛 𝑧 = !(!) (2), where C(n,z) – number of times when the sequence
of znams of z is translated by n notes.
The set of trigrams in this case can be presented as follows:
𝑇𝑟𝑍(𝑟) = 𝜑! 𝑅𝑍, 𝑟 = 𝑧! , 𝑧!!! , 𝑧!!! , where 𝑧 ∊ 𝑅𝑍(𝑟)
Combined method. This method combines the dictionary, built on the basis of
popevka studies, with the dictionary, received by means of statistical machine
translation. Users need to set rules priorities so that rules set by the expert
would be applied in the first place, followed by rules from popevka, and the
rules received from the statistical n-grams translation at the end.
Drawing up of a general SL dictionary can be presented as 𝜑! procedure ,
combining other sli dictionaries:
!
𝑆𝐿 = 𝜑! ( 𝑠𝑙! )
!!!
Within the research of the manuscript, experts can construct frequent
dvoyeznamennik and carry out the analysis of all rules, which include a certain
znam. As a result of the research, experts can reveal some patterns and set up
new rules, edit or exclude the old ones. Editing is understood as a change of
probability or a priority of the rule, as well as a change of the znams and notes
included in the rule [19].
6 Transformation (decoding) of the manuscript by means of the
dictionary
At this stage of the technique, a manuscript is decoded by means of the
dictionary, which was built at the previous stage.
Choosing suitable rules for a trigram:
∀𝑇𝑟𝑍! ∈ 𝑇𝑟𝑍: 𝑃𝑟𝑇𝑟𝑧 = 𝜑! 𝑇𝑟𝑍, 𝑆 = {𝑃𝑟! , 𝑃𝑟! , … , 𝑃𝑟! },
where p – number of the rules suitable for the translation.
Choosing the best rule from a set of the rules applicable for translation:
𝐵𝑒𝑠𝑡𝑃𝑟𝑇𝑟𝑧 𝑃𝑟𝑇𝑟𝑧 = 𝜑! 𝑃𝑟𝑇𝑟𝑧 = 𝑧′! , 𝑧′!!! , 𝑧′!!! , 𝑛! , 𝑛!!! , 𝑛!!! , 𝑝
The translated manuscript remains in the database or in the file in the .xml
format. The set of znams of the initial manuscript will be transformed to a set of
notes with the use of the dictionary (one dictionary or a set of dictionaries).
𝑁𝑠 = 𝜑! 𝐵𝑒𝑠𝑡𝑃𝑟𝑇𝑟𝑧, 𝑅𝑧! = 𝑛! , … , 𝑛! ,
Where n – quantity of notes by which znams are transferred to manuscripts.
𝑍𝑣 = 𝜑!" 𝑁𝑠 = {< 𝑉, 𝐷 >} = {< 𝑣! , … , 𝑣! >, < 𝑑! , … , 𝑑! >},
where V – pitch, D – duration.
Structure of these chants in the database was influenced by the structure of the
singing manuscripts in the first place. Besides the znam (presented by a letter
and a tracing style), it is necessary to preserve a syllable, which came across
this znam, a pometa, a page of the manuscript, peculiarity of the page
arrangement (the first and the last symbol on the line, on the page), etc.
Fig. 7. An example of chants in XML format.
Znamenny Chants translation rules retain a znam code, notes, duration and
priority. The algorithm of decoding of the Znamenny Сhants is shown in Figure
8.
Fig. 8. Algorithm of the Znamenny Сhants decoding.
7 Preparation of the research results for electronic representation
At this stage of the present technique, it is possible to play the received chant,
which allows analyzing music «by ear», comparing multiple versions of the
translation and choosing the best one.
Software components have been realized for automatization of the concluding
parts of the technique. These components are functionally combined with the
help of «Semio_muz» music player.
Fig. 9. Example of the translation by a musical editor.
Development of ASNI KS. The Automated system of the scientific researches
"Computer semiography" was developed to realize the technique stages. Figure
10 illustrates the block diagram of ASNI. The main modules are: chants
inputting, Znamenny Chants decoding, Znammeny Chants researching, and
module of chants playing. At the first stage, all chants are entered into the
database, after that they are read out by modules of decoding, playing and
researching. Translation rules are stored in the database. In the final part of the
research, experts can add, edit or delete any rules from the database.
Dictionaries and chants, obtained in the process of decoding, are stored in the
database.
Fig.10. Block diagram of decoding.
8 Evaluation
First of all the developed technique is unique and allows experts to
completely avoid manual translation of Znamenny chants. Expert rules from the
final translation dictionary are used in combined method of translation.
Comparison of methods:
Method Translation Time percent Capacity Average % of
time per page (to manual factor not translated
method) signs
Manual 90 min. 100 % 1 25 %
translation
Translation by 60 min. 66 % 1,5 15 %
alphabet
Translation by 20 min. 22 % 4,5 2%
combined
method
Secondly, the final dictionary is continuously expanded by trigrams from
dvoyeznamennik. This way the researchers have an opportunity to get one of
the possible translations of the unknown part of the manuscript. The experts
can also perform musical analysis and create new rules or correct the existing
ones.
We can use methods of hybrid machine translation systems [21] for a quality
assessment of our machine translation methods:
1) BLEU score
2) TER (Translation-Error rate)
Thus, it is necessary to develop a specialquality assessment technique for the
developed method. We are going to research methods for an assessment of
machine translation and create a new method suitable for our system.
Unfortunately, at this moment the database of the Znamenny chants is not as
big as we would like. Therefore many metrics are equal to zero due to the lack
of some parameters. But some other methods provide good results. The initial
manuscript divided in two parts (1/3 and 2/3). The system uses 2/3 of the
manuscript for learning and 1/3 for checking. As it is one and same manuscript,
the dictionary contains the majority of the discovered trigrams and the
translation assessment indicators are very high. This results will be different for
other manuscripts.
Also the majority of these metrics are based on comparison of translated texts
with the reference (human) translation. We can get such reference translation
only from dvoyeznamennik and we use the same manuscript for the training of
the dictionary.
9 Work results.
The developed methodology is of great value for researchers, as due to a huge
number of sources (manuscripts, and theoretical manuals), it is rather difficult
to systematize and work them out carefully. It is caused, as a rule, by the fact
that it is necessary to analyze and carry out quantitative estimates of some
indicators manually. Information technology is not involved in this case
effectively. The developed set of services is intended to simplify the most time-
consuming operations, which researchers-medievalists face. As a result,
researchers spend less time on checking and calculating any data.
The offered decoding methodology of the Znamenny Chants allows producing
three main components required for the Znamenny Chants decoding: a
dictionary with the transfer rules from znams to notes, a version of the
translated manuscript into the linear notation, language model and model of
Znamenny Chants translation. Transfer of chants into electronic format makes
them more available.
This work has developed software, which allows inputting Znamenny Chants
into the database, edit them, and look through them. We have realized the
language construction and translation modules that allow transferring
Znamenny Chants into a linear notation. The program is also able to build
dvoeznamenny frequency dictionaries and indexes of znams.
References
1. D.V. Razumovsky, prot. Church singing in Russian (the historical
experience and technical exposition): In 3 parts. (1867, 1868, 1869).
http://www.seminaria.ru/raritet/razum_history.htm; - in Russian.
2. S.V. Smolensky. Alphabet of znamenny chant of elder Alexander
Mezentsev. (1888). http://www.seminaria.ru/raritet/azb_mezen.htm; - in
Russian.
3. Brazhnikov M., Ways of development and problem of decoding of a
Znammeny chants, XII—XVIII centuries (1949),
http://www.seminaria.ru/raritet/brazhn_ways.htm; - in Russian.
4. Smolyakov B. G. To a problem of decoding of the znamenny notation. —
In book: Questions of the theory of music, 3rd edition (1975), pp. 41 – 69. -
In Russian.
5. Kutuzov B. P. Russian Znamenny Chants. Publisher Andrei Rublyov
(2008). - In Russian.
6. Meshcherina E. G., Musical culture of medieval Russia (2007) - in Russian.
7. Pozhidayeva G. A., Singing traditions of Ancient Russia (2007). - In
Russian.
8. Project “Fund of Znamennyh Chants”, http://znamen.ru/index.php
9. Project “D'jach'e oko”, http://dyak-oko.mrezha.ru/index.php
10. Bakhmutova I.V., Gusev V.D., Titkova T.N. L-gramm Alphabet for
decoding of the Znamenny Chants, "The Siberian magazine of industrial
mathematics". – T.1, No. 2 (1998), pp.51-66.
11. Project "Computer semiography", http://compsemiography.ru
12. Danshina I.V., Danshina M. V. Visualization and development of the
electronic edition of the semiographic chants //School of sciences for
young scientists "Computer graphics and mathematical modeling (Visual
Computing)": theses and reports (2009), pp. 89-105.
13. Danshina I.V., Danshina M. V. Statistical research of the znamenny
notation. In: Intellektual'nye tekhnologii i sistemy. Sbornik uchebno-
metodicheskikh rabot i statey aspirantov i studentov. vol. 9. pp. 71-80.
NOK «CLAIM», Moscow (2007) - in Russian.
14. Vylomova, E.A.: Recognition System of semiographic chants. In:
Intellektual'nye tekhnologii i sistemy. Sbornik uchebno-metodicheskikh
rabot i statey aspirantov i studentov. vol. 9. pp. 58-70. NOK «CLAIM»,
Moscow (2007) - in Russian.
15. Golubeva, I.V., Philippovich, A.Yu.: Syntactic analysis musical texts.
Novye informatsionnye tekhnologii v avtomatizirovannykh sistemakh. 16,
257-262 (2013) - in Russian.
16. Golubeva of I.V. Semantik in musical sign systems. The Collection of
theses and articles of the Russian-German youth remote school of sciences
"The actual and perspective directions of creation of the systems providing
the semantic analysis of data in real time" on September 27, 2012. pp. 105-
108. - in Russian.
17. Philippovich, A.Yu., Golubeva, I.V.: Research of syntax of Semiographic
Chants. Proceedings of the higher education institutions. Problems printing
and publishing. 6, 147-163 (2012) - in Russian.
18. Danshina I.V. Research of znamenny chants as sign system. Materials IV
of the international scientific El'Manuscript-2012 conference.
Petrozavodsk, Izhevsk, 2012. pp. 73-79. - in Russian.
19. Danshina M.V.: Using methods of machine translation for analysis of
ancient music manuscripts. Novye informatsionnye tekhnologii v
avtomatizirovannykh sistemakh. 16, 263-266 (2013) - in Russian.
20. Online editor of Znamenny Chants (IPSM)
http://compsemiography.ru/project/ipsm/ -
21. Molchanov, Alexander. “PROMT DeepHybrid system for WMT12 shared
translation task.” (2012).