=Paper=
{{Paper
|id=Vol-2543/rpaper09
|storemode=property
|title=Challenges of Publishing Online Scholarly Journals with Multimedia Content
|pdfUrl=https://ceur-ws.org/Vol-2543/rpaper09.pdf
|volume=Vol-2543
|authors=Nikolay Borisov,Valentina Zakharkina,Irina Mbogo,Dmitry Prokudin,Pavel Scherbakov
|dblpUrl=https://dblp.org/rec/conf/ssi/BorisovZMPS19
}}
==Challenges of Publishing Online Scholarly Journals with Multimedia Content==
https://ceur-ws.org/Vol-2543/rpaper09.pdf
Challenges of Publishing Online Scholarly Journals
with Multimedia Content
N.V. Borisov [0000-0002-1671-5525], V.V. Zakharkina[0000-0002-4950-2410], I.A. Mbogo[0000-0002-
5942-7143]
, D.E.Prokudin[0000-0002-9464-8371], P.P. Scherbakov[0000-0003-1158-7460]
Saint Petersburg State University, Saint Petersburg, 7/11 Universitetskaya Emb., Russia
n.borisov@spbu.ru, v.zaharkina@spbu.ru, i.mbogo@spbu.ru,
d.prokudin@spbu.ru, p.scherbakov@spbu.ru
Abstract. The article discusses the format of electronic scholarly journals and
publications focused on online presentation of research results. Issues of creat-
ing tools for electronic scholarly publishing are considered. Problems related to
the need for including multimedia data are described. An approach to creating
an online multimedia designer of the article is considered, online tools of the
editor and author of the article are proposed.
The organization of the component structure of the article on the basis of
fields, which allows for effective presentation and citation, is proposed. With
this approach, embedded media objects can be organized into self-sufficient
collections. An additional condition for the formation of the article is an online
layout tool based on a set of functional components. Consideration of multime-
dia components of the article as separate entities of the data structure gives flex-
ible possibilities of their visual interactive representation.
An additional advantage of the component approach is the ability to expand
the range of types of multimedia objects.
A working prototype of a multimedia scholarly publication is presented as
the implementation of the described approach. The prototype of the journal
serves as a starting point for the development of the component model and
demonstrates the first stage of development.
Keywords: Scholarly Publication, Electronic Scholarly Journal, Multimedia
Content, Electronic Collection.
1 Introduction
Currently, the results of academic research are often presented in multimedia form.
Electronic collections of multimedia documents are created when processing of the
results of humanities research by archaeologists, folklorists, ethnographers, etc. To-
day, almost all scholarly journals have their electronic versions. Some magazines are
fully electronic (i.e., web-based). This poses the problem of providing adequate tech-
nological capabilities for creating scholarly publications that integrate multimedia
content.
Copyright © 2020 for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
94
A modern scholarly article can utilize a rich array of dynamic objects going be-
yond traditional illustrations: 3D models, video, 360-degree video, image galleries,
interactive elements of virtual reality (VR) and others.
2 Web Publications
To ensure long-term preservation of information resources and exchange of meta-
information, a number of frameworks have been created. These are widely used in
research and education. Below are some of the examples of systems used for electron-
ic scholarly publications:
1. Support systems for digital repositories (institutional repositories) are widely used
to build open-access archives and electronic libraries allowing to create, store, and
distribute digital materials. These systems include open-source software platforms
such as DSpace, EPrints, GreenStone, Fedora, and others. We estimate that DSpace
is currently the most popular software for creating archives of electronic resources
(digital repositories) in an academic context. The DSpace platform was developed
jointly by Hewlett-Packard and the MIT (Massachusetts Institute of Technology)
Libraries [1, 2].
2. Some electronic scholarly journals use electronic publishing systems such as Open
Journal Systems (OJS). The main advantage of this approach is the support for a
complete publishing cycle of electronic publications. At the same time, the format
of the scholarly publication remains traditional. Materials can be presented in two
formats: PDF and static HTML. It is worth noting that HTML layout is performed
outside the system and ready-made files are expected to be loaded into the OJS.
The system does not provide for the implementation of typesetting, publishing, and
displaying of multimedia content. Lack of appropriate web interfaces leaves the
typesetting and publishing processes in “manual” mode, which implies the availa-
bility of the relevant knowledge and skills from those involved in the publishing
process.
3. Integration issues are addressed through metadata exchange technology based on
the OAI-PMH protocol.
4. The concept of “live publications” is being developed. A “live publication” is a
scholarly work posted on the Internet (in the public domain) and regularly updated
by its author(s) [3].
5. Starting in 2014, several Elsevier-owned journals began publishing reviewers'
comments along with the main article. This approach, labeled “open review,” leads
to an improvement in the quality of articles. In addition, it allows to recognize the
contribution to the publishing process made by reviewers [4].
3 An Overview of the Possibilities of Multimedia Magazines
Since the early 2000s, electronic journals have been in active development. Electronic
journals publish articles in both PDF and HTML, which broadens publication options
95
– including for multimedia materials. Limiting ourselves to analyzing academic-type
journals, we can make the following observations:
─ Acceptance by the editor of article texts and related multimedia materials is done
via by e-mail.
─ Editors ask authors to specify the exact place in the text of the articles where it is
necessary to insert the appropriate multimedia material. Thus, authors do not di-
rectly participate in the laying out of their articles.
─ None of the analyzed journals publish a full range of multimedia files. The most
frequently published formats are image files in various formats (for example, JPG,
GIF, or PNG), video (for example, AVI, MPG, or MOV), animations (SWF, GIF),
audio in various formats, and presentations. There are certain journals that also
publish, in addition to the formats listed above, virtual reality files in VRML for-
mat (e.g. “Scientific Visualization”, http://sv-journal.org/);
─ Several video players are used to display video in articles; alternatively, videos are
uploaded to a cloud server and then embedded into text.
─ The journals’ web sites are either original in concept/design or are developed on
top of existing publishing platforms.
The widest range of displayed multimedia materials was found to be published by
the Scientific Visualization journal (http://sv-journal.org). However, even this journal
required additional software installed on the reader’s computer in order to view some
multimedia objects, such as 3D (http://sv-journal.org/example/index.html).
Of particular interest is the research conducted at the M.V. Keldysh Institute of
Applied Mathematics of the Russian Academy of Sciences and focusing on the
presentation of animation and video in scientific publications [5]. “Preprints of the
M.V. Keldysh IAM” featured articles accompanied by video annotations placed on
the publication’s metadata pages (e.g. http://library.keldysh.ru/preprint.asp?id=2013-
51). As part of this research, the issue of embedding video in articles in PDF format
was solved.
4 The Functionality of Multimedia Articles
A multimedia article published on the web and displayed in a browser can give the
reader opportunities that qualitatively distinguish it from an article in traditional print
format. It is in principle possible to publish video and sound fragments, 3D objects,
scenes with elements of interactivity, as well as interactive schemas and diagrams.
Even an image – seemingly the simplest media element of all – can take on new
qualities when published within a web document (HTML). Standard open-source
software modules allow you to magnify an image presented in thumbnail form, which
makes viewing comfortable. In some cases, illustrations have proportions which make
it impossible to represent them adequately in print. For example, the height of Japa-
nese Kakemono scrolls is 2 to 2.5 times their width, which limits publications to re-
producing only their fragments in a printed article. In the browser, however, this prob-
96
lem can be resolved using software modules that provide interactive viewing of frag-
ments of high-resolution images (Fig. 1).
Fig. 1. Interactive viewing of fragments of a Kakemono scroll.
Other traditional elements of a printed article can be reproduced at a new level of
quality. Large tables and diagrams, no longer limited by the size of the printed sheet,
can be displayed using horizontal scrolling. Footnote text can be displayed in a pop-
up block when the reader hovers over it with the mouse, without shifting the user’s
main focus of attention.
Giving readers the ability to play video, reproduce sound, view 3D scenes, etc, al-
lows the author to communicate his ideas at a whole new level. At the same time,
preparation of multimedia materials naturally requires additional efforts on the au-
thor’s part. The biggest challenge here consists in the development of a technical
editor’s toolkit. Even an editor without specialized qualifications and skills should
have the ability to compose and lay out articles using a simple and intuitive web inter-
face, through inserting text fragments and loading image files, videos, etc., into the
corresponding fields.
It is important to emphasize that media elements should be represented as separate
entities at the database level. This provides additional flexibility for their presentation
on the web page and ensures that the work of the technical editor is effective. Moreo-
ver, with this approach, not only the article as a whole, but also its individual media
elements can be provided with meta-information for potential aggregators.
97
It is obvious that such an “extended” publication can only be viewed in a browser;
with this in mind, it is the web version of an article that should be considered its pri-
mary version. However, an electronic publication can also provide a static (paper)
representation – one in which all dynamic objects are converted into images.
The goal of creating an information system supporting web publishing is the im-
plementation of an online framework that would allow editorial boards to publish
scientific journals incorporating multimedia content, generate extended meta descrip-
tions and exchange formats. The main feature of the information system under devel-
opment is the implementation of a web control panel allowing editors to typeset arti-
cles using a wide selection of "brick-like” field elements, which can be combined in
arbitrary order.
A working prototype of an electronic scholarly journal with multimedia content
based on the approaches described here was implemented by the authors for the “Cul-
ture and Technologies” journal (http://cat.ifmo.ru). The prototype is based on Drupal,
a freely distributed CMF [6]. The system’s implementation used Drupal’s administra-
tive interfaces, ready-made modules, and additional modules implemented using the
API.
5 The Structure of the Multimedia Articles
Traditional scholarly articles have an established design style: text and various static
objects (images, formulas, tables, graphs and other). Visual elements are inserted
during the writing process and are supplemented by captions. When designing a mul-
timedia journal, the team decided not to depart from design traditions and thus a deci-
sion was made to also embed multimedia objects in the proper positions of text.
It should be emphasized that for certain types of multimedia, merely the browser
with its tools is not sufficient. Video is shown using a player; to display 360-degree
video, an appropriate video player plug-in is required; to display images as a slider,
one needs to link the slider to a specific set of images; displaying images as a gallery
requires linking other images to galleries; and so on. Thus, when creating an online
article, not only are various processors (video and audio players, scripts for sliders
and galleries, etc) required, but so are the tools for linking these multimedia objects to
their processors. An additional complication stems from the fact that it is not known
in advance which multimedia materials will be used, how many they will be, and how
they will be ordered.
The a priori unknown article structure precludes the use of a fixed set of fields in
the editor’s interface, since in this case, each article would be laid out from a single
template with the same sequence of elements. To implement a more flexible ap-
proach, some mechanisms have been developed allowing the reuse of article elements
in any order and quantity. In addition, a handler can be attached to each field describ-
ing an object.
98
6 Multi-media Article Online Editing Tools
An essential aspect of the work of an electronic publishing house is the availability of
online tools for creating and editing multimedia articles by people who are not quali-
fied as web developers.
To represent a number of multimedia objects a JavaScript editor such as TinyMCE
or CKEditor may be sufficient. These editors allow to format text and insert images
directly in the browser, MSWord-style. In addition, one can insert multimedia ele-
ments – for example, video, elements loaded from external multimedia storage such
as YouTube, and other elements.
However, we would argue that the functionality of such editors does not suffice for
the following reasons:
─ Each multimedia object should be a separate entity at the level of fields. Aggrega-
tors can index not only the article as a whole, but also its individual media ele-
ments, which (optionally) have their own additional meta information. These ele-
ments can, for example, be selected upon request for presentation in external col-
lections while retaining their links to the original article [6]. When using a built-in
editor, however, the article remains a single “monolithic” entity.
─ Tools for uploading files via a web interface are very limited.
─ During the process of creating an article by a content manager, it is necessary to
connect video player plug-ins, JS scripts for sliders, galleries and other handlers,
which may require additional skills in web development.
Based on these factors, the requirements for editing tools for articles with multimedia
materials were formulated:
1. The article is divided into functional components: a fragment of text, a gallery of
images, video, 360-degree video;
2. A tool allowing to insert multimedia objects into the text of an article, attaching the
appropriate handlers, is needed.
3. It should be possible to insert any number of multimedia objects into an article.
Thus, all fields must be multiple.
4. Several options can be implemented for component-specific views (for example,
you can use a slider or gallery for images);
5. Large files should be downloaded via a web interface and overcome the restrictions
associated with web server settings.
The project is being implemented using the open-source distributed content manage-
ment system Drupal, which is a very powerful development environment with a well-
documented API. When implementing the electronic journal “Culture and Technolo-
gy”, Drupal version 7 was used. The main article editing tools were implemented
using Drupal tools allowing to work with fields. For each type of multimedia element,
collections of fields were implemented with the option of being used any number of
times within one article. To visualize the various types of multimedia in a browser,
the appropriate JavaScript modules (the slider, player, etc.) are attached to each of the
99
fields. The fixed article structure is implemented this way. The placement of multi-
media elements is carried out by the technical editor by inserting a named HTML
anchor inside the text. An anchor refers to a multimedia element that has the same
label as the anchor. When an article page is displayed in the browser, JavaScript code
moving multimedia objects along the DOM tree to the place of the corresponding
anchor (Fig. 2) is executed.
Fig. 2. An example of the implementation of the administrative and user interfaces of the dy-
namic element “Slider”
100
Since its inception, the Culture and Technology journal has been publishing articles
with the following multimedia content options: image gallery, image slider, video,
360-degree video.
As an example, several articles can be cited:
─ “Application of 360 ° Video Technology for the Presentation of the Solovetsky
Monastery Cultural Heritage,” which includes 7 fragments of 360-degree video.
http://cat.ifmo.ru/en/2016/v1-i1/88 [8]
─ “Multimedia reconstruction of a theatrical event. The premiere of the play “The
Seagull” on the Alexandrinsky Stage on October 17, 1896,” which includes sliders
of images of reconstructed scenes and scene models. The article presents more than
20 scenes from the play, demonstrating the project’s level of detail. Using this
many images in a traditional article would be difficult.
http://cat.ifmo.ru/en/2016/v1-i1/65 [9]
As part of the further development of the concept of multimedia electronic collec-
tions and, in particular, electronic multimedia journals, we plan to use a slightly dif-
ferent method related to the use of Paragraphs module of Drupal 8. This module is a
constructor of various fields, allowing to achieve the necessary flexibility by reusing
fields as “bricks” in multiple places of an article.
We plan to continue further development of the journal’s functionality in several
directions:
1. Adding elements of new types (for example, 3D objects) which can be included in
a multimedia article.
2. Implementation of data export to scientometric databases and scientific reposito-
ries: eLibrary, DOAJ.
3. Linking the journal with scientific aggregators using the OAI-PMH protocol and
Google Scholar meta tags.
4. Creation of an online tool allowing to use elements of other multimedia resources
in the article directly, thereby achieving “horizontal” integration between electron-
ic collections and providing access to information stored in external databases [10].
7 Metadata Editing Tools
Generally, scholarly journals use a simple set of the Dublin core, including OJS, as
metadata. Such a set of metadata is not enough to generate information about the pub-
lication in the Russian Scientific Citation Index (РИНЦ). “The metadata block of any
scholarly publication should include its bibliographic description (authors, title,
source (for example, journal), year of publication, volume, number, start and end
pages), authors’ summary (abstract, abstract) and keywords, as well as various addi-
tional information” [7]. When developing the IS of the journal “Culture and Technol-
ogies”, all necessary meta-description fields were implemented, and additional
metadata specific to Russia (UDC, LBC, SRSTI, etc.) was taken into account (Fig. 3).
101
Fig. 3. Block of metadata fields of an electronic journal
This work was supported by the Russian Foundation for Basic Research, project 19-
07-01012
References
1. DSpace – A Turnkey Institutional Repositoriy Application, http://www.dspace.org, last ac-
cessed 2019/10/29.
2. Fedotov, A.M., Bajdavletov, A.T., Zhizhimov, O.L., Sambetbaeva, M.A., Fedotova, O.A.:
Digital Repository of Scientific and Educational Information Sysnem. Vestnik. Novosi-
birsk State Univercity. Series: Information Technologies 13(3), 68–86 (2015).
3. Gorbunov-Posadov, M.M.: Zhivaya publikaciya. In: IGM im. Keldysha RAN, redakciya ot
2018.10.02, http://www.keldysh.ru/gorbunov/live.htm, last accessed 2019/10/29 (2011).
4. What is peer review? Elsevier, URL: https://www.elsevier.com/reviewers/what-is-peer-
review, last accessed 2019/10/29.
5. Gorbunov-Posadov, M.M., Roldugin, D.S., Slepenkov, M.I., Tuzov, I.V.: Animation and
video in scientific publication. In: KIAM Preprint № 104, Moscow, No. 104,
http://library.keldysh.ru/preprint.asp?id=2014-104, last accessed 2019/10/29 (2014).
6. Borisov, N.V., Zakharkina, V.V., Mbogo, I.A., Prokudin, D.E.: Design of software plat-
form full publishing cycle for the publication of online multimedia journal. International
102
Culture & Technology Studies 2(1), 21–28 (2017). http://cat.ifmo.ru/ru/2017/v2-i1/100,
last accessed 2019/10/29.
7. Gerasimov, A.N., Еlizarov, A.M., Lipachev, Е.K.: Subsystem of Formation Metadata for
Science Index Databases on Management Platform Electronic Scientific Journals. Russian
Digital Libraries Journal 18(1–2), 6–31 (2015). https://elbib.ru/en/article/366 , last ac-
cessed 2019/10/29.
8. Borisov, N. V., Volkov, O. G., Nikitina, L. L., Nikolaev, A. O., Smolin, A. A., Stolyarov,
D. A.: Application of Video 360° Technology for the Presentation of the Solovetsky
Monastery Cultural Heritage. International Culture & Technology Studies 1(1), 24–31
(2016). http://cat.ifmo.ru/en/2016/v1-i1/88, last accessed 2019/10/29.
9. Borisov, N. V., Nikitin, A. V., Smolin, A. A., Trushin, V. A., Chepurov, A. A., Chepurov,
O. A.: Multimedia-reconstruction of the theatrical event. The premiere of performance "the
Seagull" at Aleksandrinsky scene 17 Oct 1896. International Culture & Technology Stud-
ies 1(1), 15–23 (2016). http://cat.ifmo.ru/ru/2016/v1-i1/65, last accessed 2019/10/29
10. Borisov, N.V., Zakharkina, V.V., Mbogo, I.A., Prokudin, D.E., Shcherbakov, P.P: Prob-
lems of creating an online scientific journal with multimedia content. In: Internet and
Modern Society IMS-2019, pp. 153–165. Keldysh Institute of Applied Mathematics Rus-
sian Academy of Sciences, Moskow (2019). http://ims.ifmo.ru/file/pages/2/IMS-
2019_program_final.pdf, last accessed 2019/10/29.