The article discusses the format of electronic scholarly journals and publications focused on online presentation of research results. Issues of creating tools for electronic scholarly publishing are considered. Problems related to the need for including multimedia data are described. An approach to creating an online multimedia designer of the article is considered, online tools of the editor and author of the article are proposed. The organization of the component structure of the article on the basis of fields, which allows for effective presentation and citation, is proposed. With this approach, embedded media objects can be organized into self-sufficient collections. An additional condition for the formation of the article is an online layout tool based on a set of functional components. Consideration of multimedia components of the article as separate entities of the data structure gives flexible possibilities of their visual interactive representation. An additional advantage of the component approach is the ability to expand the range of types of multimedia objects. A working prototype of a multimedia scholarly publication is presented as the implementation of the described approach. The prototype of the journal serves as a starting point for the development of the component model and demonstrates the first stage of development.
Currently, the results of academic research are often presented in multimedia form. Electronic collections of multimedia documents are created when processing of the results of humanities research by archaeologists, folklorists, ethnographers, etc. Today, almost all scholarly journals have their electronic versions. Some magazines are fully electronic (i.e., web-based). This poses the problem of providing adequate technological capabilities for creating scholarly publications that integrate multimedia content.
A modern scholarly article can utilize a rich array of dynamic objects going beyond traditional illustrations: 3D models, video, 360-degree video, image galleries, interactive elements of virtual reality (VR) and others. 2
To ensure long-term preservation of information resources and exchange of
metainformation, a number of frameworks have been created. These are widely used in
research and education. Below are some of the examples of systems used for
electronic scholarly publications:
1. Support systems for digital repositories (institutional repositories) are widely used
to build open-access archives and electronic libraries allowing to create, store, and
distribute digital materials. These systems include open-source software platforms
such as DSpace, EPrints, GreenStone, Fedora, and others. We estimate that DSpace
is currently the most popular software for creating archives of electronic resources
(digital repositories) in an academic context. The DSpace platform was developed
jointly by Hewlett-Packard and the MIT (Massachusetts Institute of Technology)
Libraries [
Since the early 2000s, electronic journals have been in active development. Electronic journals publish articles in both PDF and HTML, which broadens publication options – including for multimedia materials. Limiting ourselves to analyzing academic-type journals, we can make the following observations: ─ Acceptance by the editor of article texts and related multimedia materials is done via by e-mail. ─ Editors ask authors to specify the exact place in the text of the articles where it is necessary to insert the appropriate multimedia material. Thus, authors do not directly participate in the laying out of their articles. ─ None of the analyzed journals publish a full range of multimedia files. The most frequently published formats are image files in various formats (for example, JPG, GIF, or PNG), video (for example, AVI, MPG, or MOV), animations (SWF, GIF), audio in various formats, and presentations. There are certain journals that also publish, in addition to the formats listed above, virtual reality files in VRML format (e.g. “Scientific Visualization”, http://sv-journal.org/); ─ Several video players are used to display video in articles; alternatively, videos are uploaded to a cloud server and then embedded into text. ─ The journals’ web sites are either original in concept/design or are developed on top of existing publishing platforms.
The widest range of displayed multimedia materials was found to be published by the Scientific Visualization journal (http://sv-journal.org). However, even this journal required additional software installed on the reader’s computer in order to view some multimedia objects, such as 3D (http://sv-journal.org/example/index.html).
Of particular interest is the research conducted at the M.V. Keldysh Institute of
Applied Mathematics of the Russian Academy of Sciences and focusing on the
presentation of animation and video in scientific publications [
A multimedia article published on the web and displayed in a browser can give the reader opportunities that qualitatively distinguish it from an article in traditional print format. It is in principle possible to publish video and sound fragments, 3D objects, scenes with elements of interactivity, as well as interactive schemas and diagrams.
Even an image – seemingly the simplest media element of all – can take on new qualities when published within a web document (HTML). Standard open-source software modules allow you to magnify an image presented in thumbnail form, which makes viewing comfortable. In some cases, illustrations have proportions which make it impossible to represent them adequately in print. For example, the height of Japanese Kakemono scrolls is 2 to 2.5 times their width, which limits publications to reproducing only their fragments in a printed article. In the browser, however, this problem can be resolved using software modules that provide interactive viewing of fragments of high-resolution images (Fig. 1).
Other traditional elements of a printed article can be reproduced at a new level of quality. Large tables and diagrams, no longer limited by the size of the printed sheet, can be displayed using horizontal scrolling. Footnote text can be displayed in a popup block when the reader hovers over it with the mouse, without shifting the user’s main focus of attention.
Giving readers the ability to play video, reproduce sound, view 3D scenes, etc, allows the author to communicate his ideas at a whole new level. At the same time, preparation of multimedia materials naturally requires additional efforts on the author’s part. The biggest challenge here consists in the development of a technical editor’s toolkit. Even an editor without specialized qualifications and skills should have the ability to compose and lay out articles using a simple and intuitive web interface, through inserting text fragments and loading image files, videos, etc., into the corresponding fields.
It is important to emphasize that media elements should be represented as separate entities at the database level. This provides additional flexibility for their presentation on the web page and ensures that the work of the technical editor is effective. Moreover, with this approach, not only the article as a whole, but also its individual media elements can be provided with meta-information for potential aggregators.
It is obvious that such an “extended” publication can only be viewed in a browser; with this in mind, it is the web version of an article that should be considered its primary version. However, an electronic publication can also provide a static (paper) representation – one in which all dynamic objects are converted into images.
The goal of creating an information system supporting web publishing is the implementation of an online framework that would allow editorial boards to publish scientific journals incorporating multimedia content, generate extended meta descriptions and exchange formats. The main feature of the information system under development is the implementation of a web control panel allowing editors to typeset articles using a wide selection of "brick-like” field elements, which can be combined in arbitrary order.
A working prototype of an electronic scholarly journal with multimedia content
based on the approaches described here was implemented by the authors for the
“Culture and Technologies” journal (http://cat.ifmo.ru). The prototype is based on Drupal,
a freely distributed CMF [
Traditional scholarly articles have an established design style: text and various static objects (images, formulas, tables, graphs and other). Visual elements are inserted during the writing process and are supplemented by captions. When designing a multimedia journal, the team decided not to depart from design traditions and thus a decision was made to also embed multimedia objects in the proper positions of text.
It should be emphasized that for certain types of multimedia, merely the browser with its tools is not sufficient. Video is shown using a player; to display 360-degree video, an appropriate video player plug-in is required; to display images as a slider, one needs to link the slider to a specific set of images; displaying images as a gallery requires linking other images to galleries; and so on. Thus, when creating an online article, not only are various processors (video and audio players, scripts for sliders and galleries, etc) required, but so are the tools for linking these multimedia objects to their processors. An additional complication stems from the fact that it is not known in advance which multimedia materials will be used, how many they will be, and how they will be ordered.
The a priori unknown article structure precludes the use of a fixed set of fields in the editor’s interface, since in this case, each article would be laid out from a single template with the same sequence of elements. To implement a more flexible approach, some mechanisms have been developed allowing the reuse of article elements in any order and quantity. In addition, a handler can be attached to each field describing an object.
An essential aspect of the work of an electronic publishing house is the availability of online tools for creating and editing multimedia articles by people who are not qualified as web developers.
To represent a number of multimedia objects a JavaScript editor such as TinyMCE or CKEditor may be sufficient. These editors allow to format text and insert images directly in the browser, MSWord-style. In addition, one can insert multimedia elements – for example, video, elements loaded from external multimedia storage such as YouTube, and other elements.
However, we would argue that the functionality of such editors does not suffice for
the following reasons:
─ Each multimedia object should be a separate entity at the level of fields.
Aggregators can index not only the article as a whole, but also its individual media
elements, which (optionally) have their own additional meta information. These
elements can, for example, be selected upon request for presentation in external
collections while retaining their links to the original article [
Based on these factors, the requirements for editing tools for articles with multimedia materials were formulated: 1. The article is divided into functional components: a fragment of text, a gallery of images, video, 360-degree video; 2. A tool allowing to insert multimedia objects into the text of an article, attaching the appropriate handlers, is needed. 3. It should be possible to insert any number of multimedia objects into an article.
Thus, all fields must be multiple. 4. Several options can be implemented for component-specific views (for example, you can use a slider or gallery for images); 5. Large files should be downloaded via a web interface and overcome the restrictions associated with web server settings.
The project is being implemented using the open-source distributed content management system Drupal, which is a very powerful development environment with a welldocumented API. When implementing the electronic journal “Culture and Technology”, Drupal version 7 was used. The main article editing tools were implemented using Drupal tools allowing to work with fields. For each type of multimedia element, collections of fields were implemented with the option of being used any number of times within one article. To visualize the various types of multimedia in a browser, the appropriate JavaScript modules (the slider, player, etc.) are attached to each of the fields. The fixed article structure is implemented this way. The placement of multimedia elements is carried out by the technical editor by inserting a named HTML anchor inside the text. An anchor refers to a multimedia element that has the same label as the anchor. When an article page is displayed in the browser, JavaScript code moving multimedia objects along the DOM tree to the place of the corresponding anchor (Fig. 2) is executed. Since its inception, the Culture and Technology journal has been publishing articles with the following multimedia content options: image gallery, image slider, video, 360-degree video.
As an example, several articles can be cited:
─ “Application of 360 ° Video Technology for the Presentation of the Solovetsky
Monastery Cultural Heritage,” which includes 7 fragments of 360-degree video.
http://cat.ifmo.ru/en/2016/v1-i1/88 [
As part of the further development of the concept of multimedia electronic collections and, in particular, electronic multimedia journals, we plan to use a slightly different method related to the use of Paragraphs module of Drupal 8. This module is a constructor of various fields, allowing to achieve the necessary flexibility by reusing fields as “bricks” in multiple places of an article.
We plan to continue further development of the journal’s functionality in several directions: 1. Adding elements of new types (for example, 3D objects) which can be included in a multimedia article. 2. Implementation of data export to scientometric databases and scientific repositories: eLibrary, DOAJ. 3. Linking the journal with scientific aggregators using the OAI-PMH protocol and
Google Scholar meta tags.
4. Creation of an online tool allowing to use elements of other multimedia resources
in the article directly, thereby achieving “horizontal” integration between
electronic collections and providing access to information stored in external databases [
Generally, scholarly journals use a simple set of the Dublin core, including OJS, as
metadata. Such a set of metadata is not enough to generate information about the
publication in the Russian Scientific Citation Index (РИНЦ). “The metadata block of any
scholarly publication should include its bibliographic description (authors, title,
source (for example, journal), year of publication, volume, number, start and end
pages), authors’ summary (abstract, abstract) and keywords, as well as various
additional information” [