=Paper=
{{Paper
|id=Vol-2485/paper7
|storemode=property
|title=Multi-windows Rendering Using Software OpenGL in Avionics Embedded Systems
|pdfUrl=https://ceur-ws.org/Vol-2485/paper7.pdf
|volume=Vol-2485
|authors=Boris Barladian,Lev Shapiro,Kurban Mallachiev,Alexey Khoroshilov,Yuriy Solodelov,Alexey Voloboy,Vladimir Galaktionov,Igor Koverninskiy
}}
==Multi-windows Rendering Using Software OpenGL in Avionics Embedded Systems==
https://ceur-ws.org/Vol-2485/paper7.pdf
Multi-windows Rendering Using Software OpenGL in Avionics
Embedded Systems
B.Kh. Barladian1, L.Z. Shapiro1, K.A. Mallachiev3, A.V. Khoroshilov3, Y.A. Solodelov2, A.G. Voloboy1,
V.A. Galaktionov1, I.V. Koverninskiy2
1
The Keldysh Institute of Applied Mathematics Russian Academy of Sciences, Moscow, Russia
2
FGUP «GosNIIAS» State Research Institute of Aviation Systems, Moscow, Russia
3
Ivannikov Institute for System Programming Russian Academy of Sciences, Moscow, Russia
Elaboration of modern airplane cockpit has tendency to use large displays instead of a lot of separate indicators. The large display
should combine information about flight navigation and state of plane equipment. Information coming from a wide variety of devices
should be displayed simultaneously. Therefore multi-windows rendering is vitally important here. Its implementation must be embedded
in real-time operating system which controls the aircraft. Development of a Safety Critical Compositor for multi-windows rendering for
OpenGL SC 1.0.1 software is considered in the paper. It works under the real-time operating system JetOS newly designed for aircraft.
Development is based on the use of extensions designed to work in multi-core systems in addition to standard JetOS partitioning services.
Keywords: aircraft cockpit display, multi-windowing, OpenGL SC 1.0, real-time operating system, embedded systems.
widescreen displays (so called multi-function displays). The
1. Introduction information coming from a wide variety of devices should be
displayed simultaneously. In particularly it may be airspeed,
Modern airborne systems are designed based on the
attitude indicator, altimeter, turn and slip indicator, vertical speed
architecture called Integrated Modular Avionics. A key feature
indicator and so on. At the same time such technical
of this architecture is the capability to execute several functional
characteristics as engine speed, oil pressure and fuel quantity
applications implementing the software part of avionics systems
should be displayed too. In addition, it is useful to visualize a
on the same computer. The necessary condition for this
map of the area, various pneumatic, hydraulic and electrical
capability is time and resource sharing between applications.
circuits, data from weather radars, various kinds of warnings, etc.
This mode of operation is offered by a real time operating system
This information is usually generated by independent subsystems
which is an integral part and most important component of the
and should not interfere with each other in accordance with the
airborne system.
requirements of ARINC653 [5].
In the development of modern airplane cockpits there is a
Nevertheless it is often necessary to display images from
tendency to use large displays to combine in it information about
several subsystems on one screen. Modern operating systems
the flight navigation and state of plane equipment. Fig. 1 and 2
solve this problem by supporting of a multi-window interface
show the cockpit aircraft evolution from the Sukhoi Superjet to
when each application’s content is rendered into its own window.
the MS-21.
A simplified approach is to allow each application to open a non-
overlapping window onto the display. While the last method
allows for faster drawing its implementation for safety critical
systems requires significant efforts. A compositor elaboration is
needed to support efficient multi-windowing.
Various approaches to implementation of the compositor for
safety critical systems are considered in [1]. One of compositor
implementations is the CoreAVI’s EGL_EXT_compositor
extension for EGL for OpenGL SC 1.0.1 and OpenGL SC 2.0
[2]. However its source codes are closed ones and it does not
allow to use them for our goal. The OpenGL SC library [3] we
Fig.1. Cockpit of the Sukhoi Superjet aircraft. are developing is designed to work under the JetOS operating
system [4]. This defines development specific and imposes
essential requirements to the developed code and algorithms. In
particular, the need of OS certification requires full access to the
source codes of both the OpenGL SC library and the compositor.
On the other hand, when developing the composer we can take
advantage of the specific opportunities of the JetOS to improve
performance. These features, in particular, include the ability to
use several processor cores.
2. Types of Composition
Fig.2. Cockpit of the MS-21 aircraft. In the paper [1] two main types of graphical composition are
considered – composition into the hardware level and
The number of displays in the cockpit of the MS-21 aircraft composition into a framebuffer.
has been reduced in comparison to the cabin of the Sukhoi While the benefits of the hardware level composition include
Superjet aircraft, but the displays have become much wider and good performance, power conservation and efficiency in dealing
allow displaying more information. with a large number of updates, this approach requires additional
The flight and equipment operation information is generated bandwidth to display all windows. It also requires specific
by numerous flight management systems. This information support for framebuffer driver which is not accessible for us in
should be displayed for pilots in the easy-to-read form. The currently used hardware.
generated information should be displayed graphically on the
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
The composition into the framebuffer approach combines
not_first = false;
elements from multiple applications and off-screen buffers into a
While (true)
single framebuffer. The framebuffer then renders the data to the
{
display. The composition into the framebuffer requires only one
If (not_first)
layer to display all buffers. In fact it is the only available
{
approach in our case. Data visualization scheme used is shown
Wait the message that image was displayed
in Fig. 3. Each application renders the data using OpenGL SC in
}
own off-screen buffer. These buffers then are passed to the
not_first = true;
compositor. It forms from them the single framebuffer layer and
Render image
visualizes it on display by using frame buffer library. The main
Sent message that image is ready for display
problem here is effective synchronization of independently
}
running applications and the compositor. The implementation of
synchronization depends on means provided by the operating
Fig. 4. Application synchronization algorithm.
system.
Appropriately the processing by the compositor images
Application1 Application2 Application3 rendered by applications can be represented by the following
2 2 3 algorithm (Fig. 5):
for (int ic = 0; ic < application number; ic++)
Buffer1 Buffer2 Buffer3 {
Wait not more than 1 millisecond the message that the
rendered image from ic-th application is ready to display
Compositor If the message came
{
Display image from the appropriate shared memory
Frame buffer library block by using frame buffer library
Send the message that image from the given application
was displayed
Display }
}
Fig. 5. Compositor synchronization algorithm.
Fig. 3. Compositor - Information flow to Display
Let’s consider an example of the compositor work. Resultant
3. Solution via JetOS partitions image generated from buffers produced by two applications is
presented in Fig. 6.
First implementation of compositor was based on using
standard JetOS tools developed according ARINC 653 standard
[5]. In this case several applications and compositor work in a
single processor. Each application and compositor work on their
own JetOS partitions. The JetOS provides memory and time
partitioning in accordance to the ARINC 653 requirements.
Partitions are scheduled on a fixed cyclic basis. To assist this
cyclic activation, the OS maintains a major time frame of fixed
duration, which is periodically repeated throughout the module’s
runtime operation. Partitions are activated by allocating one or
more “partition windows” within this major time frame. Each
“partition window” is defined by its offset from the start of the
major time frame and expected duration. The order of partition
activation is defined by the system integrator using configuration
tables. This provides a deterministic scheduling methodology
whereby the partitions are furnished with a predetermined
amount of time to access processor resources. A module may
contain several partitions running with different periods.
The rendered images are passed from the application to the
compositor by using special shared memory blocks. Each
application uses the own memory block for image rendering. Fig. 6. Composition of two application buffers.
This memory block has read-only access for the compositor.
Two applications – the Primary Flight Display (PFD) on the
Synchronization between applications and compositor is
left side of image and the Counter on its right side – work
provided by queuing messages transmitted between partitions via
simultaneously (in fact in line due to the requirements of ARINC
special communication channels. Two channels are used
653) and images produced by applications are visualized by the
between each application and the compositor. First channel is
compositor. Suggested approach works correctly but
used by application to inform the compositor that image is ready
visualization speed in given example is insufficient for avionic
to display. The second channel is used by the compositor to
applications. Both applications work with speed ~5 frames per
inform the application that the image was displayed and the
second. There are several reasons for such behavior. First at all
application can use the image buffer again. The application
the typical avionic processor PowerPC [6] has low performance.
synchronization scheme can be represented by the algorithm
The second reason is that all partitions work on a single processor
shown in Fig. 4.
core and have predetermined amount of time to access its
resources. We can slightly optimize this time subdivision only
taking into account real applications needs. In the given example these memory blocks as with events (Fig. 7). As an argument all
the frame time was subdivided in the following manner: these functions use a pointer either to the first half or to the
1. PFD – 45ms second half of the appropriate 16-byte shared memory block.
2. Counter – 15ms To pass rendered image from application to the compositor
3. Compositor – 16ms each application uses memory block shared with the compositor.
This schedule provides more or less balanced access to the Now the work of the application can be represented by the
processor resources for applications with essentially different following algorithm (Fig. 8):
resource requirements.
While (true)
But future acceleration is possible by using of all processor
{
cores only. Processor PowerPC (P3041) [6] used by us has four
AMP_WaitEvent(End_copy);
cores while perspective PowerPC (P4080) [7] can have eight
AMP_ResetEvent(End_copy);
cores.
Render image
AMP_SetEvent(Start_copy);
4. Multicore solution
}
In case of multi core system JetOS supports ability to run
multiple modules (instances of JetOS) on one device. These Fig. 8. AMP application algorithm.
modules operate independently on different processor cores. This
JetOS feature is called AMP – Asymmetric Multi-Processing. Appropriately the processing of images rendered by
This AMP feature makes possible to use processor power in applications by the compositor can be described by the following
significantly more efficient way. Appropriate project algorithm (Fig. 9):
configuration using AMP feature is called AMP project. AMP for (int ic = 0; ic < application number; ic++)
project supports shared memory blocks which we use for images {
passing between modules. But it does not provide currently If (AMP_GetEventState(Start_copy[ic]) = AMP_UP)
queuing messages which we used for synchronization in solution {
via JetOS partitions. Due to this reason we decided to use small AMP_ResetEvent (Start_copy[ic]);
shared memory blocks for synchronization in multicore solution. Display image from the appropriate shared memory
As in the case of previous solution we need to support two events block by using frame buffer library
for synchronization of interaction of each application with the AMP_SetEvent(End_copy[ic]);
compositor: }
1. Start_copy – when the image is prepared by application and }
is ready for display by the compositor; Fig. 9. AMP compositor algorithm.
2. End_copy – when the image was already displayed by the
compositor and the appropriate memory block can be used At the initialization state all End_copy events are set to
again by application for rendering the next frame. AMP_UP state while Start_copy are set to AMP_DOWN state.
The proposed technology of using multi-core processor
#define AMP_UP 1 under JetOS allows to increase the speed of visualization for the
#define AMP_DOWN 0 example on Fig. 6 till 8.8 frames per second for PFD application
typedef int* AMP_EVENT; and till 44 frames per second for Counter application. It should
/// Get event state. be noted that the rendering speed for application PFD is still
int AMP_GetEventState(AMP_EVENT ev) insufficient. Even with one application running in the JetOS
{ partition solution the rendering speed does not exceed 7.4 frames
return ev[0]; per second. When using multi-core technology the speed slightly
} increases due to the work of the frame buffer library in a separate
/// Set the event in the state "up". core. Partially it is due to the fact that the PFD application is
void AMP_SetEvent(AMP_EVENT ev) overcomplicated itself. Later this application is modified to
{ increase its efficiency.
ev[0] = AMP_UP;
}
/// Set the event in the state "down".
void AMP_ResetEvent(AMP_EVENT ev)
{
ev[0] = AMP_DOWN;
}
/// Infinitely wait while event is in state "down".
void AMP_WaitEvent(AMP_EVENT ev)
{
RETURN_CODE_TYPE ret;
while (ev[0] == AMP_DOWN)
{
TIMED_WAIT(MILLISECOND, &ret);
} Fig. 10a. Example of AMP solution for the compositor.
Visualization of PFD + map + Counters.
Fig. 7. Event emulation
This pair of events is implemented by using a 16-byte
memory block shared between modules. The first half of this
block is used for Start_copy event and the second half – for
End_copy event. For convenience and a more intuitive interface
we implemented the set of functions which emulates work with
performance the possibility of using multi-core processor options
directly in the OpenGL SC library should be considered.
6. References
[1] A Safety Critical Compositor for OpenGL SC 1.0.1 and
OpenGL SC 2.0.
http://www.coreavi.com/sites/default/files/compositor_whi
tepaper_final.pdf.
[2] EGL_EXT_compositor.
Fig. 10b. Example of AMP solution for the compositor. http://www.coreavi.com/sites/default/files/coreavi_produc
Visualization of PFD + relief. t_brief_-_egl_ext_compositor.pdf.
[3] B.Kh. Barladian, A.G. Voloboy, V.A. Galaktionov, V.V.
Knyaz’, I.V. Koverninskii, Yu.A. Solodelov, V.A. Frolov,
and L.Z. Shapiro, Efficient Implementation of OpenGL SC
for Avionics Embedded Systems. Programming and
Computer Software, 2018, Vol. 44, No. 4, pp. 207–212.
DOI: 10.1134/S0361768818040059.
[4] K.A. Mallachiev, N.V. Pakulin, A.V. Khoroshilov, Design
and architecture of real-time operating system. Proceedings
of the Institute for System Programming, vol. 28, issue 2,
2016, pp. 181-192.ISSN 2220-6426 (Online), DOI:
Fig. 10c. Example of AMP solution for the compositor. 10.15514/ISPRAS-2016-28(2)-12.
Visualization of PFD + state of doors. [5] ARINC Standards Store:
https://www.aviation-ia.com/product-categories.
[6] Universal Data Processor Module MUPD/P3041-VPX 3U,
http://www.nkbvs.ru/products/elektronnie-modyli/vpx-
3u/moduli-universalnogo-protsessora-dannix-mypd-
p3041/.
[7] QorIQ® Processors Based on Field Proven Power
Architecture Technology P-Series.
https://www.nxp.com/products/processors-and-
microcontrollers/power-architecture-processors/qoriq-
platforms/p-series:QORIQ-POWER-ARCHITECTURE-P-
Fig. 10d. Example of AMP solution for the compositor. SERIES.
Visualization of PFD + navigation display.
Additional examples of images produced by suggested multi
window approach are shown in Fig. 10. The following rendering
speed was reached for these examples:
Fig. 10a:
PFD – 16 frames per second
Map – 16 frames per second
Counter – 16 frames per second
Fig. 10b:
PFD – 16 frames per second
Relief – 9.2 frames per second
Fig. 10c:
PFD – 10.3 frames per second
State of doors – 21.7 frames per second
Fig. 10d:
PFD – 10.3 frames per second
Navigation display - 21 frames per second
5. Conclusion
Analysis of visualization algorithms for various data used in
embedded avionics systems shows that JetOS partitioning
services alone do not secure the required performance. The use
of extensions for work in multi-core systems provided by JetOS
improves the performance. However, the rendering speed is still
not sufficient in some cases. In order to further increase