The paper discusses details of the pilot display visualization that uses the hardware acceleration capabilities of the Vivante graphics processor in the JetOS aviation operating system. Previously the OpenGL Safety Critical library was implemented without hardware acceleration. This was done in such a way because software library is easier to certify in accordance with the avionics requirements. But usage of the software OpenGL does not provide acceptable visualization speed for modern Flight Display and 3D relief applications. So more complex visualization approach utilized the GPU acceleration capabilities was elaborated. Although the OpenGL library was implemented for a specific GPU and took into account its specificity, the described approach to adapt the MESA open source library can be used for other GPUs. An effective algorithm for multi-window visualization using the implemented library with hardware acceleration is present. The described approach allows you to achieve the visualization speed acceptable for the pilot display of the aircraft.
In [
In [
The goal of our research is to find a solution which provides visualization of a pilot
display in multiwindows mode with acceptable speed. And in the same time the
solution should possible to certificate for usage in civil aircrafts. The OpenGL software
solution mentioned above is close to this goal but not fully acceptable due to low
speed reason. This is why it was decided to investigate possibility to use the GPU
hardware acceleration. Because of certification requirement the MESA package [
MESA (also called Mesa3D and the Mesa 3D Graphics Library) is an open source
software implementation of OpenGL, Vulkan, and other graphics API. MESA
translates the API specifications into manufacturer specific graphics hardware drivers. It is
focused on providing high performance when working with 3D-graphics through the
hardware acceleration provided by GPUs. For a particular GPU an appropriate
specific driver is required. Some processor manufacturers, such as AMD or Intel, develop
drivers for the open-source MESA package for their processors by themselves. Other
manufacturers, such as Nvidia or Vivante, completely replace the entire MESA,
providing their own proprietary implementation of the graphics API (OpenGL
library). For such equipment the MESA development community creates alternative
drivers (for example, Nouveau for Nvidia or Etnaviv for Vivante). The
implementation of various versions of OpenGL (including OpenGL SC) based on commercial
drivers is considered in [
Therefore an open source driver is needed to use the hardware acceleration of a graphics processor in JetOS RTOS. Unfortunately a simple way – to use existent drivers from MESA – does not work because MESA is elaborated mainly for Linux and Windows. But the JetOS RTOS has a lot of own specifics. So in our case, when using the i.MX6 platform with the Vivante GPU, the only way is to investigate possibility to adapt MESA and Etnaviv driver codes. It should be noted that the system requires two libraries for working with graphics: the first one provides the OpenGL API and the second one directly displays the image built by the OpenGL library on the monitor screen (the frame buffer library). In this paper we consider only the implementation of the OpenGL.
So the implementation of the GPU hardware support for visualization in the JetOS operating system can be divided into two stages: 1. Adaptation of the MESA package to work under control of the JetOS RTOS; 2. Direct implementation of the OpenGL SC standard based on the adapted package.
High Speed Visualization in the JetOS Aviation Operating System Using Hardware… 3 2
The MESA package is currently adapted only for use under operating systems such as
Linux, Android and Windows [
We succeeded to solve to some extent all the problems indicated in points 1–5 and
to implement the hardware OpenGL (HWGL) library based on the MESA package.
All aviation applications from [
After MESA code adaptation the OpenGL Safety Critical should be implemented
to make our library acceptable for avionics software. This work should be carried out
separately for standard OpenGL SC versions 1.0.1 and 2.0.1. The OpenGL SC 1.0.1
standard [
The hardware OpenGL SC 1.0.1 library implemented on the basis of the MESA
package was tested on the same aviation applications that were used for elaboration of the
software library in [
It should be noted that the speed of 60 frames per second indicated in the table for some applications is restricted by the display vertical synchronization frequency. The speed of HWGL for these applications is higher. Thus, the use of our HWGL library in the JetOS operating system allows to achieve significant visualization acceleration due to the use of hardware acceleration of the Vivante GPU. The software implementation even using four cores is faster for one specific application only. 4
An important requirement for the aircraft's on-board system software is multi-window visualization capabilities when various information, generated by numerous independent flight control systems, is simultaneously displayed on one multi-function display. For the software implementation of OpenGL, a compositor was developed in
High Speed Visualization in the JetOS Aviation Operating System Using Hardware… 5
work [
Unfortunately both compositor approaches cannot be used for OpenGL with hardware acceleration. A typical aviation platform has only one GPU. We cannot use separate OpenGL instance in each partition or instance of JetOS. So to solve this problem we developed a new approach when OpenGL commands are executed in one special partition of the JetOS operating system.
The application using visualization via the OpenGL library for can be divided into two parts: 1. Preparation of data necessary for OpenGL operation: geometry parameters, various attributes, camera parameters, etc. 2. Direct execution of OpenGL functions with prepared data.
In our implementation these components are run in different partitions of the JetOS operating system. This allows us to separate data preparation and OpenGL execution which has to use the only GPU in case of hardware acceleration. Each application stores the indices of the OpenGL functions needed for visualization together with the corresponding parameters in a special array. It is located in the memory shared with the partition that performs real calls of the OpenGL functions (we will further call it the OpenGL partition). When all the calls necessary for generating the one frame image are prepared then the OpenGL partition is started. Only this partition uses the HWGL library and load GPU. Fig. 1 illustrates the implemented approach.
The code for each application is used almost unchanged. Minimal changes are necessary to provide necessary viewport position on the screen and to synchronize the operation of applications and the OpenGL partition. Encoding of calls to the OpenGL functions and writing the necessary data to the corresponding buffer is performed by the special OGLOUT library. This library replaces the real calls to OpenGL functions by writing the necessary data and indices of the corresponding OpenGL functions.
The encoding itself is relatively simple. If the function passes a fixed number of parameters, then the index of the called function is written to the current position of the array, followed by the values of the parameters. This is applicable to the most of OpenGL functions, such as glViewport(), glClearColor(), glClearDepthf(), glClearStencil(), etc. In more complex cases when number of parameters is not defined at the time of the call are passed (for example, glVertexPointer()) the encoding technology is somewhat complicated. During the call of such functions, instead of the pointer, the index of the reserved position in the output array is written. Then from this position values are written by this pointer in functions such as glDrawArrays() or glDrawElements().
Decoding the information recorded by applications is performed in the OpenGL partition using the OGLIN library sequentially for each array generated by the applications. The OGLIN library consists of one function process_ogl_input_array(). It sequentially reads data from the array written by the application, then according to the index of the OpenGL function it goes to the piece of code written for this function and extracts the data transferred from subsequent elements of the array including pointer values. After that it calls the specified OpenGL function with extracted parameters.
Disadvantage of this approach is that it is hard to implement the OpenGL functions
that return parameter values (such as, for example, glGenLists(), glGetError(),
glGetString(), glIsEnabled(), glGetPointerv(), glGetTexParameteriv()). However an
analysis of the currently used practical applications in onboard equipment has shown
that these functions are not used in them. Calls to these functions are also absent in
the OGLX library [
Synchronization of the applications and the OpenGL partition is done using special
objects called events [
The developed algorithm was tested using multi-window visualization of two pairs of practical applications: 1. PFD (Primary Flight Display) and DOORS (state of aircraft doors), demonstrated on Fig. 2; 2. PFD (Primary Flight Display) and ND (Navigation Display), demonstrated on Fig. 3.
High Speed Visualization in the JetOS Aviation Operating System Using Hardware… 7
Table 2 shows multi-window visualization speed (in frames per second) for the software OpenGL (SWGL) and the hardware OpenGL (HWGL). Visualization via the SWGL library has the possibility to run each application with own instance of OpenGL and compositor on own processor core. Visualization via the HWGL uses the algorithm described in previous chapter. Column SWGL 1 presents speed for use of 1 core, while SWGL 3 – 3 cores were used.
It should be noted that the visualization approach presented in this paper (HWGL + new multi-window algorithm) finally provides the speed which 2-3 times higher than visualization speed in case of software libraries using. Taking into account that the pilot display intended for visualization of information about flight and state of aircraft systems (but not for a movie demonstration) the speed in 15-20 frames per second can be considered as acceptable. 6
Studies have shown the possibility to use the GPU hardware acceleration in avionics. All study components – the JetOS real-time operating system, the OpenGL SC standard, the Vivante GPU – impose own limitations and particularities. Additionally, we always need to think about certification of on-board software according to the DO178C avionics standard. All these complicated the project much. But finally we succeeded to elaborate the approach and the algorithms which, on the one hand, provide acceptable visualization speed for the aircraft pilot display and, on the other hand, can be certified to be used in a real aviation operating system.
The technology of the hardware OpenGL SC library creation basing on the adaptation of the open-source MESA package can be applicable not only for the Vivante GPU but for other graphics hardware as well.