=Paper=
{{Paper
|id=Vol-2744/paper1
|storemode=property
|title=A Technology to Synthesize 360-Degree Video Based on Regular Dodecahedron in Virtual Environment Systems
|pdfUrl=https://ceur-ws.org/Vol-2744/paper1.pdf
|volume=Vol-2744
|authors=Petr Timokhin,Mikhail Mikhaylyuk,Klim Panteley
}}
==A Technology to Synthesize 360-Degree Video Based on Regular Dodecahedron in Virtual Environment Systems==
https://ceur-ws.org/Vol-2744/paper1.pdf
A Technology to Synthesize 360-Degree Video Based on
Regular Dodecahedron in Virtual Environment Systems*
Petr Timokhin 1[0000-0002-0718-1436], Mikhail Mikhaylyuk2[0000-0002-7793-080X],
and Klim Panteley3[0000-0001-9281-2396]
Federal State Institution «Scientific Research Institute for System Analysis of the Russian
Academy of Sciences», Moscow, Russia
1
webpismo@yahoo.de, 2 mix@niisi.ras.ru, 3 kpanteley@mail.ru
Abstract. The paper proposes a new technology of creating panoramic video
with a 360-degree view based on virtual environment projection on regular do-
decahedron. The key idea consists in constructing of inner dodecahedron surface
(observed by the viewer) composed of virtual environment snapshots obtained
by twelve identical virtual cameras. A method to calculate such cameras’ projec-
tion and orientation parameters based on “golden rectangles” geometry as well
as a method to calculate snapshots position around the observer ensuring synthe-
sis of continuous 360-panorama are developed. The technology and the methods
were implemented in software complex and tested on the task of virtual observing
the Earth from space. The research findings can be applied in virtual environment
systems, video simulators, scientific visualization, virtual laboratories, etc.
Keywords: Visualization, Virtual Environment, Regular Dodecahedron, Video
360, Panoramic Mapping, GPU.
1 Introduction
In modern human activities the importance of the visualization of virtual prototypes of
real objects and phenomena is increasingly grown. In particular, it’s especially required
for training specialists in space and aviation industries, medicine, nuclear industry and
other areas, where the mistake cost is extremely high [1-3]. The effectiveness of work-
ing with virtual prototypes is significantly increased if an observer experiences a feeling
of being present in virtual environment. It is important that observer will have a possi-
bility not only to sense such visual experience but also to reproduce it. For instance,
this possibility allows to self-analyze the correctness of actions in virtual environment
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
*
The publication is made within the state task on carrying out basic scientific re-
searches (GP 14) on topic (project) “34.9. Virtual environment systems: technolo-
gies, methods and algorithms of mathematical modeling and visualization” (0065-
2019-0012).
2 P. Timokhin, M. Mikhaylyuk, K. Panteley
or demonstrate results to expert community. To implement such possibility, new tech-
nologies and formats of video recording the virtual environment are actively researched
and developed. The examples are the technology of creating stereo presentations using
autostereoscopic monitors [4] and the technology for deferred synthesis of 4K stereo
clips [5]. In recent years, a technology of 360-degree video [6] has been developed
rapidly, which point is omnidirectional capture of the environment (with covering im-
aginary sphere surrounding the observer). When playing such a video, the observer can
change view direction and sense an effect of immersion into visualized environment.
Two major approaches to create 360-video can be highlighted. At the first approach
spherical panorama is unfolded onto a plane directly using some cylindrical carto-
graphic projection (equirectangular, Lambert equal-area) [7, 8]. At the second approach
panorama is projected on the faces of some convex polyhedron (frustum, cube, rhombic
dodecahedron) [9] which is unfolded onto a plane. Thanks to the equivalency of cube
faces (there are no poles with expressed distortions as at the first approach) cubemap
projection has been taken widely into use. However, projection detail degree in cube
face centers is almost twice as smaller as near the edges, because of poor sphere ap-
proximation (just by 6 faces, 90 degrees each). To avoid blurring the panorama, the
total cubemap resolution should be increased. That’s leading to increase of the redun-
dancy of video stream bitrate preventing playback smoothness.
Desire to reduce projection non-uniformity preserving its universality inspired us to
use regular dodecahedron for virtual environment footage. Compared to the cube the
dodecahedron has twice as much faces, therefore, its form is closer to a sphere. The
dodecahedron is also the only Platonic solid in which all plane angles between edges at
a vertex are obtuse (108 degrees). Owing to this the dodecahedron has more balanced
(and less visible) projection non-uniformity than the Platonic solids with triangle faces
(tetrahedron, octahedron, and icosahedron). The problem is that in virtual world (as in
real) camera can make rectangular snapshots, and dodecahedron faces are regular pen-
tagons as we know. Therefore, it poses a challenge of constructing virtual environment
projection onto regular dodecahedron ensuring the visualization of continuous 360-pan-
orama. In the paper an effective technology to solve this task is proposed, which is
based on original scheme to locate virtual cameras and virtual environment snapshots
obtained by them. The solution is implemented using the C++ language and OpenGL
graphics library.
2 Constructing the projection onto regular dodecahedron
Consider the task of creating 360-video for the observer being movable in virtual envi-
ronment. For each frame of the 360-video we will project virtual environment onto the
imaginary dodecahedron surrounding the observer (see Fig. 1). In order to construct
such projection we place twelve identical virtual cameras (С0,…,С11) with symmetrical
frustums in observer position PV, direct them into the faces (pentagons) centers of the
imaginary dodecahedron and make snapshots of virtual environment. Herewith, the ori-
entation of the imaginary dodecahedron is tightly bound to observer’s view vector v
and «up» vector u, as shown in Figure 1.
A Technology to Synthesize 360-Degree Video Based on Regular Dodecahedron… 3
15
2
16
19
3
17 u 5
18 1
v
14
6
4
8
0
11 12
7
13
9
10
Fig. 1. Imaginary dodecahedron surrounding the observer
The key point of implementation of this scheme is calculating Сi camera’s projection
and orientation parameters providing the dodecahedron face (pentagon) to be inscribed
into camera’s rectangular viewport, namely: 1) calculating vertical FOV (field of view)
angle and viewport aspect (the ratio of its width to its height), and 2) calculating the
coordinates of view vi and «up» ui vectors of camera Сi in World Coordinate System
(WCS). Let’s consider these stages in more detail.
2.1 Calculation of the aspect and vertical FOV
As noted above, virtual environment projection onto a pentagon of imaginary do-
decahedron is done by individual camera Ci. The camera has symmetrical frustum ver-
tically and horizontally. This is necessary in order to scale increase (resulting from per-
spective projection) would be similar at all directions from pentagon center to its sides
(see Fig. 2), and snapshots captured by adjacent cameras would be docked to each other
in the same scale. For this reason, a camera with oblique frustum can’t be used here,
since in that case, the lower half of pentagon will be of larger scale than the upper one.
Figure 2 shows, that camera’s Ci viewport sizes must be proportionate to the sizes
of the rectangle ABCD. Its center O coincides the pentagon MUSTQ center, AB = CD
= 2Rpenta, BC = AD = d, and AD is distanced from MQ by h = Rpenta - rpenta, where Rpenta,
rpenta are radii of circumscribed and inscribed circle of the pentagon, and d is pentagon
diagonal [10]:
Rpenta = ( MQ 2 ) cosec ( 5) , rpenta = ( MQ 2 ) ctg ( 5) , d = 2MQ sin ( 3 10 ) . (1)
4 P. Timokhin, M. Mikhaylyuk, K. Panteley
B S C
Rpenta
U d/2
T
O
PV rpenta
M Q
h
A D
Fig. 2. Placing the pentagon into the viewport of the camera Сi
Then aspect value of camera’s Ci viewport can be expressed as
BC d 2sin ( 3 10 )
aspect = = = 0.951 . (2)
AB 2Rpenta cosec ( 5)
To find camera’s Ci angle , consider the right triangle PV OS . Its side PVO is the
radius of inscribed sphere of the dodecahedron [10], and is calculated as
PV O = ( MQ 2 ) ctg ( 5) tg ( 2 ) , (3)
where = 2arcsin ( cos ( 3) sin ( 5) ) 116.565 is dihedral angle of regular dodec-
ahedron. From triangle PV OS the expression for tg ( 2 ) can be written:
Rpenta cosec ( 5) 1
tg ( 2 ) = = = = sec ( 5) ctg ( 2 ) , (4)
PV O ctg ( 5) tg ( 2 ) cos ( 5) tg ( 2 )
from where we get the expression for the required angle :
= 2arctg ( sec ( 5) ctg ( 2) ) 74,8 . (5)
Obtained aspect and values are set for each Ci camera using gluPerspective oper-
ator of OpenGL library.
A Technology to Synthesize 360-Degree Video Based on Regular Dodecahedron… 5
2.2 Calculating the coordinates of view and «up» vectors of camera Сi
After we have set projection parameters (common to all cameras С0,…,С11 ) calcu-
lating the coordinates of vi view and ui «up» vectors for each Сi camera is required.
These vectors direct the camera into its face of imaginary dodecahedron, as shown on Fig. 2.
At first, the correspondence between cameras С0,…,С11 and dodecahedron faces is
set. We enumerate the dodecahedron vertices as shown in Figure 1 and write the se-
quence H of pentagons through them: 0 - {4, 7, 10, 13, 0}, 1 - {0, 1, 2, 3, 4}, 2 - {4, 3,
5, 6, 7}, 3 - {7, 6, 8, 9, 10}, 4 - {10, 9, 11, 12, 13}, 5 - {13, 12, 14, 1, 0}, 6 - {12, 11,
18, 19, 14}, 7 - {1, 14, 19, 15, 2}, 8 - {3, 2, 15, 16, 5}, 9 - {6, 5, 16, 17, 8}, 10 - {9, 8,
17, 18, 11}, 11 - {18, 17, 16, 15, 19}. In order to calculate vi and ui vectors we'll need
the coordinates of dodecahedron faces centers. For this, the calculation of dodecahe-
dron vertices coordinates in WCS system will be discussed in more depth.
In this work, dodecahedron vertices coordinates calculating is done using “golden
rectangles” [11] – three mutually perpendicular rectangles which size related to
number (see Fig. 3). The number is “golden ratio” of two values, in which the greater
number relates to the smaller, as well as their sum relates to the greater of them [12]:
( )
= 1 + 5 2 = 2sin(3 10) = 2 cos( 5) . (6)
YOCS
15
16
2 5
19
17
14 6
18
OOCS
ZOCS 0 XOCS
8
11
9 7
12
10
2
Fig. 3. The relation between dodecahedron vertices and “golden rectangles”
6 P. Timokhin, M. Mikhaylyuk, K. Panteley
Define the 8 dodecahedron vertices to be vertices of inscribed cube of size 2, and
remaining 12 - as “golden rectangles” vertices. Denote by P0,…,P19 the vertices coor-
dinates in dodecahedron’s Object Coordinate System (OCS) OocsXocsYocsZocs. The origin
Oocs of the coordinate system is located in the dodecahedron center, the axis OocsXocs
passes through the middle of dodecahedron edge {6, 8}, the axis OocsYocs - through the
middle of the edge {15, 16}, and the axis OocsZocs - the middle of the edge {11, 18}.
Based on Figure 3 we draw up the Table 1 of coordinates P0,…,P19.
Table 1. The coordinates of the dodecahedron vertices in OCS system
Coordinates Values
P0 , P2 , P5 , P7 ( −1 −1 −1) ( −1 1 −1) (1 1 −1) (1 −1 −1)
P9 , P12 , P17 , P19 (1 −1 1) ( −1 −1 1) (1 1 1) ( −1 1 1)
P1 , P6 , P8 , P14 ( − 0 −1 ) ( 0 −1 ) ( 0 1 ) ( − 0 1 )
P3 , P4 , P11 , P18 ( 0 1 − ) ( 0 −1 − ) ( 0 −1 ) (0 1 )
P10 , P13 , P15 , P16 (1 − 0) ( −1 − 0) ( −1 0) (1 0)
Denote by {vi,0, vi,1, vi,2, vi,3, vi,4} vertices numbers of some ith pentagon from H
sequence (see the Section 2.2 beginning). The center coordinates Oi of the ith pentagon
are calculated as average of its vertices coordinates taken from the Table 1:
1
Oi = Pj .
5 jvi ,0 ,...,vi ,4
(7)
From the expression (7) the coordinates of view vector vocs,i and «up» vector uocs,i of
camera Сi in OCS system are easy obtained:
vocs ,i = Oi , uocs ,i = Pvi ,2 − Oi , (8)
and desired coordinates of vi and ui vectors in WCS system can be calculated as
vi = M v−1vocs ,i , ui = M v−1uocs ,i , (9)
where M v is the upper-left 3x3 submatrix of the matrix V transforming coordinates
from WCS system to left-handed view coordinate system (View Coordinate System,
VCS) [13]. In our work VCS system is tightly bound to OCS system of the dodecahe-
dron: the origin Ovcs coincides the origin Oocs and dodecahedron center, the axis OvcsYvcs
coincides in direction with the axis OocsYocs and the vector u, the axis OvcsZvcs - with the
axis -OocsZocs and the vector v, and the axis OvcsXvcs – with the axis OocsXocs.
We set calculated vi and ui vectors for each Ci camera using OpenGL library operator
gluLookAt.
A Technology to Synthesize 360-Degree Video Based on Regular Dodecahedron… 7
3 Reconstructing the panorama from dodecahedron snapshots
Consider the task of playback 360-video based on the projection onto regular dodeca-
hedron. Every frame of such video contains 12 snapshots of virtual environment, cor-
responding to twelve dodecahedron faces. In order to reconstruct 360-panorama, we
will create separate virtual scene, place an observer in the center, and arrange snapshots
(textured rectangles) around the observer, as demonstrated in Figure 4.
Fig. 4. The principle of docking dodecahedron snapshots
Snapshots’ rectangles are tightly bound with our virtual scene WCS system in the
same way, as OCS system is bound with dodecahedron on Figure 3, i.e. axis Owcs X wcs
passes through the middle of the edge formed by 3 and 9 snapshots cross-section (num-
bers correspond to pentagons numbers), and etc.
Consider calculating the coordinates of the ith rectangle vertices in WCS system.
Denote by Ai , Bi , C i , Di its vertices coordinates and by M i , U i , S i , Ti , Qi , Oi -
vertices and center coordinates of the pentagon placed in AiBiCiDi (see Fig. 5). As seen
from the Figure 5, we don’t use the addition h (introduced in Section 2.1) at visualiza-
tion, since h is required only to obtain correct perspective projection while taking a
snapshot. Note, that vertices and center coordinates of the pentagon are also calculated
according to Table 1 and equation (7).
Let’s introduce unit vectors e1 = MiQi MiQ and e2 = OiSi OiSi . Then for de-
sired vertices coordinates of AiBiCiDi the following expressions can be written:
d d
Bi = Si − e1 , Ci = Si + e1 , Ai = Bi − ( Rpenta + rpenta
)e2 , Di = Ci − ( Rpenta + rpenta
)e2 , (10)
2 2
8 P. Timokhin, M. Mikhaylyuk, K. Panteley
are the diagonal and the radiuses of circumscribed and in-
where d , Rpenta and rpenta
scribed circle of the pentagon, calculated similarly to d , Rpenta and rpenta from (1).
Fig. 5. Calculating the coordinates of snapshot vertices
To reconstruct the 360-panorama, we calculate vertices coordinates for all the twelve
rectangles using formulas from (10) and texture them with snapshots. Figure 6 shows
linking dodecahedron faces to the textures-snapshots of the virtual environment, written
in a frame of 360-video (see Fig. 7).
8 9 10 11
4 5 6 7
0 1 2 3
Fig. 6. The arrangement of dodecahedron face textures in a frame of 360-video
A Technology to Synthesize 360-Degree Video Based on Regular Dodecahedron… 9
4 Results
Developed technology was implemented in software complex for 360-video synthesis
based on virtual environment projection onto regular dodecahedron. The complex is
written in C++ using graphics library OpenGL and a set of FFmpeg libraries for coding/
decoding digital videos. The proposed solution includes a module for capturing 360-
frames, to be embedded in visualization system, and a player for created 360-video.
The solution was tested in visualization system of virtual experiment for Earth ob-
servation from the International space station (ISS) [14]. Using the capture module, a
video of the flight along the section of the ISS 3rd day orbit track was created. The
flight passed through Balkhash lake (from 73 to 78 degrees eastern longitude). Video
capturing was carried out at the frequency of 25 frames/s based on video compression
standard H.264 and MP4 container. Dodecahedron snapshots had sizes of 896x848 pix-
els, and 360-video frames – size of 3584х2544 pixels (are multiples of 16 according to
H.264). A frame example from the synthesized 360-video is demonstrated in Figure 7.
During a 360-video playback viewer can “turn their head” up/down and right/left and
zoom in/out the image. The Figure 8 illustrates the visualization of virtual Earth pano-
rama reconstructed at several viewing directions from 360-frame shown in Figure 7.
Fig. 7. A frame of synthesized 360-video based on regular dodecahedron
We also compared the quality of the projection of the virtual environment onto a
regular dodecahedron and onto a cube. We evaluated changing the number of pixels
that could be occupied by the projection of a test segment onto face texture for the
dodecahedron and the cube. The segment occupied the largest number pmax of pixels in
the region of face vertex, and the smallest pmin - in face center. In the case of dodecahe-
dron, the pmin/pmax ratio was about 0.6, while for a cube it was only about 0.3 (ideally,
10 P. Timokhin, M. Mikhaylyuk, K. Panteley
for a sphere, this ratio is 1). Thus, the use of a projection onto a dodecahedron can
reduce the loss of detaility by up to 2 times compared to cubemap projection. Mean-
while, video stream bitrate depends on, how efficiently dodecahedron faces are ar-
ranged in the frame. While experimenting with software complex a packing scheme for
dodecahedron faces was developed. This packing allows the number of unused frame
parts to be extremely reduced (almost twice) without any quality loss. Implementation
of such packing algorithm is planned in extended research of developed technology.
a)
b)
Fig. 8. Virtual Earth panorama at viewing directions: a) forward; b) nadir
5 Conclusion
This work relates to the direction of research and development of new technologies and
methods for the synthesis of 360-video in virtual environment systems, based on the
projection of the environment on the faces of convex polyhedra. Despite the great pop-
ularity of the cubemap projection, it has several disadvantages that affect the quality
and redundancy of the video stream bitrate. In this paper, we propose a new technology
based on the projection of virtual environment on the faces of a regular dodecahedron.
The technology includes projection construction stage resulting in 12 rectangular snap-
shots of the virtual environment, and the stage of 360-panorama reconstruction from
these images. To implement the first stage, methods to calculate the projection and the
A Technology to Synthesize 360-Degree Video Based on Regular Dodecahedron… 11
orientation parameters of virtual cameras are proposed, by which 12 snapshots are syn-
thesized. To implement the second stage, an original scheme of snapshots arrangement
(docking) around the observer is proposed, which provides synthesis of continuous 360-
panorama, as well as a method to calculate the positions of such snapshots. The devel-
oped technology and methods were implemented in the software complex and tested in
the visualization system of virtual experiments to observe the Earth from the ISS. Test-
ing confirmed the adequacy of the results obtained to the task, and their applicability in
the areas of virtual environment systems, video simulators, virtual laboratories, etc.
References
1. Zhdanov, D.D., Galaktionov, V.A., Voloboy A.G., Zhdanov A.D., Garbul’ A.A., Potemin
I.S., Sokolov V.G.: Photorealistic Rendering of Images Formed by Augmented Reality Op-
tical Systems. Programming and Computer Software 44(4), 213–224 (2018).
2. Barladian, B.K., Shapiro, L.Z., Mallachiev, K.A., Khoroshilov, A.V., Solodelov, Yu.A.,
Voloboy, A.G., Galaktionov, V.A., Koverninskii, I.V.: Visualization Component for the
Aircraft Real-Time Operating System JetOS. Programming and Computer Software 46(3),
167–175 (2020).
3. Mikhaylyuk, M.V., Torgashev, M.A.: The System of Visualization “GLView” for Simula-
tors and Virtual Environment Systems. In: 25th International Conference on Computer
Graphics and Vision (GraphiCon 2015), pp. 96–101. Protvino, Russia (2015).
4. Andreev, S.V., Bondarev, A.E., Galaktionov, V.A., Bondareva, N.A.: The problems of stereo
animations construction on modern stereo devices. Scientific Visualization 10(4), 40–52 (2018).
5. Timokhin, P.Yu., Mikhaylyuk, M.V., Vozhegov, E.M., Panteley, K.D.: Technology and
methods for deferred synthesis of 4K stereo clips for complex dynamic virtual scenes. Trudy
ISP RAN/Proc. ISP RAS 31(4), 61–72 (2019).
6. Corbillon, X., Simon, G., Devlic, A., Chakareski, J.: Viewport-Adaptive Navigable 360-
Degree Video Delivery. In: 2017 IEEE International Conference on Communications (ICC),
pp. 1–7. IEEE, Paris (2017).
7. Jiang, H., He, G., Yu, W., Wang, Z., Li, Y.: Scalable Video Coding Based on the User’s View
for Real-Time Virtual Reality Applications. In: Zeng, B. et al. (eds.) Advances in Multimedia
Information Processing – PCM 2017, LNCS 10736, pp. 766–775. Springer (2018).
8. Yu, M., Lakshman, H., Girod, B.: A Framework to Evaluate Omnidirectional Video Coding
Schemes. In: 2015 IEEE International Symposium on Mixed and Augmented Reality, pp. 31–
36. IEEE, Fukuoka (2015).
9. Fu, C.-W., Wan, L., Wong, T.-T., Leung, C.-S.: The Rhombic Dodecahedron Map: An Ef-
ficient Scheme for Encoding Panoramic Video. IEEE Transactions on Multimedia 11(4),
634–644 (2009).
10. Zwillinger, D.: CRC Standard Mathematical Tables and Formulas. 33 rd edition. CRC Press,
Boca Raton (2018).
11. Vasilieva, V.N.: Golden Section and Golden Rectangles When Building Icosahedron, Dodec-
ahedron and Archimedean Solids Based On Them. Geometry & Graphics 7(2), 47–55 (2019).
12. Akhtaruzzaman M., Shafie A.A.: Geometrical Substantiation of Phi, the Golden Ratio and the
Baroque of Nature, Architecture, Design and Engineering. International Journal of Arts 1(1),
1–22 (2011).
13. Hill F.S. Computer Graphics Using OpenGL. Second Edition. Prentice Hall (2001).
14. Mikhaylyuk, M.V., Timokhin, P.Y., Maltsev, A.V.: A method of Earth terrain tessellation on
the GPU for space simulators. Programming and Computer Software 43(4), 243–249 (2017).