=Paper=
{{Paper
|id=Vol-2485/paper3
|storemode=property
|title=Examination of the Nvidia RTX
|pdfUrl=https://ceur-ws.org/Vol-2485/paper3.pdf
|volume=Vol-2485
|authors=Vadim Sanzharov,Alexey Gorbonosov,Vladimir Frolov,Alexey Voloboy
}}
==Examination of the Nvidia RTX==
https://ceur-ws.org/Vol-2485/paper3.pdf
Examination of the Nvidia RTX
V.V. Sanzharov1 , A.I. Gorbonosov3 , V.A. Frolov2,3 , A. G. Voloboy2
vsan@protonmail.com|alexey.gorbonosov@graphics.cs.msu.ru|vova@frolov.pp.ru|voloboy@gin.keldysh.ru
1
Gubkin Russian State University of Oil and Gas, Moscow, Russia;
2
Keldysh Institute of Applied Mathematics RAS, Moscow, Russia;
3
Moscow State University, Moscow, Russia
Hardware acceleration of ray tracing is an active research field, but only with the release of Nvidia Turing
architecture GPUs it became widely available. Nvidia RTX is a proprietary hardware ray tracing acceleration
technology available in Vulkan and DirectX APIs as well as through Nvidia OptiX. Since the implementation
details are unknown to the public, there are a lot of questions about what it actually does under the hood. To
find answers to these questions, we implemented classic path tracing algorithm using RTX via both DirectX and
Vulkan and conducted several experiments with it to investigate the inner workings of this technology. We tested
actual hardware implementation of RTX technology on RTX2070 GPU and the software fallback in the driver on
GTX1070 GPU. In this paper we present results of these experiments and speculate on the internal architecture of
RTX.
Keywords: photo-realistic rendering, ray tracing, hardware acceleration, GPU
1. Introduction ray tracing algorithm - scene and camera data were
uploaded from the host and the chip produced the ren-
Ray tracing is a cornerstone of photo-realistic
dered image. Like the ray casting solutions, SaarCOR
image synthesis. Since first papers on ray tracing
used packet tracing (in groups of 64 rays). The archi-
[19], [5], computer graphics researchers developed a
tecture was fully pipelined to further mitigate memory
plethora of different techniques to somehow acceler-
access latency - simultaneously traversing one group of
ate the computations associated with ray tracing.
rays, loading data for the next group and intersec-tion
The hardware acceleration ray tracing had limited operation performed on another group of rays. An
success out of research papers. Until the RTX tech- example of ray tracing hardware which was com-
nology by Nvidia was released in their Turing archi- mercially available is ART AR250/350 rendering pro-
tecture GPUs. It was stated that Turing hardware cessor with a custom RISC processor core [4]. The
contains special so-called «RT cores» which acceler- solution was used to accelerate offline rendering and
ate ray tracing. In the official Turing architecture was packaged as x86 PC with 16, 36 or 48 render-
whitepaper [22] it is stated that RT core contains two ing processors as PCI-X cards and gigabit networking
units which perform bounding box and ray-triangle system. Software side included RenderMan compli-
intersection tests. But since RTX is closed source, ant renderer and network communication interfaces
we don’t know for sure how exactly it is implemented and plugins for 3D applications (CATIA, 3ds Max,
and if this is all that is to ray tracing acceleration in Maya). Details about the custom rendering processor to
Turing GPUs. In this paper, we present information on our knowledge were never published.
several experiments we did with an RTX GPU. We
All works mentioned to this point concern fixed
analyze the experiments’ results and speculate on pos-
function hardware. One of the first solutions with
sible techniques used in RTX hardware to accelerate
programmable stages is RPU (ray processing unit)
ray tracing. But first of all, let’s review the research in
[20]. The traversal and primitive intersection tasks
ray tracing acceleration hardware to understand what
are implemented in fixed function units. RPU sup-
techniques were already tried out in hardware imple-
ported custom shaders with features such as recur-
mentations and how well did they perform.
sive function calls, trace instruction to initiate tracing of
an arbitrary ray, asynchronous load instruction to hide
1.1 Related work in ray tracing acceleration memory latency. RPU also featured geometry shaders,
hardware instancing support and shader tables to look up specific
shader to execute for a particular geome-try object.
First dedicated hardware solutions closely related to As SaarCOR and ray casting solutions, RPU also uses
ray tracing were PCI cards for volume data visu-alization packet ray-tracing which can result in performance
which implemented ray casting and Phong shading (such drops in the case of incoherent rays. The TRaX
as [9, 12]). Even though these hard-ware traced only architecture [16] implements a different solution -
primary rays, it already implemented techniques to many identical cores consisting of simple thread
increase the efficiency of parallel tracing such as grouping processors. It can be viewed as general pur-pose
rays to make use of memory access coherence [9]. architecture and is used in other papers to simu-late their
Another notable product was SaarCOR architecture [13] hardware [7]. In the ray-tracing application TRaX
and its updated version in an FPGA chip [14]. The accelerates single ray performance and features MIMD
SaarCOR chip implemented the whole execution model as opposed to groups of 4 or
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
more rays and SIMD model in previously mentioned reach wide public. But since the implementation de-
architectures. The authors in [10] aimed to address tails are closed (like [4, 21]), it is unclear how exactly
problems with incoherent rays by using N-wide SIMD does it work and what acceleration techniques it uses. In
processing architecture with filtering of rays to find this paper, we aim to understand the principles be-hind
coherent groups. The filtering is applied at traver-sal, ray tracing acceleration in Nvidia RTX hardware by
intersection and shading stages of the ray tracing measuring the performance in several scenarios us-ing
algorithm. Vulkan and DirectX12 API.
In [1] authors simulate architecture close to that of
Nvidia Fermi GPU. One of the key aspects of it 2. Experimental analysis of Nvidia RTX
(related to ray tracing) is work compaction. When a First let’s briefly review available information
warp (group of 32 threads) has more than a half of rays about inner workings of RTX. Access to RTX ray
terminated, it terminates and the non-terminated rays are tracing functionality is available through Vulkan API,
copied to the next warp. This mechanism allows to Microsoft DirectX 12 (DXR) and Nvidia OptiX API
mitigate the effect of incoherent rays and preserve the libraries [23]. We used both Vulkan and DirectX 12
parallelism. Another suggestion in this work is related for our experiments.
to stack memory layout for threads. Also [1]
implements the idea of partitioning BVH into treelets
2.1 Known details
(which approximately matches cache sizes) and group-ing
rays according to treelets they intersect. Another In summary, for both graphics APIs the corre-
architecture - STRaTA [7] is built on top of the TRaX sponding extensions add functionality to create ray
[16] and implements modified treelet technique of [1] tracing pipeline with the corresponding new shader
and streaming approach to processing rays associated types, commands and objects for acceleration struc-
with each treelet. STRaTA adds special small buffers to tures, and tools to associate shader groups with accel-
memory hierarchy to store rays. eration structures (i.e. shader binding table).
In [15] authors focus on improvements related to Acceleration structure is represented as two-level
memory access, in particular, completely avoiding tree. Bottom level acceleration structure (BLAS) ob-
random memory access during ray traversal. Their jects contain actual vertices and top level acceleration
approach is based around presenting data needed for ray structure (TLAS) contains BLAS object instances i.e.
tracing in two streams - stream of geometry data split transformation matrices. The building process is done on
in segments and stream of rays collected as a queue the GPU, acceleration structure is some form of BVH
per geometry segment they intersect. This al-lows for [17].
fetching geometry and rays from main mem-ory into Ray tracing pipeline has five shader types - ray
caches before they are needed for traversal. generation, miss, closest hit, any hit and intersection.
Work [6] in addition to MIMD execution model Shader programs of first three types are mandatory
and treelets proposes using reduced precision BVH and the last two are optional. All stages of ray tracing
traversal which also allows for chip area and power algorithm are programmable. There is built-in ray-
savings. Another specific point of [6] is that au- triangle intersection shader which is used by default.
thors propose small solution which can be integrated Official whitepaper [22] states that RT core has ray-
into existing GPU architecture. There are also works triangle intersection unit inside. In [18] authors show
focused on developing mobile ray tracing hardware 2-3.5 times improvement in performance of their algo-
(such as [8, 11]). These solutions usually have such rithm of point location in tetrahedral meshes when us-ing
common properties as MIMD execution model, hard- built-in triangle intersection unit on Turing hard-ware
ware traversal and intersection units. Raycore [11] has while Volta hardware (which has no RTX cores, so
distinctive properties that separate it from other ar- software fallback is used for RTX functionality)
chitectures - it’s fully fixed function Whitted-style ray shows performance loss in the same scenario.
tracing [19], it uses kD-tree as acceleration structure
and includes hardware unit for kD-tree construction. 2.2 Experiments
Summary. Overall, quite a few different architec-
To understand how RTX works under the hood we
tures and hardware acceleration techniques for ray
conducted several experiments. As a base for our
tracing were proposed over the years. Detailed re-
investigations we implemented a basic path tracing
view and comparison can be found in [2]. Some of the
algorithm [5] and compare it to Open Source imple-
mentioned architectures had been implemented in FP-
mentation of path tracing in Hydra Renderer [24].
GAs. Production level hardware applications besides
Implementation of a minimal path tracer using
Nvidia RTX are represented by [4] and mobile GPUs by
RTX in Vulkan or DirectX 12 would require devel-
Imagination technologies [21]. However, both of
oper to:
those have no published details, [4] is discontinued
and [21] is not yet available. Therefore, RTX is the 1. build acceleration structures using ray tracing extension
first hardware ray tracing acceleration technology to API;
2. create ray tracing pipeline containing at least ray
generation, closest hit and miss shader programs;
3. create shader table to bind shader programs to ac-
celeration structures;
4. create and execute command buffers on created
pipeline.
There are several design options even in the minimal
implementation using RTX which can potentially affect
performance. For example, the shading and lighting
code can be executed in a ray generation shader, in
a single (closest) hit shader or in several hit shaders.
We tested two different implementations according to Fig. 1. Time spent by ray tracing "draw call" per frame (1
best practices of RTX for Vulkan and DX12: sample per pixel, 1024 x 1024 resolution) depending on
1. impl_1 (Vulkan): ray generation shader creating rays traced per depth level. Depth = 3
ray(s) for each pixel in a cycle until the specified
tracing depth is reached; Next, we modified impl_2 with tracing several rays at
2. impl_2 (DirectX): ray generation shader spawning each depth level essentially transforming it into an
primary ray and closest hit shader taking care of implementation of branched (recursive) path tracing. As
can be seen in fig. 1, the time increases consis-tently
generating rays until specified depth is reached. To
with the number of rays, even slower in some cases.
measure performance in all our experiments we For example, with 4 rays per depth level the total
used Nvidia Nsight Graphics software and 2 GPUs—
number of rays is 7 times higher than for 1 ray per
GTX1070 and RTX2070. It is known that while
depth level (21 against 3). And the performance drop is
RTX2070 has hardware acceleration for ray tracing,
6 times for Sponza and 3.6 for Hairballs.
GTX1070 has software implementation of RTX. Using
this setup we captured frames from our path tracing 3. Results and discussion
application and logged time spent by vkCmdTraceR-
aysNV (Vulkan) or DispatchRays (DirectX 12) func- Conclusion #1: Nvidia RTX is primarily aimed at
tion and «BVH4TraversalInstKernel» kernel in Hy-dra accelerating random access to memory during ray
Renderer. In our first set of experiments we ran tracing. More specifically, traversing BVH tree with a
implemented path tracer on three scenes (Sponza, sets of random rays. This conclusion stems from (fig 2,
CrySponza, Hairballs) with different tracing depth. right), where we can see that hardware implemen-tation
From measured time we calculated frames per second on the small scene (Sponza) wins only 2 times (477 vs
and approximate amount of rays traced per second as: 1140) with «coherent» and «sorted» sets of primary
rays. But breaks away 4-5 times for the same Sponza
and incoherent rays (122 vs 561). Moreover, large
rays = width ∗ height ∗ spp ∗ f ps (1) scene (Hair Balls) shows same 4-5 times for both
primary (58 vs 283) and secondary (50 vs 210) rays.
The fact that acceleration is preserved on the scene
where the bottleneck is the memory confirms our con-
width, height – rendering resolution, spp – samples per clusion.
pixel, f ps – frames per second. Conclusion #2: Nvidia RTX implements some ray-
grouping/ray-sorting. It’s done probably in
combination with GPU work creation (see conclusion
scene primary secondary tertiary #4). This assumption is confirmed by the fact that on
Sponza, impl_1 807 437 806 simple scenes (like Sponza) hardware implementa-tion
Sponza, impl_2 928 777 694 doesn’t have significant performance drop when we
Sponza, Hydra_SW 480 122 130 move from primary to secondary rays (table 1, fig1).
Crysponza, impl_1 806 419 388 At the same time software implementation sees its
Crysponza, impl_2 754 635 216 performance degrade much faster. However, on the
Crysponza, Hydra_SW 276 92 80 scene where ray grouping could not help (Hair balls),
Hairballs, impl_1 275 223 256 both hardware and software implementation don’t have
Hairballs, impl_2 567 155 141 significant performance difference between primary and
Hairballs, Hydra_SW 61 50 56 secondary rays.
Table 1. Million rays traced per second (Mrays/s), 1
sample per pixel, 1024 x 1024 resolution, RTX2070
Fig. 2. Comparison on GTX1070 (left) and RTX2070 (right) (Open Source implementation vs Nvidia RTX). The left part
of each image (green) shows performance for primary (coherent) rays, and the right part (red) for secondary (random)
rays.
Sponza (66K tris) Cry Sponza (262K tris) Hair Balls (224M tris)
Fig. 3. Test Scenes
Fig. 4. Supposed internal architecture of Nvidia RTX. According to the results of our experiments, the hardware
implementation should be closely connected with the texture units, or it is part of texture unit. We believe the most
interesting part is related to reordering of memory access and thus it should work in analogue to well known memory
access reordering inside texture units. In this way, traversal unit itself could be small enough and probably implements
reduced precision BVH traversal [6] (or some analogue) for better cache efficiency and reducing HW cost.
Conclusion #3: Despite the Nvidia attempt, – Eurographics Association, 2010. – p. 113-122.
placing the whole code in a single kernel («CPU [2] Deng Y. et al. Toward real-time ray tracing: A survey on
style» or «uber kernel») is still inefficient for GPUs. hardware acceleration and microarchitecture tech-niques //
We make such conclusion because of 2 main reasons. ACM Computing Surveys (CSUR). – 2017. –�. 50. – №. 4.
First, open source implementation with separate ker-nel – p. 58.
[3] Gribble C. P., Ramani K. Coherent ray tracing via
in Hydra Renderer benefits almost 2 times over
stream filtering //2008 IEEE Symposium on Interac-tive
Nvidia RTX for pure software case (fig. 2, left). Sec-
Ray Tracing. – IEEE, 2008. – p. 59-66.
ond, when comparing 2 slightly different implementa- [4] Hall. D. The AR350: Today’s ray trace rendering
tions of RTX in Vulkan and DX12 we have found dra- processor. //Eurographics/SIGGRAPH workshop on
matic changes in performance depending on a slight Graphics hardware - Hot 3D Session 1, 2001
change in the complexity of shaders in [5] Kajiya J. T. The rendering equation //ACM SIG-GRAPH
«impl_1» (more complex) vs «impl_2» (simpler), table computer graphics. – ACM, 1986. – �. 20. –№. 4. – p.
1. This can be explained by occupancy drop 143-150.
depending on code complexity and register pressure. [6] Keely S. Reduced precision hardware for ray tracing.//Proc.
Conclusion #4: Nvidia RTX uses GPU work cre-ation HPG. – 2014. – p. 29-40.
for rays. This conclusion is confirmed by simple [7] Kopta D. et al. An energy and bandwidth efficient ray
tracing architecture //High-performance Graphics. –ACM,
observation. When we generated random amount of
2013. – p. 121-128.
rays (10 to 40), we got 2 times slower in comparison
[8] Lee W. J. et al. SGRT: A mobile GPU architecture for
with 10 rays. In contrast to ray tracing, when we cal- real-time ray tracing //High-performance graphics
culated Perlin Noise with random noise function calls conference. – ACM, 2013. – p. 109-119.
(10 to 40), we got exactly 4 times of what we should [9] Meißner M. et al. VIZARD II: a reconfigurable inter-
have without GPU work creation. Our experiment active volume rendering system //ACM Eurographics conf.
with recursive ray tracing (fig.1) also confirms GPU on Graphics hardware. – Eurographics Associa-tion,
work creation presence since the time is proportional to 2002. – p. 137-146.
the number of rays. [10] Nah J. H. et al. T&I engine: traversal and intersection
engine for hardware accelerated ray tracing //ACM
4. Final conclusion Transactions on Graphics (TOG). – ACM, 2011. – �.
30. – №. 6. – p. 160.
Our main conclusion is that Nvidia RTX is some [11] Nah J. H. et al. RayCore: A ray-tracing hardware ar-
sort of «general» technology, oriented to speeding up chitecture for mobile devices //ACM Transactions on
random memory access and irregular work distribu- Graphics (TOG). – ACM, 2014. – �. 33. – №. 5. – p. 162.
tion on GPUs. In this way we can expect in near fu- [12] Pfister H. et al. The VolumePro real-time ray-casting
ture different sets of algorithms (at least some spatial system. //Computer graphics and interactive tech-
search algorithms) to be hardware accelerated. niques. – N.Y.: Association for Computing Machinery.
We believe Nvidia puts a lot of efforts in their com- – 1999. – p. 251-260.
piler and software support of GPU work creation. On [13] Schmittler J., Wald I., Slusallek P. SaarCOR: a hard-
the example of this technology we can see, that «the ware architecture for ray tracing //ACM SIGGRAPH conf.
on Graphics hardware. – Eurographics Associa-tion,
golden age of software» has ended and the «the golden
2002. – p. 27-36.
age of compilers and HW/SW projects» has started. [14] Schmittler J. et al. Realtime ray tracing of dy-
Despite the overall complexity of Vulkan and namic scenes on an FPGA chip //ACM SIG-
DX12, such improvements make GPU implementation of GRAPH/EUROGRAPHICS conf. on Graphics hard-ware.
complex rendering engine much simpler for devel-oper. – ACM, 2004. – p. 95-106.
On the other hand, this simplicity is achieved at the [15] Shkurko K. et al. Dual streaming for hardware-
cost of tying the project to a fairly heavy tech-nology. accelerated ray tracing //High Performance Graphics.
We believe that efficient software implemen-tation of – ACM, 2017. – p. 12.
RTX will be complex and expensive due to GPU work [16] Spjut J. et al. TRaX: A multi-threaded architecture
creation and specific compiler that Nvidia puts inside for real-time ray tracing //Symposium on Application
RTX — even Nvidia’s software implemen-tation on Specific Processors. – IEEE, 2008. – p. 108-114.
[17] Stich M. Real-time raytracing with Nvidia RTX,
GTX1070 essentially loses to simple and
GTC EU 2018
straightforward open source ray tracing implementa-tion [18] Wald I. et al. RTX Beyond Ray Tracing: Exploring
in Hydra Renderer. the Use of Hardware Ray Tracing Cores for Tet-Mesh
Point Location. Authors’ Preprint — to be presented at
5. Acknowledgments
High-Performance Graphics 2019
This work was sponsored by RFBR 18-31-20032 [19] Whitted T. An improved illumination model for shaded
grant. display //ACM SIGGRAPH – ACM, 1979. –�. 13. – №. 2.
– �. 14.
6. References [20] Woop S., Schmittler J., Slusallek P. RPU: a pro-
grammable ray processing unit for realtime ray trac-
[1] Aila T., Karras T. Architecture considerations for tracing
incoherent rays //High-performance Graphics.
ing //ACM Transactions on Graphics (TOG). – ACM,
2005. – �. 24. – №. 3. – p. 434-444.
[21] Imagination technologies. PowerVR Ray Tracing.
2019. URL = https://www.imgtec.com/graphics-
processors/architecture/powervr-ray-tracing/
[22] Nvidia Turing architecture whitepaper. 2019
URL = https://www.nvidia.com/content/dam/en-zz/
Solutions/design-visualization/technologies/turing-
architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf
[23] Nvidia RTX Ray tracing developer resources. 2019
URL = https://developer.nvidia.com/rtx/raytracing Ray
[24] Tracing Systems, Keldysh Institute of Applyed
Mathematics, Moscow State Uiversity. Hydra Ren-
derer. Open source rendering system. 2019 URL =
https://github.com/Ray-Tracing-Systems/HydraAPI Vulkan
[25] specification. 2019 URL = https://
www.khronos.org/registry/vulkan/specs/1.1-extensions/
html/vkspec.html