Neural Radiance Cache Technique Aims to Make Path Tracing Faster on NVIDIA GPUs Thanks to Tensor Cores and AI

At this year’s GTC, NVIDIA presented an interesting technique called Neural Radiance Cache whose goal is making path tracing faster and better thanks to AI and the Tensor Cores available in GeForce RTX GPUs.

As you can probably guess by its name, NRC is a new way to cache radiance based on neural networks. Originally described in the research paper ‘Real-time Neural Radiance Caching for Path Tracing’, the technique trains the neural networks (using path traced data) to predict radiance at any point in the 3D scene.

Instead of tracing the full path of rays from point X to Y and beyond, developers can simply query the Neural Radiance Cache to get an accurate estimate of that radiance. The advantage is immediately obvious, as the paths can be terminated earlier, thus improving both the speed (due to decreased texture loads, shading operations, and other savings) and the quality of the path tracing application.

As a bonus perk, Neural Radiance Cache produces less noise than regular path tracing since the estimated radiance comes from the cache. According to NVIDIA GeForce DevTech Engineer Jakub Boksansky, who explained NRC in the GTC 2023 talk, the technique is also very easy to integrate, needing very few parameters and being independent of materials, lights, shaders, et cetera. Last but not least, it’s trained at runtime, so it supports entirely dynamic scenes by default.

In practice, NRC works by running a path tracer and gathering the output data as training points. The path tracer is also modified to terminate 96% of the paths early, usually after two or three bounces, while only the remaining 4% of the paths are traced from beginning to end. The latter paths are used specifically to train the algorithm.

Neural Radiance Cache, which is said to be particularly great at global illumination and indirect lighting, will be added to the upcoming RTX GI 2.0 SDK. It’ll be a game-ready API coupled with an easy to learn user interface to speed up the integration even in existing games.

Right now, NRC only supports Microsoft’s DirectX 12, though Vulkan compatibility will be added at a later date. As mentioned earlier in the article, Neural Radiance Cache also requires a GeForce RTX GPU since it relies on its Tensor Cores to work.

NRC will be available with two APIs, a core NRC API that can be integrated into existing path tracers and a Diffuse GI API that won’t need a path tracer since it has a built-in one. In the latter case, users will have to submit geometry data to receive the predicted indirect Diffuse GI.

The core Neural Radiance Cache API currently has a 2ms overhead in its beta release, though 1ms is due to DX12 to CUDA interoperability which will be removed in the final version, leaving only 1ms of overhead which should be fully compensated with the speeding up of path tracing operations.

The Diffuse GI API is cheaper. NVIDIA estimates slightly over half a millisecond in overhead, but it is also a more limited subset of the full Neural Radiance Cache features. The 0.9 beta release of Neural Radiance Cache is coming soon, said NVIDIA, and it will include sample applications for both the core API and the Diffuse GI API.

It may be a while before any game developers implement NRC. For example, RTX Direct Illumination was originally presented at GTC 2021 and will only be implemented for the first time in the upcoming Cyberpunk 2077 RT: Overdrive mode, due to be available as a technology preview with the next title update for the PC version coming on April 11th.