From Point Clouds to Photorealism: How PRoGS Advances the State of Neural Rendering

What if a real-time automated decision-making process needs a computer vision system to produce photorealistic visuals in a mere fraction of a second? In many cases, this requires too much computational power — if it’s even possible at all. When computer vision uses 3D Gaussian splatting (3DGS), it provides a photorealistic image, but the process takes far too long. For more on this topic, see our article on computer vision for disaster responses.

This article discusses a new approach, Progressive Rendering of Gaussian Splats (PRoGS), which uses a contribution-based prioritization system that prioritizes each Gaussian based on how much it contributes to the overall quality of the scene. This technology was discussed in a paper written for the 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

How PRoGS Bridges the Gap Between Point Clouds and High-Quality Renderings

PRoGS produces high-quality renderings more efficiently by prioritizing larger, more opaque Gaussians and rendering them first. According to the paper, the process follows three basic steps:

1. PRoGS analyzes the training views of a 3D scene and identifies the biggest Gaussians that are the most opaque. Due to their size and opacity, these contribute more to the image of what’s actually in the scene.
2. PRoGS creates a prioritized list of the Gaussians it identified, ranking them according to how important they are for the overall quality of the scene.
3. Instead of loading all Gaussians at the same time, PRoGS renders them according to the prioritized order in step 2. As a result, it displays the most visually impactful Gaussians first, producing a more recognizable version of the scene faster.

While the scene is still blurry, it’s still recognizable, and gets clearer and clearer as PRoGS loads more Gaussians.

How PRoGS Produces High-Quality Renderings from a Visual Perspective

From a visual perspective, the PRoGS process is similar to an image being sketched step by step: It’s obvious early on what the artist is depicting, yet the specifics of the image only become evident as the artist fills in the granular details. Here’s how the process breaks down from a visual perspective:

1. Initial stage. PRoGS produces a blurry but easily recognizable depiction of the image consisting of only the most important Gaussian splats.
2. Intermediate stage. The image gets more definition as PRoGS adds more Gaussian splats. At this point, textures and edges start to sharpen.
3. Final stage. The image reaches full clarity as all splats get loaded. This results in a complete, high-quality image.

PRoGS vs. Traditional Rendering

The researchers also compare the results of the PRoGS process with those of a traditional rendering, showing the enhanced clarity produced by PRoGS with relatively few splats loaded.

The two processes are pitted against each other in the rendering of an image of a truck. With the traditional rendering tool, which uses a web viewer by Antimatter, it is only distinguishable that the image is a truck after 10% of the splats have been loaded.

However, with the PRoGS rendering tool, it is clear that the image is of a truck after only 0.2% of splats have been loaded.

The team also uses standard image quality benchmarks to objectively demonstrate the higher quality produced by PRoGS, including:

1. Peak-Signal-to-Noise Ratio (PSNR)
2. Learned Perceptual Image Patch Similarity (LPIPS)
3. Structural Similarity Index Measure (SSIM)

In combination, these measurements demonstrate that PRoGS significantly outperforms previous methods.

Potential Applications for PRoGS

While the potential use cases for PRoGS would include any system that depends on image rendering, here are some of the more straightforward applications:

1. Augmented Reality (AR) and Virtual Reality (VR): With PRoGS in the rendering loop, users observing AR and VR scenes can distinguish objects far faster than if traditional methods were used. The system could also produce recognizable images without using nearly as much memory. Since each scene can be distinguished after a fraction of splats have been loaded, the AR or VR experience could meet user expectations without consuming nearly as much data.
2. Robotics: Armed with PRoGS, a robot using computer vision to perceive a changing visual environment can understand its surroundings in real time without needing as many computational resources.
3. For instance, a robot working in an auto parts fulfillment facility can decide whether an object is a clutch plate or a transmission gear nearly instantly.
4. For remotely operated robots, PRoGS opens the possibility of controlling them using cloud-based platforms instead of local, hard-wired networks. Less data needed to produce a recognizable image means lower bandwidth requirements for the network. Therefore, even Wi-Fi-based networks connected to cloud tools may be able to control vision-enabled robots in real time.

PRoGS Introduces Faster, More Efficient Neural Rendering

ProGS renders accurate 3D images faster while using less computational resources. This means robotics, AR/VR, and other systems can save time and computing power, especially when they need to produce results in real time. ProGS holds significant promise for the future of neural rendering, specifically because it enables relatively modest computing systems to produce realistic images in far less time.

Download “PRoGS: Progressive Rendering of Gaussian Splats”