Gaussian Splatting
Gaussian Splatting represents a 3D scene as millions of tiny, colored, semi-transparent blobs that can be rendered in real time.
Overview
It delivers NeRF-like photorealism while running fast enough for interactive viewing.
Deep Dive
Introduced at SIGGRAPH 2023, 3D Gaussian Splatting reconstructs scenes from photos like NeRF but uses an explicit representation instead of a hidden neural network. Each scene is a cloud of 3D Gaussians, fuzzy ellipsoidal blobs, and each blob stores a position, a size and orientation (its covariance), an opacity, and color. Instead of slowly shooting rays through a network, the method "splats" these blobs directly onto the screen and blends them, a process closer to traditional rasterization and therefore very fast. Training starts from a sparse point cloud produced by camera calibration, then optimizes the blobs while adaptively adding detail where the scene is under-reconstructed and pruning where it is over-populated. The result is real-time rendering at 1080p with quality rivaling the best NeRFs, which is why it spread rapidly through graphics and capture tools.
Technical Insight
The key is a differentiable tile-based rasterizer. The 3D Gaussians are projected to 2D, sorted by depth, and alpha-blended per screen tile, so rendering avoids the per-pixel ray marching that makes NeRF slow. Color is stored with spherical harmonics, letting each blob change appearance with viewing angle to capture reflections. Because the whole pipeline is differentiable, the same photo-matching gradient descent used by NeRF optimizes blob positions, shapes, opacities, and colors, while a densification step grows or splits Gaussians to add missing detail.
Strategic Impact
Speed and scale
Visual AI can automate inspection, detection, and tagging tasks at scale.
Build choices
Creative teams can prototype concepts faster with fewer manual revisions.
Team and workflow
Operations can use image and video signals that were previously hard to process.
The Future of Gaussian Splatting
Gaussian Splatting is moving fast from research into products for 3D capture, mapping, and virtual production, partly because it renders in real time on consumer GPUs and even browsers. Active work targets shrinking file sizes (scenes can be large), handling dynamic and animated scenes, relighting, and editing individual objects. Expect tighter integration with game engines and AR/VR, hybrid methods that combine splats with meshes, and capture from phone video. It is increasingly seen as a practical complement to, or replacement for, NeRF wherever interactive speed matters.
Real-World Implementation
Creating real-time, explorable 3D captures of rooms or products for the web
Virtual production and film previsualization with photorealistic, navigable sets
Fast 3D scanning of objects and environments from a phone or drone video
Building interactive AR/VR scenes that run smoothly on consumer hardware
Risks & Guardrails
Image rights and consent can become legal risks if provenance is unclear.
Model performance can vary across lighting, demographics, and environments.
False positives may go unnoticed unless confidence thresholds are monitored.
Implementation Roadmap
Define acceptance criteria for precision, recall, and error costs.
Test with data that matches real production conditions.
Add human review for low-confidence or high-impact predictions.
Track model drift and revalidate after camera or dataset changes.
Keep Exploring
Free newsletter
Keep up with AI in 3 minutes a day
One short email each weekday with the three AI stories that actually matter. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Gaussian Splatting quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Next guide
Gaussian Processes
Frequently asked questions
What is Gaussian Splatting?
Gaussian Splatting represents a 3D scene as millions of tiny, colored, semi-transparent blobs that can be rendered in real time. It delivers NeRF-like photorealism while running fast enough for interactive viewing.
How does Gaussian Splatting represent a 3D scene?
The scene is an explicit cloud of 3D Gaussians, each a fuzzy ellipsoid with position, shape, opacity, and color.
Why is Gaussian Splatting much faster to render than the original NeRF?
Projecting and blending blobs via tile-based rasterization avoids NeRF's slow per-pixel ray marching, enabling real-time speeds.
What information does each 3D Gaussian store?
Each blob carries a position, a covariance (size and orientation), an opacity, and color information.
What is used to let a blob's color change with viewing angle?
Color is encoded with spherical harmonics, so each Gaussian can appear different from different directions, capturing reflections.
How does training typically begin in Gaussian Splatting?
It starts with sparse points from structure-from-motion camera calibration, then optimizes Gaussians from there.