RAGA

RAGA: Real Time Ray Traced Gaussian Shadow Casting for 3DGS Avatar-Scene Interaction

Real-time shadow casting for animated 3D Gaussian Splatting avatars in 3DGS scenes. Unlike prior methods that use binary hit tests, RAGA computes shadows entirely in Gaussian space via closed-form ray–Gaussian line integrals and includes a stable avatar proxy to eliminate temporal flickering.

• gaussian-splatting
• avatar
• 3d-avatar
• realtime
• face-animation

Authors: Aymen Mir, Riza Alp Guler, Jian Wang, Peter Wonka, Bing Zhou, Gerard Pons-Moll

Categories: cs.CV

Comment: ECCV 2026. Project Page at https://miraymen.github.io/raga/

Published 2026-06-28 · Updated 2026-06-28

Links

Paper & demos

• arXiv
• Project page

Impact

• Semantic Scholar1 citation

Abstract

We study the problem of physically plausible shadow casting when animating 3D Gaussian Splatting (3DGS) avatars, either individually or in multi-avatar and object-interaction scenarios, within existing 3DGS scenes. In contrast to prior methods that rely on binary hit tests and mesh-based shadow casters, our method performs shadow computation entirely in Gaussian space, without requiring any mesh reconstruction. We introduce RAGA, a Ray-Traced Gaussian Shadow Casting formulation based on exact ray-Gaussian line integrals. For each occluding Gaussian, we integrate the opacity profile along the shadow ray and normalize by the theoretical maximum integral, producing a weight that captures how the ray traverses the occluder rather than merely whether an intersection occurred. To reduce temporal variance from clothing deformations in animated avatars, we further introduce an avatar proxy representation that stabilizes shadow casting while preserving visual fidelity. We implement RAGA using custom CUDA kernels integrated with the NVIDIA OptiX framework; as such, our shadow tracer runs at rates of about 50 FPS. We evaluate on single-avatar, multi-avatar, and avatar-object interaction scenarios across multiple datasets, demonstrating substantially improved shadow realism, temporal stability, and scene coherence. Our project page is available at https://miraymen.github.io/raga/.

---

Introduction and Motivation

3D Gaussian Splatting (3DGS) has emerged as a powerful representation for high-quality scene reconstruction and, more recently, for animatable human avatars embedded in real-world environments. These capabilities open the door to applications in virtual production, CG compositing, and simulation pipelines. Yet a critical gap persists: when animated 3DGS avatars are composited into 3DGS scenes, physically plausible shadows are almost universally absent. Without shadows, inserted humans appear visually detached from their surroundings — a problem that becomes especially pronounced in dynamic settings where shadows provide essential spatial and temporal cues.

Existing approaches fall into three broad categories, none of which fully solves the problem:

• Classical shadow mapping relies on surface intersections and depth tests, which assume a well-defined inside–outside structure. 3DGS is a volumetric, opacity-based representation with no such structure, making classical pipelines inapplicable.
• Inverse rendering / relighting methods for 3DGS decompose scenes into materials and lighting and recover self-shadows via baked occlusion volumes, ray-traced visibility, or learned shadow parameters. These are per-scene optimizations on static geometry; compositing a new animated avatar into the scene does not update the baked or learned shadow representation.
• Ray-based splatting methods (e.g., 3DGRT, RaySplat) support ray–Gaussian intersection for rendering but rely on binary hit tests for shadow casting, ignoring how deeply the ray traverses each Gaussian. Furthermore, they assume mesh-based shadow casters and do not address the fully volumetric setting where both caster and receiver are 3D Gaussians.

RAGA addresses this gap with a principled, fully Gaussian-based shadow casting framework. The key idea is to cast rays from scene Gaussians toward the light source and model light attenuation using a normalized ray–Gaussian line integral — a closed-form quantity that captures exactly how much of each occluding Gaussian the shadow ray traverses, rather than merely whether an intersection occurred.

Capability Comparison

The table below summarizes the capabilities of prior methods versus RAGA across five axes: composability (can shadows be cast at composition time onto new scenes?), 3DGS shadow caster support, 3DGS shadow receiver support, dynamic 3DGS avatar support, and temporal stability.

† Can be adapted for use with 3DGS shadow casters.

Method

RAGA operates entirely in Gaussian space. Given a 3DGS scene and one or more animated 3DGS avatars, the goal is to compute physically plausible per-frame shadows without any mesh reconstruction. The method proceeds in three tightly coupled stages: exact ray–Gaussian intersection, a closed-form truncated line integral, and normalization by the theoretical maximum integral. An avatar proxy provides temporal stabilization.

Gaussian Representation and Shadow Ray

Each occluding Gaussian is parameterized by its mean $\boldsymbol{\mu} \in \mathbb{R}^3$, rotation matrix $\mathbf{R} \in \mathrm{SO}(3)$, and per-axis scales $\mathbf{s} = (s_x, s_y, s_z)$, giving the covariance:

The unnormalized Gaussian kernel (with maximum value 1 at the center) is:

For a receiver point $\mathbf{o}$ and a point light at position $\boldsymbol{\ell}$, a shadow ray is defined as:

Exact Ellipsoid Intersection

To simplify intersection tests, the method applies a whitening transform defined by the matrix:

so that the transformed coordinate $\mathbf{y} = \mathbf{L}(\mathbf{x} - \boldsymbol{\mu})$ satisfies $\|\mathbf{y}\|^2 = (\mathbf{x}-\boldsymbol{\mu})^\top \boldsymbol{\Sigma}^{-1}(\mathbf{x}-\boldsymbol{\mu})$. The shadow ray in this space becomes:

Contributions are restricted to the $\chi^2$-confidence ellipsoid (using $\chi^2 = 9$, a $3\sigma$ cutoff): $$\|\mathbf{y}\|^2 \le \chi^2$$ This yields a quadratic constraint along the ray: $$\|\mathbf{o}' + \tau\mathbf{v}'\|^2 = A\tau^2 + 2B\tau + C, \quad A = \|\mathbf{v}'\|^2,\; B = \mathbf{o}'^\top\mathbf{v}',\; C = \|\mathbf{o}'\|^2$$

The point of closest approach to the Gaussian center along the ray occurs at $\tau_c = -B/A$, and the squared perpendicular Mahalanobis distance is: $$d_\perp^2 = C - B^2/A$$

If $d_\perp^2 > \chi^2$, the ray misses; otherwise, with $\Delta = \chi^2 - d_\perp^2 \ge 0$, the entry and exit times are: $$\tau_\text{in} = \tau_c - \sqrt{\Delta/A}, \quad \tau_\text{out} = \tau_c + \sqrt{\Delta/A}$$ These are clamped to $[0,1]$ and the Gaussian is discarded if $\tilde{\tau}_\text{in} \ge \tilde{\tau}_\text{out}$.

Why not just use the peak response? Evaluating the Gaussian at the closest point $\tau_c$ gives $g(\mathbf{x}(\tau_c)) = \exp(-\tfrac{1}{2}d_\perp^2)$, which depends only on the perpendicular distance and is entirely direction-agnostic. Two rays at the same distance but traversing vastly different depths through the Gaussian receive identical weights. Worse, a ray barely grazing a large Gaussian along its major axis can have a small $d_\perp$ and thus a high peak response, despite minimal volumetric obstruction. The line integral resolves both issues.

Truncated Line Integral in Closed Form

Along the shadow ray, the Gaussian kernel reduces to a 1D Gaussian in $\tau$: $$g(\mathbf{x}(\tau)) = \exp\!\left(-\tfrac{1}{2}d_\perp^2\right) \cdot \exp\!\left(-\tfrac{1}{2}A(\tau - \tau_c)^2\right)$$

Integrating over the clamped ellipsoid intersection interval $[\tilde{\tau}_\text{in},\, \tilde{\tau}_\text{out}]$ admits a closed-form solution via the error function: $$I_\text{line} = \exp\!\left(-\tfrac{1}{2}d_\perp^2\right) \sqrt{\frac{\pi}{2A}} \left[\operatorname{erf}\!\left(\sqrt{\tfrac{A}{2}}(\tilde{\tau}_\text{out}-\tau_c)\right) - \operatorname{erf}\!\left(\sqrt{\tfrac{A}{2}}(\tilde{\tau}_\text{in}-\tau_c)\right)\right]$$

This integral accumulates opacity over the full traversal length, naturally producing lower weights for shorter paths through the Gaussian and correctly differentiating between rays that graze versus rays that pass through the core of an occluder.

Normalization by Maximum Line Integral

To make the integral dimensionless and comparable across Gaussians of varying size, it is normalized by the maximum attainable line integral $I_\text{max}$ — the value obtained when a ray passes through the Gaussian center along its longest axis $s_\text{max} = \max(s_x, s_y, s_z)$: $$I_\text{max} = \sqrt{2\pi}\;\frac{s_\text{max}}{\|\mathbf{v}\|}\;\operatorname{erf}\!\left(\sqrt{\tfrac{\chi^2}{2}}\right)$$

The normalized thickness factor is then: $$\eta = \operatorname{clamp}\!\left(\frac{I_\text{line}}{I_\text{max}},\, 0,\, 1\right) \in [0,1]$$

$\eta$ is near 0 for grazing intersections and approaches 1 for a center pass along the major axis. This is the key design choice: it captures how the ray traverses the Gaussian, not merely whether an intersection occurred. The same Gaussian crossed along its narrow direction yields a lower $\eta$ than one crossed along its long axis, correctly reflecting the reduced volumetric obstruction.

Shadow Transmittance Update

Each occluder carries an opacity parameter $\alpha \in (0,1)$. Three transmittance update rules are considered:

Linear normalized model: $$T \leftarrow T\,(1 - \alpha\,\eta)$$

Exponentiated normalized model (used in all experiments): $$T \leftarrow T\,(1-\alpha)^\eta$$ This is equivalent to attenuating with effective opacity $\alpha_\text{eff} = 1 - (1-\alpha)^\eta$.

Beer–Lambert integral model: $$T \leftarrow T\,\exp\!\left(-\kappa(\alpha)\,I_\text{line}\right), \quad \kappa(\alpha) = -\ln(1-\alpha)$$

The exponentiated model provides the best empirical trade-off between physical plausibility and numerical stability (validated by ablation).

BVH Acceleration via NVIDIA OptiX

To accelerate intersection queries over potentially hundreds of thousands of avatar Gaussians, a BVH is built over per-Gaussian axis-aligned bounding boxes (AABBs) that conservatively enclose each $\chi^2$ ellipsoid. The projection radius of the ellipsoid along world axis $d \in \{x,y,z\}$ is: $$e_d = \sqrt{\chi^2\,(s_x^2 R_{d1}^2 + s_y^2 R_{d2}^2 + s_z^2 R_{d3}^2)}$$ yielding $\text{AABB} = [\boldsymbol{\mu} - \mathbf{e},\; \boldsymbol{\mu} + \mathbf{e}]$. Only BVH candidates invoke the exact quadratic test and integral computation. The BVH is built and traversed using NVIDIA OptiX, leveraging hardware RT cores for near real-time shadow tracing at approximately 50 FPS.

Shadow Rendering

After computing transmittance $T_k \in [0,1]$ for each scene Gaussian $k$, its color is modulated during 3DGS rasterization: $$\mathbf{c}_k' = T_k\,\mathbf{c}_k$$ where $T_k = 1$ indicates full illumination and $T_k \to 0$ indicates full shadow. The modified colors are then splatted with standard $\alpha$-blending to produce the final shadowed rendering.

Avatar Proxy for Temporally Stable Shadows

A critical challenge in dynamic avatar shadow casting is temporal flickering. The pose-dependent Gaussian generator $\mathcal{H}(\boldsymbol{\theta}_t)$ — a neural network mapping body pose $\boldsymbol{\theta}_t$ to 3D Gaussian primitives — produces outputs that fluctuate in topology, opacity, and anisotropic orientation to satisfy view-dependent photometric losses. When these raw primitives are used as shadow occluders, the shadow silhouette flickers and oscillates rapidly between frames.

RAGA addresses this by decoupling render-ready geometry from shadow-casting geometry via a stable avatar proxy built in three steps:

1. Canonical topology locking. The generator is frozen at a canonical reference pose $\boldsymbol{\theta}_\text{ref}$ (e.g., T-pose), caching the configuration once: $$\mathcal{P}_\text{proxy} = \{(\boldsymbol{\mu}_k^{(0)}, \mathbf{s}_k^{(0)}, \alpha_k^{(0)})\}_{k=1}^N = \mathcal{H}(\boldsymbol{\theta}_\text{ref})$$ This fixes the number of primitives $N$ and their initial attributes, preventing artifacts from primitives appearing and disappearing between frames.

2. Isotropic regularization. The anisotropic scale is overridden with a conservative isotropic radius $r_k = \max(s_{k,x}^{(0)}, s_{k,y}^{(0)}, s_{k,z}^{(0)})$, making the covariance $\boldsymbol{\Sigma}'_k = r_k^2\,\mathbf{I}$. Since a sphere's silhouette is rotation-invariant, the shadow projection becomes independent of primitive orientation.

3. Simplified kinematics. At inference, the proxy is articulated using Linear Blend Skinning (LBS) applied strictly to positions: $$\boldsymbol{\mu}_k^{(t)} = \sum_{j \in \mathcal{J}} w_{kj}\,\mathbf{M}_j(\boldsymbol{\theta}_t)\,\boldsymbol{\mu}_k^{(0)}$$ where $\mathcal{J}$ denotes skeleton joints and $w_{kj} \ge 0$ are normalized skinning weights. Crucially, the orientations of the proxy primitives are not updated; since they are isotropic spheres, their rotation is irrelevant to ray intersection, guaranteeing a continuous shadow signal with respect to pose.

Experiments

Setup and Pseudo-GT Protocol

Since RAGA addresses a novel problem with no prior ground truth, a pseudo-GT protocol is established using mesh-based shadow rendering in two settings:

• ScanNet++ scenes: Both a 3DGS reconstruction and a mesh reconstruction are available. The avatar is replaced with a posed SMPL mesh and classical shadow mapping is used onto the scene mesh. Shadow attenuation is rendered onto a transparent surface and composited with the 3DGS scene. Five ScanNet++ scenes are used for quantitative evaluation.
• General 3DGS scenes (e.g., SuperSplat): For scenes without a mesh reconstruction, an approximate floor plane is constructed from the 3DGS point cloud, the avatar is replaced with a posed SMPL mesh, and classical rendering casts shadows onto the floor proxy.

The authors note that mesh-based shadow casting is not a practical general solution: (1) mesh extraction from 3DGS is lossy and non-trivial; (2) SMPL-based mesh proxies discard clothing and hair detail from shadow silhouettes; and (3) mesh proxies do not generalize to arbitrary 3DGS objects inserted into scenes. The avatar proxy, while smoother than raw Gaussians, retains significantly more geometric fidelity than a bare SMPL mesh.

Avatars and Baselines

Avatars are sourced from ActorsHQ (48 cameras), AvatarReX, and NeuralDome (for multi-avatar and avatar-object scenarios). Two Gaussian-based baselines are constructed from recent ray-tracing methods:

• 3DGRT (modified): Ray–Gaussian intersections are detected via an icosahedron approximation per Gaussian; detected intersections accumulate opacity for ray attenuation. Neither RAGA's line integral nor avatar proxy is used.
• RaySplat (modified): Intersections are detected via the analytic test from RaySplat; opacity is accumulated for each detected Gaussian. Again, neither RAGA's line integral nor proxy is used.
• Mesh Shadow (oracle): Shadows are cast from the posed SMPL mesh onto a transparent mesh receiver via classical shadow mapping, then alpha-blended onto the 3DGS scene.

Shadow Quality Evaluation

Three pixel-space metrics are computed in an avatar-centric region of interest:

• SAE (Shadow Attenuation Error): Mean absolute error between predicted and pseudo-GT shadow maps within the shadow region (lower is better).
• SM-IoU (Shadow Matte IoU): Intersection-over-union of binarized shadow masks (threshold 0.1) (higher is better).
• BF (Boundary F-measure): F-score between shadow boundaries with 2-pixel tolerance (higher is better).

RAGA achieves the best results among all Gaussian-based approaches, approaching the mesh oracle. 3DGRT's binary hit test based on the icosahedron approximation produces hard shadow boundaries without penumbrae, leading to poor SAE and BF scores. RaySplat's collision-based approach reduces to max-response evaluation that lacks proper attenuation modeling.

Qualitative Comparison

Temporal Stability

Temporal shadow consistency (TSC) is measured as the mean per-pixel absolute difference between consecutive shadow frames: $$\text{TSC} = \frac{1}{F-1}\sum_{t=1}^{F-1}\frac{1}{|\Omega|}\sum_{p\in\Omega}|S_t(p) - S_{t-1}(p)|$$ Lower TSC indicates more stable shadows.

The avatar proxy reduces TSC by more than 3× compared to using raw avatar Gaussians, and by more than 6× compared to the best baseline.

Perceptual Study

A two-alternative forced-choice (2AFC) perceptual study is conducted with 12 naive raters across 5 ScanNet++ scenes. Raters view 5 video clips (10 seconds at 30 fps) per scene with randomized method order and placement, comparing RAGA against a no-shadow baseline. Raters prefer RAGA 73% of the time on average.

Ablation Studies

Attenuation Model and Normalization

The exponentiated model consistently outperforms both linear and Beer–Lambert variants. Without $I_\text{max}$ normalization, large anisotropic Gaussians cast disproportionately dark shadow blobs due to their larger raw line integrals — an artifact clearly visible in the visual ablation below.

Avatar Proxy Ablation

Both topology locking and isotropic regularization contribute meaningfully to temporal stability. Removing either component significantly increases TSC, and removing both (no proxy) more than triples the flickering metric.

Runtime Analysis

Runtime is analyzed on an NVIDIA H100 GPU with approximately 500K scene Gaussians, varying avatar complexity from 20K to 400K Gaussians.

The RAGA shadow tracer (implemented as a CUDA kernel integrated with NVIDIA OptiX) runs at approximately 50 FPS for typical avatar sizes, enabling near real-time shadow casting even for dense avatars. The end-to-end pipeline, including 3DGS rasterization, achieves near real-time performance thanks to OptiX hardware ray tracing and BVH acceleration.

Qualitative Results Across Scenarios

RAGA produces soft, physically plausible shadows that follow avatar motion and respect scene geometry across a diverse range of scenarios. The method handles single avatars, multiple interacting avatars, and avatars interacting with 3DGS objects placed in 3DGS scenes — all without any mesh reconstruction.

Summary of Contributions

• Fully Gaussian-space shadow casting framework for 3DGS avatars animated in 3DGS scenes, without requiring mesh reconstruction at any stage.
• RAGA formulation: ray-traced shadow casting based on exact ray–Gaussian line integrals with closed-form error function solutions, replacing the approximate proxy-geometry intersections used in prior work.
• Avatar proxy strategy: canonical topology locking combined with isotropic regularization and LBS-based articulation, providing temporally stable shadows under dynamic avatar deformations.
• Hardware-accelerated implementation: custom CUDA kernels integrated with NVIDIA OptiX BVH, achieving ~50 FPS shadow tracing on an H100 GPU.
• Comprehensive evaluation: quantitative and qualitative results on single-avatar, multi-avatar, and avatar-object scenarios across ScanNet++, ActorsHQ, AvatarReX, NeuralDome, and SuperSplat datasets, plus a perceptual study showing 73% preference over no-shadow rendering.

Limitations and Future Work

The authors acknowledge several limitations of the current system:

• Light model: Only point and directional light sources are modeled; area lights are not supported.
• Baked scene lighting: Lighting on both avatar and scene is baked into the 3DGS representation. RAGA modulates scene Gaussian colors to create shadows but does not relight the avatar or scene under novel illumination.
• Manual light specification: Light source positions are not estimated from the scene; the user must specify them manually.

Future directions include automatic light source estimation from 3DGS scenes, fully relightable avatars that respond to dynamic illumination, and support for area lights.