agentleFS
jaccenjaccen/Awesome-Gaussian-Skills/skills/nerf-to-3dgs-migrator/SKILL.md

nerf-to-3dgs-migrator

Skill186 starsChanged 57 days agoApache-2.0View on GitHub

Migrate NeRF-based methods to 3DGS via the SLAT unified encode-decode framework. Analyzes component

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 9 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Before any migration, understand these fundamental differences:

    line 38
  2. Distinctive:

    When migrating NeRF methods that use custom density/sampling strategies, consider these modern alternatives to vanilla 3DGS ADC:

    line 261
  3. Distinctive:

    Provide runnable code: All code templates should be syntactically correct and importable.

    line 333
  4. Distinctive:

    No invented data: Never fabricate migration rules, compatibility data, or method characteristics not in the loaded reference files.

    line 348
  5. Distinctive:

    No hallucinated citations: Never invent paper titles, authors, DOIs, arXiv IDs, or venue names.

    line 349
  6. Distinctive:

    No silent speculation: If you are uncertain about a technical detail, explicitly flag it with "[UNCERTAIN]" rather than presenting it as …

    line 350
  7. Distinctive:

    No method misattribution: Do not assign features, results, or mechanisms from one method to another.

    line 351
  8. Distinctive:

    No oversimplified comparisons: Do not reduce multi-dimensional trade-offs to a single "better/worse" judgment without context.

    line 352
  9. Distinctive:

    If you cannot find a method, pattern, or data point in the loaded files, say so explicitly.

    line 367

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Representation boundary: read ../../references/slat-unified-representation.md before claiming learned encoding or conversion capabilities. The toolkit voxel prototype is not TRELLIS; arbitrary-format encoding, CAD decoding and additive loss guarantees are not established.

NeRF-to-3DGS Migration Guide

You are a 3D reconstruction expert with deep knowledge of both NeRF and 3D Gaussian Splatting paradigms. Help users migrate their NeRF-based methods to 3DGS, or design new methods that combine insights from both.

Core Paradigm Differences

Before any migration, understand these fundamental differences:

AspectNeRF3DGS
RepresentationContinuous (MLP + volumetric)Discrete (explicit Gaussians)
RenderingVolume rendering (ray marching)Splatting (α-compositing)
SamplingAlong rays (coarse-to-fine)Point-based (all Gaussians)
QueryPoint sampling + MLP forwardDirect attribute lookup
Density controlImplicit (MLP output)Explicit (clone/split/prune)
MemoryBounded (MLP params)Unbounded (grows during training)
SpeedSlow (per-pixel ray march)Fast (parallel rasterization)
Quality ceilingHigh (continuous)High (adaptive density)

SLAT: Why NeRF→3DGS Migration Works

v1.6.0 upgrade: This skill's migration workflow is now grounded in the SLAT (Structured LATent representation) framework. See ../../references/slat-unified-representation.md for the full theory.

The SLAT Perspective on NeRF→3DGS

NeRF and 3DGS are not two unrelated representations — they are two decodings of the same structured latent. This is why migration is possible at all:

NeRF (continuous MLP field)
       │
       ▼  ENCODE: sample density + color on voxel grid
┌──────────────────────┐
│   SLAT               │
│   (sparse voxel      │
│    latent)           │
└──────┬───────────────┘
       │
       ├── DECODE → 3D Gaussians (discrete, explicit)
       └── DECODE → NeRF (continuous, implicit) ← original source

Under SLAT, NeRF→3DGS migration is a re-decode operation: encode the NeRF's continuous field into structured latent (by sampling on a voxel grid), then decode to discrete Gaussians. Each component migration step in this skill corresponds to a SLAT feature channel mapping:

Migration Step (this skill)SLAT Feature ChannelWhy It Maps
Positional Encoding → SHAppearance featureBoth encode view-dependent color; SH is 3DGS-native
Density (σ) → Opacity (α)Geometry occupancyσ sampled at voxel → α per Gaussian
Color MLP → SH coefficientsAppearance featureMLP output → explicit SH per Gaussian
Deformation Field → offsetsDeformation hookTemporal field → per-Gaussian offset at time t
Appearance embedding → featuresAppearance featurePer-image vector → per-Gaussian feature
Hash Grid → per-Gaussian featuresGeometry+appearanceMulti-resolution → flat per-Gaussian vector
Coarse-to-Fine → Progressive trainingTraining scheduleBoth control resolution progression

Conversion Loss Budget for NeRF→3DGS

Under SLAT, NeRF→3DGS has low total conversion loss because:

  • Encoding loss is low: NeRF's continuous field can be densely sampled, capturing nearly all information
  • Decoding loss is low: 3DGS is a natural decode target — discrete Gaussians can approximate any continuous field

This explains why NeRF→3DGS migration generally preserves quality, while the reverse (3DGS→NeRF) loses the explicit structure advantage.

When SLAT Helps vs When Direct Migration Is Better

ScenarioSLAT-GuidedDirect Component Migration
Migrating one method, one-on-one❌ Overkill✅ Simpler, faster
Migrating to also support Mesh output✅ Encode once, decode to 3DGS + Mesh❌ Must redo for Mesh
Need to quantify migration quality✅ Loss budget framework❌ No unified metric
Designing a new hybrid NeRF-3DGS method✅ SLAT provides the theoretical basis❌ Ad-hoc
Quick prototype migration❌ Latent overhead✅ Direct is faster

Migration Workflow

Step 1: Component Analysis

Analyze the source NeRF method and classify each component:

┌─────────────────────────────────┐
│     NeRF Method Components      │
├─────────────────┬───────────────┤
│ Component       │ Migration     │
│                 │ Strategy      │
├─────────────────┼───────────────┤
│ Positional      │ → Per-Gaussian│
│ Encoding        │   SH/feature  │
├─────────────────┼───────────────┤
│ Density MLP     │ → Opacity     │
│ (σ)             │   attribute   │
├─────────────────┼───────────────┤
│ Color MLP       │ → SH coeffs   │
│ (c)             │   or feature  │
├─────────────────┼───────────────┤
│ Deformation     │ → Offset on   │
│ Field           │   μ/R/S       │
├─────────────────┼───────────────┤
│ Appearance      │ → Per-Gaussian│
│ Embedding       │   feature vec │
├─────────────────┼───────────────┤
│ Hash Grid /     │ → Per-Gaussian│
│ Feature Grid    │   features    │
├─────────────────┼───────────────┤
│ Regularization  │ → Modify ADC  │
│ (TV, depth,     │   or add loss │
│  normal)        │               │
├─────────────────┼───────────────┤
│ Coarse-to-Fine  │ → Progressive │
│ Sampling        │   training    │
└─────────────────┴───────────────┘

Step 2: Component-by-Component Migration

2.1 Positional Encoding → Per-Gaussian Features

NeRF approach: Points are sampled along rays, encoded via PE/hash grid, fed to MLP.

3DGS equivalent: Each Gaussian has explicit features stored as attributes.

Migration options:

NeRF Encoding3DGS MappingCode Pattern
Frequency PE (sin/cos)SH coefficients (built-in)Direct: SH is 3DGS's native encoding
Hash grid (Instant-NGP)Per-Gaussian feature vectorStore N-dim feature per Gaussian, concatenate with SH
Tri-plane encodingPer-Gaussian feature vectorSame as above
Multi-resolution hashAdaptive feature dimensionUse higher SH degree for important regions

Code template (PyTorch):

# Before: NeRF — encoding is computed on-the-fly
def query_mlp(points, rays):
    encoded = hash_grid(points)  # (N, D)
    density = density_mlp(encoded)
    color = color_mlp(encoded, rays)

# After: 3DGS — encoding is stored per-Gaussian
class GaussianModel:
    def __init__(self):
        self._xyz = nn.Parameter(...)       # position (N, 3)
        self._opacity = nn.Parameter(...)   # opacity (N, 1)
        self._features = nn.Parameter(...)  # encoded features (N, D)  ← NEW
        self._sh = nn.Parameter(...)        # SH coefficients (N, 3*K)

2.2 Density (σ) → Opacity (α)

Key difference: NeRF density σ ∈ [0, ∞), 3DGS opacity α ∈ [0, 1].

Migration:

# NeRF: α = 1 - exp(-σ * δ) where δ is step size
# 3DGS: α = sigmoid(raw_opacity)

# If you need density-like behavior from opacity:
density_from_opacity = -torch.log(1 - opacity + 1e-6) / voxel_size

2.3 Volume Rendering → Splatting

NeRF: C = Σ c_i * α_i * T_i (along ray, with T = Π(1 - α_j)) 3DGS: Same formula but Gaussians are sorted by depth, not sampled along ray.

Critical change: In NeRF, points are implicitly ordered by distance along ray. In 3DGS, you must explicitly sort all Gaussians by depth before compositing.

# NeRF: ordered by construction (ray march)
# 3DGS: must sort explicitly
sorted_indices = torch.argsort(depths, dim=0)  # depth = (N, 1)
gaussians_sorted = gaussians[sorted_indices]

2.4 Deformation Field → Gaussian Attribute Offsets

NeRF: Deformation field is queried at each sampled point. 3DGS: Apply deformation as offsets to Gaussian parameters.

# NeRF approach
def deform(points, t):
    delta = deformation_mlp(points, t)
    return points + delta

# 3DGS approach
class DeformableGaussians:
    def apply_deformation(self, t):
        # Option 1: Direct offset on position
        self._xyz = self.base_xyz + self.deformation_net(self.base_xyz, t)

        # Option 2: Offset on rotation and scale too
        self._rotation = self.base_rotation + delta_rotation(t)
        self._scaling = self.base_scaling * scale_factor(t)

2.5 Appearance Embedding → Per-Gaussian Appearance

# NeRF: appearance is a learned vector per-image
# 3DGS: store appearance-modulating features per Gaussian

class AppearanceGaussians:
    def __init__(self, num_gaussians, appearance_dim=32):
        self._appearance = nn.Parameter(
            torch.randn(num_gaussians, appearance_dim) * 0.01
        )

    def get_color(self, sh_features, image_idx):
        # Combine SH features with appearance
        combined = torch.cat([sh_features, self._appearance], dim=-1)
        return self.color_net(combined)

Step 3: Identify Incompatibilities

NeRF Feature3DGS CompatibilityWorkaround
Continuous opacity fieldImplicit → Explicit lossReplace with per-Gaussian opacity
Transmittance accumulationSame formula, different orderSort Gaussians by depth
Hierarchical samplingNot needed (all Gaussians visible)Remove, use ADC instead
NeRF-W / appearance per-imageNot native to 3DGSAdd per-Gaussian appearance features
SDF regularizationNo native SDF in 3DGSAdd depth/normal loss as post-hoc
Multi-resolution featuresExplicit per-GaussianStore feature vector, interpolate if needed
Ray-based queriesPoint-based queriesRestructure query pipeline

Recent Densification Alternatives (2026)

When migrating NeRF methods that use custom density/sampling strategies, consider these modern alternatives to vanilla 3DGS ADC:

MethodArXivWhat It ReplacesKey Difference
Softmax-GS (CVPR'26 Findings)2604.27437α-compositing renderingReplaces α-compositing with softmax competition — NeRF methods using volume density (σ) should note this alternative blending formulation when migrating the compositing step
LeGS (SIGGRAPH'26)2605.00408Heuristic clone/split/prune ADCRL-based density control learns when/where to add/remove Gaussians — replaces the fixed-threshold heuristics that NeRF-to-3DGS migrations often keep from vanilla 3DGS
Structure-Aware Densification (SIGGRAPH'26)2604.28016Vanilla isotropic splitFrequency-aware anisotropic splitting — when NeRF methods use frequency-based sampling or multi-resolution features, this provides a more principled densification strategy
BA-GS (CVPR'26 Best Paper)—COLMAP/SfM initializationSfM-free 3DGS — eliminates COLMAP dependency by jointly optimizing camera poses and Gaussian parameters; critical for NeRF methods where custom camera estimation must be preserved in migration
D4RT (CVPR'26 Best Paper)—Static 3DGS + per-frame deformation4D dynamic reconstruction in Gaussian framework — provides the migration path for NeRF methods with temporal/deformation components (D-NeRF, HyperNeRF, etc.)

Step 4: Training Adaptation

Key changes to the training loop:

# 1. Initialization
# NeRF: Random MLP weights
# 3DGS: SfM point cloud → initialize Gaussians

# 2. Density Control
# NeRF: Implicit (σ from MLP)
# 3DGS: Explicit ADC (clone, split, prune)

# 3. Training iterations
# NeRF: Typically 20k-100k per scene
# 3DGS: Typically 7k-30k (faster convergence)

# 4. Learning rates
# 3DGS standard:
#   position: 0.00016 * decay(0.01, step, 30000)
#   opacity: 0.05
#   scaling: 0.005
#   rotation: 0.001
#   SH: 0.0025 (degree 0), 0.000125 (degree 1+)

Output Format

## Migration Plan: [Source Method] → 3DGS

### Method Overview
[Brief summary of the NeRF method]

### Component Mapping
| NeRF Component | 3DGS Equivalent | Complexity |
|---------------|-----------------|------------|
| ... | ... | Low/Med/High |

### Step-by-Step Migration
1. **Step Name**: [Description] + [Code template]

### Potential Issues
1. **Issue**: ... → **Solution**: ...

### Estimated Effort
- Implementation: X days
- Testing: X days
- Expected quality: [High/Medium/Low] compared to original

### Code Skeleton
[Minimal working code structure]

Knowledge Base

This skill references a knowledge base of 1066 methods across 26 categories (updated for v0.9.14 cycle).

Rules

  1. Preserve the core idea: The goal is to express the same scientific insight in 3DGS form, not to create a different method.
  2. Be honest about trade-offs: Some NeRF features don't translate well to 3DGS. Say so.
  3. Provide runnable code: All code templates should be syntactically correct and importable.
  4. Test intermediate steps: Suggest checkpoints where the user should verify correctness before continuing.

Red Lines

The following are categorical prohibitions. Violating any of these invalidates the output:

  • No invented data: Never fabricate migration rules, compatibility data, or method characteristics not in the loaded reference files. If a value is not found, write "data not available" or "N/A".
  • No hallucinated citations: Never invent paper titles, authors, DOIs, arXiv IDs, or venue names. Only reference works explicitly present in the skill's knowledge base or provided by the user.
  • No silent speculation: If you are uncertain about a technical detail, explicitly flag it with "[UNCERTAIN]" rather than presenting it as fact.
  • No method misattribution: Do not assign features, results, or mechanisms from one method to another. Each method's data is specific to that method.
  • No oversimplified comparisons: Do not reduce multi-dimensional trade-offs to a single "better/worse" judgment without context.

Related Skills

  • 3dgs-method-compare — Method comparison (use for comparing NeRF vs 3DGS approaches)
  • 3dgs-paper-reader — Paper analysis (use for understanding NeRF and 3DGS papers)
  • 3dgs-code-reviewer — Code review (use for verifying migration implementation)
  • cad-mesh-3dgs — CAD/Mesh integration (shares SLAT framework for Mesh↔3DGS conversion theory; use for surface extraction post-migration)
  • 3dgs-mcp-renderer — MCP rendering (use for code-first export of migrated 3DGS scenes: export_scene_code partitions procedural geometry vs 3DGS splat based on SLAT encode-decode analysis)
  • SLAT unified representation — See ../../references/slat-unified-representation.md for the shared theoretical framework

Guardrail: Do Not Apply From Memory

Do NOT try to apply the logic, method data, bug patterns, or technical details described in this skill from memory. Always read the SKILL.md and referenced files from disk before producing any output. The knowledge base is updated frequently; stale memory may produce outdated, inaccurate, or fabricated results.

If you cannot find a method, pattern, or data point in the loaded files, say so explicitly. Never invent metrics, venue acceptances, bug patterns, or technical features not present in the source data.

If you like it, please star this repo https://github.com/jaccen/Awesome-Gaussian-Skills

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.