SplatBench

Public Gaussian-Splat encoder leaderboard. 28 scenes — 14 real (Mip-NeRF 360) and 14 deterministic synthetic probes for the failure modes that matter (texture, transparency, motion, depth, banding, foliage, specular, lowlight, portrait).

Every result is reproducible from the published corpus IDs. BLAKE3 hashes are part of the scene record. Fidelity is mean CIE ΔE94 + ML-Score over 8 deterministic orbit frames rendered through @catetus/viewer.

splatbench-v0.json Submit your encoder Run it yourself Continuous bench → Methodology

Canonical-11 leaderboard

Standard 11-scene corpus used by the Inria 3D Gaussian Splatting paper (Mip-NeRF 360 + Tanks-and-Temples + Deep Blending). Inputs are the official 30k-iteration pretrained PLYs; encoder is wmv-vq45-no-prune-tight; renderer is gsplat (PyTorch, SH degree 3) at 512x512, 8 frames per scene along an orbit camera path.

11 / 11
scenes measured
19.8×
mean compression
47.45 dB
mean PSNR
0.9991
mean SSIM
Scene Splats Input Output Ratio PSNR (dB) SSIM
bicycle 6,131,954 1520.7 MB 69.50 MB 21.9× 49.87 0.9998
truck 2,541,226 630.2 MB 29.84 MB 21.1× 47.78 0.9996
garden 5,834,784 1447.0 MB 69.70 MB 20.8× 48.38 0.9997
stump 4,961,797 1230.5 MB 60.21 MB 20.4× 48.85 0.9998
room 1,593,376 395.2 MB 19.83 MB 19.9× 50.59 0.9996
kitchen 1,852,335 459.4 MB 23.24 MB 19.8× 48.30 0.9995
drjohnson 3,405,153 844.5 MB 43.06 MB 19.6× 47.24 0.9988
playroom 2,546,116 631.4 MB 32.24 MB 19.6× 43.46 0.9973
train 1,026,508 254.6 MB 13.36 MB 19.1× 45.51 0.9986
counter 1,222,956 303.3 MB 16.30 MB 18.6× 44.93 0.9987
bonsai 1,244,819 308.7 MB 18.65 MB 16.6× 47.07 0.9991

Machine-readable: canonical-11.json · input PLY manifest with MD5s: MANIFEST.md.

SplatBench · Leaderboard

28 scenes — 14 real (Mip-NeRF 360 + Dany Bittel cluster_fly LOD ladder, all CC-BY-4.0, measured via the v2 ingest harness in apps/bench-ingest) and 14 deterministic synthetic probes for per-class failure modes (texture, transparency, motion, depth, banding, foliage, specular, lowlight, portrait, …) — run through the full analyze → optimize → pack pipeline at each preset. Compression is measured against the raw .ply; fidelity is mean CIE ΔE94 over 8 deterministic orbit frames rendered through @catetus/viewer, scaled 0–1 against the published 3% perceptibility threshold. Every scene is identified by a BLAKE3 hash of its canonical IR so runs reproduce byte-for-byte.

QAT-PLY (Scaffold-GS lossless): 37.3% aggregate PLY compression with PSNR gain on every scene tested (6/6). Per-scene save range 33.8–40.5%, ΔPSNR range +0.032 dB to +0.581 dB (mean +0.172 dB). All six scenes strictly improve on PSNR, SSIM, and LPIPS — see the new column below.

Scenes
28 (14 real + 14 synthetic)
web-mobile median
22.68×
size-min median
25.03×
Best ratio
59.64× (size-min)
Corpus total
24.71× across 28 scenes
Only the two shipping SPZ presets are on the leaderboard — the CLI exposes ~40 more. full preset list ↗ Source

pro — splat-aware perceptual score (catches floater + edge-loss artifacts ΔE94 misses). premium — differentiable repack PSNR delta vs naive opacity-prune at the same byte budget. Negative cells (translucent volumes) are a known failure mode where opacity-prune is structurally near-optimal; we publish them honestly rather than tune them away. Values are published; both columns require catetus-pro to reproduce.

Scene Class Source Splats Input SPZ out Ratio splat-transform vs QAT-PLY scaffold-gs Fidelity (ΔE94) ML Score pro Repack ΔPSNR premium analyze
cluster_fly_xxl indoor-close-up REAL 3,506,799 789.3 MB 25.3 MB 31.21× 15.79× +98% — pending — — 0 ms
splatbench_banding_proxy smooth-color-gradient synth 10,000 2.4 MB 91.1 KB 26.60× 5.23× +409% — 0.03% 95.17% +12.62 dB 29 ms
cluster_fly_xl indoor-close-up REAL 624,180 140.5 MB 5.4 MB 25.85× 12.77× +102% — pending — — 0 ms
splatbench_floater_proxy noisy-capture synth 250,000 59.1 MB 2.3 MB 25.84× 10.69× +142% — 0.07% 92.66% -2.23 dB 248 ms
splatbench_texture_proxy high-frequency-texture synth 15,000 3.5 MB 142.6 KB 25.48× 6.47× +294% — 0.09% 94.26% +8.27 dB 32 ms
bicycle_mipnerf360_iter7k outdoor-scene REAL 3,616,103 855.3 MB 33.6 MB 25.46× — — 2.86% 0.00% +5.80 dB 4.46 s
cluster_fly_l indoor-close-up REAL 301,958 68.0 MB 2.7 MB 24.87× 10.37× +140% — pending — — 0 ms
splatbench_depth_proxy depth-bimodal synth 15,000 3.5 MB 149.6 KB 24.30× 6.88× +253% — 0.02% 95.34% +21.91 dB 33 ms
cluster_fly_m indoor-close-up REAL 145,617 32.8 MB 1.4 MB 23.93× 7.49× +219% — pending — — 0 ms
stump_mipnerf360_iter7k outdoor-scene REAL 3,807,536 900.5 MB 39.2 MB 22.97× 18.40× +25% — pending — — 3.93 s
bonsai_mipnerf360_iter7k indoor-real-estate REAL 1,157,141 273.7 MB 12.0 MB 22.81× 16.53× +38% — 0.60% 46.51% +7.43 dB 1.24 s
splatbench_portrait_proxy salient-region-portrait synth 12,000 2.8 MB 129.0 KB 22.55× 5.77× +291% — 0.04% 95.12% +24.04 dB 15 ms
splatbench_specular_proxy specular-highlights synth 12,000 2.8 MB 132.3 KB 21.97× 5.82× +277% — 0.04% 95.14% +15.41 dB 16 ms
splatbench_motion_proxy anisotropic-streak synth 12,000 2.8 MB 133.5 KB 21.79× 5.71× +282% — 0.15% 91.49% +10.51 dB 30 ms
splatbench_dense_proxy dense-large-scene synth 2,000,000 473.0 MB 21.7 MB 21.75× — — 0.00% 95.54% +6.99 dB 2.18 s
cluster_fly_s indoor-close-up REAL 25,627 5.8 MB 277.0 KB 21.33× 6.06× +252% — pending — — 0 ms
splatbench_indoor_proxy indoor-real-estate synth 100,000 23.7 MB 1.1 MB 20.58× 5.91× +248% — 0.03% 95.12% +6.58 dB 105 ms
splatbench_product_proxy product-scan synth 10,000 2.4 MB 118.2 KB 20.50× 4.96× +313% — 0.02% 95.28% +14.69 dB 11 ms
splatbench_lowlight_proxy low-light-dynamic-range synth 8,000 1.9 MB 96.5 KB 20.10× 6.56× +206% — 0.03% 95.27% +13.97 dB 11 ms
splatbench_outdoor_proxy outdoor-scene synth 500,000 118.3 MB 6.0 MB 19.67× 13.88× +42% — 0.02% 95.16% +5.60 dB 483 ms
splatbench_transparency_proxy volumetric-translucency synth 25,000 5.9 MB 337.2 KB 17.96× 5.39× +233% — 0.07% 94.69% -8.30 dB 51 ms
splatbench_foliage_proxy dense-translucency synth 30,000 7.1 MB 412.3 KB 17.63× 6.04× +192% — 0.09% 93.96% +10.35 dB 34 ms
bonsai_scaffold_gs indoor-real-estate REAL 411,066 123.9 MB — — — 59.5% +0.581 dB — — — 0 ms
bicycle_scaffold_gs outdoor-scene REAL 912,445 275.0 MB — — — 59.5% +0.087 dB — — — 0 ms
garden_scaffold_gs outdoor-scene REAL 745,121 224.6 MB — — — 59.5% +0.198 dB — — — 0 ms
stump_scaffold_gs outdoor-scene REAL 746,258 202.1 MB — — — 66.2% +0.076 dB — — — 0 ms
treehill_scaffold_gs outdoor-scene REAL 720,712 195.2 MB — — — 66.2% +0.056 dB — — — 0 ms
flowers_scaffold_gs outdoor-scene REAL 705,051 191.0 MB — — — 66.2% +0.032 dB — — — 0 ms

Comparison columns (in progress)

We are running competing encoders against the same SplatBench corpus and will publish per-scene compression × fidelity numbers for each. Nothing is excluded from the table — losses are reported in the same place as wins. If you maintain an encoder and want a column, send a PR or open an issue with reproduction instructions.

Encoder Maintainer License Status Notes
qat-scaffold-gs Catetus Apache-2.0 shipped Scaffold-GS-PLY QAT codec. 37.3% aggregate PLY save with PSNR gain on every scene tested (6/6); ΔPSNR range +0.032 dB to +0.581 dB.
qat-3dgs Catetus Apache-2.0 shipped Vanilla Inria 3DGS single-PLY codec. 4.84% lossless save on bonsai mipnerf360 iter-7k (287 MB -> 273 MB); bit-exact round-trip; 0 dB delta by construction. Strips nx/ny/nz (always zero in Inria PLYs) from the body. Full int8-SH retrain tier ships as qat-3dgs-bundle below.
qat-3dgs-bundle Catetus Apache-2.0 shipped Vanilla Inria 3DGS premium tier. int8 SH (f_rest_*) quant-aware retrain on A100 (5000 iters, per-channel scales, STE). Bonsai N=1 smoke: 59.27% PLY save / +0.81 dB PSNR / 0.933 SSIM / 7 min A100. Multi-scene generalization in follow-up.
splat-transform PlayCanvas / Snap MIT shipped v2.1.1 median 6.56× across 21 synthetic scenes (Catetus web-mobile median: 22.68×).
SOG PlayCanvas MIT pending Self-Organizing Gaussians (20× compression claim). We will reproduce on bonsai + bicycle.
SOGS Morgenstern et al. research pending 2D grid + image-codec spatial reorder. Reference impl is Python; we will benchmark via Docker.
CodecGS-Lite research research pending PLAS-sort + AV1/HEVC parameter texture. 146× headline on bicycle in the paper; reproduction in progress.

Submit your encoder

We want SplatBench to be the canonical reference. If you ship a Gaussian-Splat encoder, get on the table. There are three ways to submit:

  1. PR a runner. Open a PR against benches/encoders/ with a shell script that takes input.ply and outputs output.{spz,glb,custom}. Our harness handles the rest (decode, render, score).
  2. Send raw numbers. Email bench@catetus.com with per-scene compression + fidelity numbers and we'll add a footnote-cited column while we work on first-class reproduction.
  3. Use the API. The Catetus Cloud API exposes the same pipeline. Free-tier API keys for benchmark runs are available on request.

Submission rules: