VMAF Isn't the Whole Story: Choosing the Right Quality Metric for Encoding Decisions
VMAF, SSIM, PSNR, and butteraugli each measure different failure modes. Here's how to combine them into a QA gate that catches what single metrics miss.
Every encoding pipeline needs a quality gate — something that says “this output is good enough to ship” before it reaches viewers. The default choice is VMAF, Netflix’s perceptual metric, and it’s genuinely good. But teams that gate on VMAF alone get surprised: banding passes, temporal artifacts pass, and occasionally a blatantly broken encode scores 95.
Quality metrics are instruments with blind spots. Here’s what each one actually sees.
What Each Metric Measures
Metric │ Perceptual? │ Temporal? │ Blind Spots
──────────────┼─────────────┼───────────┼──────────────────────────────
PSNR │ ✗ │ ✗ │ Everything perceptual
SSIM │ partial │ ✗ │ Noise, blur, film grain
VMAF │ ✓ │ ✗* │ Banding, chroma artifacts
VMAF-motion │ ✓ │ partial │ Flicker between frames
butteraugli │ ✓ │ ✗ │ Video-specific artifacts
*VMAF computes per-frame then aggregates — it doesn’t see what happens between frames.
Where VMAF Lies to You
Three failure modes we’ve caught in production that scored 90+ VMAF:
- Static banding — smooth gradients posterize into steps; VMAF’s detail-loss models don’t flag flat-region discontinuities
- Temporal pumping — encoder breathing artifacts where quantization oscillates frame-to-frame; invisible to per-frame scoring
- Chroma subsampling artifacts — 4:2:0 smearing on saturated red edges (think neon signage, lipstick on skin); VMAF weights luma heavily
The Composite QA Gate We Run
# Pseudocode of the gate
def passes_qa(reference, encoded):
vmaf = compute_vmaf(reference, encoded) # baseline perceptual
ssim_low = min_frame_ssim(reference, encoded) # catch worst-frame
banding = detect_banding(encoded) # flat-region analysis
return (vmaf >= 92
and ssim_low >= 0.94
and banding.score < threshold)
The crucial addition is per-frame minimums, not aggregates. A 10-minute video averaging VMAF 96 can hide a 3-second burst of VMAF 60 — which is exactly the moment viewers notice. Score every frame, gate on the 1st percentile.
A Pragmatic Recommendation
QA pipeline:
reference ──┐
├──► VMAF (aggregate + p1 min) ──► must pass
encoded ────┼──► worst-frame SSIM ──► must pass
└──► banding detector ──► must pass
flagged? └──► human spot-check on shot changes
Our metric-tuning notes and detector implementations are at EncodeLab Media - Video Codec Benchmarks & Encoding Pipelines.
TL;DR
- VMAF ≥ 92 is a good shipping gate — necessary but not sufficient
- Add a per-frame floor metric (SSIM p1) to catch momentary collapses
- Add a banding detector for gradient-heavy content
- Keep a human-in-the-loop on the 5% of outputs nearest the threshold — that’s where all the interesting failures live