How it is measured

The score

• BD-rate (Bjøntegaard) on each encoder's achieved-rate curve against the zero line's, for the same quality: negative is fewer bits. Curves are interpolated (PCHIP) and only their reliable overlap counts; nothing is extrapolated.
• J is PSNR-YUV, VMAF v1 and MS-SSIM in equal thirds, averaged over the full curve and its lower-rate half, and over content classes. PSNR-YUV and VMAF v1 alone are shown too; MS-SSIM alone only where the run recorded it. VMAF is libvmaf's v1 model, scored on every frame.
• No subjective test was run. x264 and x265 default to psychovisual tuning these metrics cannot judge, so the headline zero line is x264's own metric tuning (--tune zerolatency,psnr), and the default-tuned x264 is shown beside it. Our presets were tuned on this metric mix.
• Our streams are scored on the encoder's reconstruction, the references' on their standard-decoded output. Every build behind these numbers passed the landing conformance check: short streams of every shipped preset decoded bit-exactly by two standard decoders per format and by ours.

Clips and rates

Clips are encoded only at rates at most half their source's bitrate, as many are already compressed. Our savings are largest at these low rates, and so are the references'; over a higher range (up to a few megabits at 720p) every gap shrinks, ours included, on the small subset with room for those rates (rate-control audit).

Settings

• Low delay, real time: every encoder in its own real-time rate control at the same targets, forward prediction only (no B-frames, no lookahead), one key frame, one thread, .
• openh264 runs through our harness (tools/openh264-harness, a small C program over its API, as its own command-line tool cannot set every parameter): Constrained Baseline with CAVLC, medium complexity, camera real-time usage, adaptive quantization off, scene-change detection off, frame skipping off, one thread.
• 8-bit 4:2:0 only. No 10-bit, no other chroma formats.
• Ours do not code repeated frames that would not change the picture (holds); the references code every frame. Holds are rare on these clips.
• The references are mature, general-purpose, multi-threaded encoders for many more uses; we measured only these settings, and their other modes may do better. Full rules (rate validity, chaining, flagged points): chart method.

Machines

Speed is measured at a fixed quantizer near the middle rung on three development clips, which a CBR encode costs our encoders slightly more than; the PC is shared and loaded, so differences of a few percent there are noise.

Builds and versions

Command lines

Each encoder's real-time CBR command at the example rung (the harness writes these; tool paths shortened to program names), and the fixed-QP command its speed was timed with.

Window length

The headline windows are 150 frames with one key frame (5 s at 30 fps), and our presets were tuned on such windows. A short window favours an encoder that invests in its first key frame, as ours do. The same build was therefore also measured over 600 frames with a key frame every 10 s, on held-out sources long enough for it. In brackets, 95% bootstrap intervals.