Reading and reuse guide

Methodology for humans and agents

This site preserves useful measurements without turning unlike workloads into a single leaderboard. The gallery shows footage, the benchmark index exposes sanitized records, and this page explains how to interpret both.

Observed media

A valid MP4 proves that a graph executed and produced decodable media. It does not establish visual quality or relative speed.

Matched comparison

A speed ratio is meaningful only when the complete workload and runtime identity match closely enough to isolate the changed component.

Human review

Transcript, motion, identity, lip synchronization, and artifact review remain separate from timing and media compatibility.

Comparison contract

Match the workload before computing a speedup

If any material field differs, describe the row as directional or compatibility evidence rather than a direct performance comparison.

1Generation mode: FL2VA, Ref2VA, or another graph family
2GPU, runtime fingerprint, model/checkpoint, and workflow revision
3Width, height, duration, frame count, and frame rate
4Step count, sampler, scheduler, sigma shifts, and denoise path
5Attention backend, cache configuration, accelerator, and LoRA stack
6Seed, prompts, references, native-audio contract, and model residency

Evidence levels

What each record can support

1 · Controlled run

Matched inputs, real sampler evidence, exact runtime identity, valid output, and human review. Suitable for a bounded performance conclusion.

2 · Execution record

Real timing and valid media, but one or more comparison fields differ. Useful for compatibility and profiling.

3 · Recovered or partial record

A source preserves only total time, media, or selected fields. Useful as historical evidence with limitations stated.

4 · Cached replay

The graph reused prior node outputs. Useful for cache-behavior diagnosis, never as generation speed evidence.

Public schema

Data dictionary

Field	Meaning
elapsedSeconds	Observed end-to-end or execution time recorded by the source.
samplingSeconds	Sampler-node time when separately measured; null means unavailable.
requested	The intended dimensions, duration, fps, steps, and native-audio setting.
media	Observed output probe: dimensions, duration, frames, codecs, and validity.
likelyCachedReplay	Derived replay warning. Exclude these rows from speed rankings.
samplerWasCached	Explicit source assertion only; null means the source did not say.
workflow	Revision, package, backend, cache, sampler, and scheduler when recorded.
source.file	Relative locator back to the evidence record; no absolute path is exposed.

Agent reuse protocol

Seven rules for automated analysis

Load the JSON file and preserve its schema version.
Exclude rows where likelyCachedReplay is true.
Group by the complete comparison contract, not just backend name.
Use sampling time for kernel changes and total time for user-facing latency.
Keep null fields unknown; never fill them from a neighboring record.
Cite source.file when reporting a measurement.
Pair timing conclusions with transcript, media, and human-review evidence.

Download JSON Download CSV

Example filter

const data = await fetch("./data/benchmark-records.json")
  .then(response => response.json());

const comparable = data.records.filter(record =>
  !record.likelyCachedReplay &&
  record.mode === "fl2va" &&
  record.requested.width === 768 &&
  record.requested.height === 1344 &&
  record.requested.durationSeconds === 15 &&
  record.requested.fps === 24
);

// Also match steps, sampler, scheduler, model,
// references, audio, runtime, and residency.

Privacy boundary

Statistics only

The public dataset contains no source benchmark media, raw prompts, personal-reference records, provider job IDs, or absolute filesystem paths. The gallery media remains a separately curated set.