03:00:0050% OFFClaim

Technical recipe

Extract a video frame by time base: CFR, VFR and exact index checks

A runnable Python and FFmpeg frame-extraction recipe with original CFR/VFR fixtures, rational PTS tables, three selection rules, PNG pixel hashes and 16 executed checks.

Select an actual decoded frame using an explicit at, after or zero-based frame-index policy, then verify its timestamp and saved PNG pixels. Arbitrary requested seconds do not necessarily name an existing frame.

Download the full brief
Original VFR frame-selection timing diagram
Original timing diagram and tiny synthetic RGB fixtures, with executed PNGs, framehash logs and a 16-scenario receipt. This is a local FFmpeg test, not a generated-model result.

01 · Environment and inputs

  • Download the complete kit and unzip it into a controlled local directory. The provided videos are original synthetic timing fixtures, not H3 results.

  • Prepare Python 3 and an installed FFmpeg binary. The recorded run used Python 3.12.3 and FFmpeg 6.0 on macOS arm64; no ffprobe or additional Python package is required.

  • Set FFMPEG_BIN if ffmpeg is not on PATH. Choose a fresh output directory; the fixture runner and extractor refuse overwrite.

  • For your own clip, decide whether time is relative to its first decoded PTS and whether the output should follow display rotation or retain raw orientation. This script uses first-PTS-relative seconds.

02 · Run and inspect

  1. 01 · Build the decoded timestamp inventory

    Run the fixture suite before touching a production clip. CFR has ten frames at 0.00, 0.10 … 0.90 s; VFR has eight at 0.00, 0.04, 0.12, 0.28, 0.32, 0.60, 0.64 and 0.92 s. Both measured time bases are 1/1000. The CSV and inventory JSON tie every zero-based decoded index to its integer PTS; seconds equal PTS multiplied by the time base. Do not infer VFR positions from average FPS.

    export FFMPEG_BIN=/YOUR/INSTALLED/ffmpeg
    python3 run-fixtures.py --out-dir ./my-run
  2. 02 · Choose at, after or an exact decoded frame index

    at selects the latest frame start at or before the requested relative time; after selects the first at or after it. For VFR target 0.20 s, at returns index 2 at 0.12 s, while after returns index 3 at 0.28 s. --rule frame --index 3 also returns 0.28 s; an exact index is not an exact arbitrary timestamp. The script decodes from the beginning and rejects a time outside the first-to-last PTS range rather than inventing the final frame duration.

    python3 extract-frame.py --input executed/vfr.mkv --rule at --time 0.20 --out ./at.png
    python3 extract-frame.py --input executed/vfr.mkv --rule after --time 0.20 --out ./after.png
    python3 extract-frame.py --input executed/vfr.mkv --rule frame --index 3 --out ./index3.png
  3. 03 · Verify the original PTS and the saved pixels separately

    Read the PNG selection receipt and inventory sidecar, not the filename. The source framehash is generated with preserved timestamps, passthrough frame scheduling and an explicit time base; its #tb header defines the integer PTS units. A second decode verifies that the PNG RGB hash matches the selected source frame. The framehash packet-duration column is not used to infer VFR display holds. File SHA-256 identifies bytes; decoded RGB SHA-256 identifies pixels under this conversion and orientation.

    python3 -m json.tool at.selection.json
    python3 -m json.tool at.inventory.json
  4. 04 · Choose rotation handling and keep failed checks visible

    Display extraction follows default autorotation; raw extraction applies -noautorotate before input. The executed 90-degree metadata fixture produced 180×320 display pixels and 320×180 raw pixels. A nonzero 2-second source start was also tested using relative times. These checks do not cover every display matrix, mirrored source or damaged timestamp stream. Keep the selected policy in the receipt and stop on an ambiguous inventory instead of forcing CFR or interpolation.

    python3 extract-frame.py --input executed/rotation-90.mp4 --rule frame --index 0 --orientation display --out ./display.png
    python3 extract-frame.py --input executed/rotation-90.mp4 --rule frame --index 0 --orientation raw --out ./raw.png

03 · Validate the output

  • The actual downloaded suite passes its 16 named scenarios and records the running Python and FFmpeg versions plus the exact SHA-256 hashes of both executed scripts.
  • The CFR/VFR table records decoded index, integer PTS and rational time base, including every VFR gap.
  • At 0.20 s the VFR at/after outputs have the distinct verified indices and PTS stated in the table; no synthetic 0.20-s frame is substituted.
  • The saved PNG decodes to the same RGB hash as the selected source frame under the chosen orientation.
  • Negative time, time beyond the last PTS and an out-of-range index stop with an error.
  • Rotation dimensions are checked against the explicit display/raw choice; nonzero stream start uses first-PTS-relative time.
  • The receipt makes no claim that model quality, all codecs, all display transforms or every real input timestamp has been validated.

Read the actual execution receipt

04 · Boundaries and troubleshooting

The frame looks earlier or later than the requested seconds.
Check the selected rule and the actual PTS table. A request between two VFR timestamps legitimately selects different frames for at and after; do not change the reported timestamp to match the request.
A saved PNG is sideways or its dimensions are exchanged.
Inspect the desired display orientation. Compare --orientation display with raw; PNG pixels follow the selected policy, while video rotation side data is not assumed to remain a useful PNG viewing instruction.
The extractor rejects the last part of the video.
The bounded policy ends at the last known PTS and does not invent its display duration. Select an explicit last-frame index or establish that duration separately before extending the policy.
Frame indices disagree with a timestamp derived from FPS.
Use the decoded presentation inventory and the rational time base. Average FPS, packets and keyframes are different identities, especially for VFR.
The hash differs on another FFmpeg installation.
Compare decoded RGB hashes under the same pixel format and orientation before comparing encoded PNG bytes. Record the new tool version; the supplied receipt proves the recorded fixture run only.

Keep the PNG, its selection receipt and the source frame table together. The selected rule, actual timestamp and pixel check make the extraction reviewable; they do not manufacture a frame at a timestamp the source never contained.

Files and evidence

Explore production checks and handoffs