Code: Megapixel99/video-timeline
The usual way to hand a video to a language model is to pull a handful of screenshots and let it reason over them. The trouble isn't in reading any one frame; it's in reading the gaps between them. A screenshot doesn't say when it happened, how long after the last one, or what moved in between. So the model fills that in with a plausible story. VTL replaces those screenshots with a measured timeline. The first question worth asking is whether it actually sees more of a video than evenly spaced sampling does at the same cost.
It does, and the gap is large. Both methods get the same frame budget, run over five real videos: eighteen minutes of a narrated slide deck, two screen recordings, a scroll capture, and handheld phone footage. Then you count what each one misses. Across 28 shots, uniform sampling never looks at 9 of them. That is a third of the video, gone. It also spends 12 of its 40 frames re-photographing pictures that differ from the previous frame taken by under 7%. VTL misses no shot and wastes 2 frames. You can check this yourself. python3 tests/benchmark.py runs the comparison on generated fixtures, or on your own files if you pass them.
The mechanism is simple enough to disagree with. Uniform sampling puts a frame every T seconds, regardless of what is on screen. VTL puts frames where the picture changes, drops the near-duplicates, and enforces a floor. No shot goes without at least one frame, and no stretch longer than the coverage limit goes unobserved. The budget uniform sampling spends re-photographing a motionless slide, VTL spends on the shots uniform never reached.
