TL;DR— Multimodal models in production are bottlenecked less by model capability and more by the sampling decisions that turn video, audio, and images into tokens— frame rate, chunk length, resolution and crop. These knobs are usually left at framework defaults, and they silently determine what the model can ever perceive, no matter how strong the underlying model is. Treat sampling as an architecture decision, and evaluate it as a first-class failure mode.
Every multimodal demo uses a clip that's fifteen seconds long, an image that fits in one tile, or an audio file that's a single clean sentence. That's not cherry-picking in the pejorative sense— it's just what fits inside the boundary conditions where the pipeline works without anyone thinking about the pipeline. The moment you put a real video, a real phone call, or a real high-resolution scan through the same system, you hit a wall that has nothing to do with the model's reasoning ability. You hit a wall built entirely out of preprocessing defaults.
The industry talks about multimodal models as if perception and reasoning are the hard parts. In production, the hard part is upstream of both: the sampling step that converts continuous signal— pixels over time, sound pressure over time, pixels over space— into a fixed budget of tokens the model can actually consume. Get that step wrong and it doesn't matter how capable the underlying model is. You've already thrown away the information it needed.
