TL;DR — Multimodal models look like a single system in the demo but are actually a pipeline of independently-trained encoders bolted onto a shared decoder. Each encoder carries its own training distribution, and in production the whole system is bottlenecked by whichever modality has the narrowest one — usually vision or audio, not text. Aggregate accuracy hides this because failures are confident and silent, not error-shaped.
Ask a multimodal model to describe a stock photo of a golden retriever on a beach and it nails it every time. Ask the same model to read a cropped screenshot of a spreadsheet, or transcribe a phone call with crosstalk, or tell you what happened between frame 400 and frame 500 of a security video, and the failure modes get strange fast. Not "I don't know" strange — confidently wrong strange. The model doesn't hedge. It just describes something plausible that isn't there.
The instinct is to blame the model. The more useful frame is architectural: a multimodal model isn't one model. It's a set of independently-trained encoders — one per modality — projected into a shared embedding space and handed off to a decoder that was mostly trained on text. Each encoder has its own training distribution, its own capacity, its own blind spots. The fused system inherits all of them, and in production, quality is bottlenecked by whichever encoder has the narrowest distribution. That's almost never the text side.
