A table states its values. A chart encodes them — as a length, an angle, a position along an axis, a colour — and reading it means inverting that encoding from pixels. Models are good at describing charts and much less good at inverting them, and the gap is predictable from which encoding the chart used.
Why a chart is harder than a table
Recovering a value from a bar chart requires locating the top of the bar, locating two labelled ticks, and interpolating linearly between them. That is a measurement over a patch grid whose spatial precision is bounded by the patch size — a 14-pixel patch on a downscaled chart can easily span several percent of the plot height. There is nothing in the architecture that measures; attention pools, it does not rule off distances.
So the model does what it does everywhere else: produces a plausible continuation. It will round to a pretty number. It will report the value the axis label suggests. It will get the ordering of the bars right — which is genuinely easy from a gist representation — and the magnitudes approximately, and it will state both with the same confidence.
Failure by encoding
