A model writing German produces 1,234,56. It has correctly learned that German uses a decimal comma and has not learned that German therefore uses something else for the thousands, so it applied one rule and left the other alone. The result is not a foreign-looking number. It is not a number.
The half-right bug
The two separators are a matched pair. A locale that uses a comma to mark the decimal cannot also use a comma to group thousands, because the string would be unparseable — which is exactly what happens when a generator applies only one half of the pair.
It happens because the decimal comma is the salient, frequently-stated fact (“Germans use a comma for decimals”) while the thousands separator is the consequence nobody writes down. A model following an instruction to use European number format has strong evidence for the first and weak evidence for the second, and applies what it has.
Three variants of the resulting output are worth recognising:
