"It works fine on my Mac, but it crashes the moment we hand it to Windows" is a report a lot of developers eventually hear. More often than not, the culprit is a character encoding mismatch. This post walks through the basics of text encoding, then looks at a real build-pipeline crash this project ran into, and the test written to make sure it never happens again.
What an Encoding Actually Is
Note: a character encoding is the lookup table a computer uses to convert characters into bytes for storage or transmission. The same character can turn into a completely different byte sequence depending on which encoding is used.
When a person sees the letter "A" or a Japanese character like "あ", the computer underneath is storing that as some sequence of bytes. Which byte sequence gets produced depends entirely on the encoding in use.
"あ".encode("utf-8") # b'\xe3\x81\x82' (3 bytes)
