Somewhere in your data platform right now, a single configuration property is quietly deciding a meaningful percentage of your storage bill, your query latency, and your compute spend. It is probably set to whatever the defaults were in 2019, nobody has looked at it since, and it is the compression codec.
Compression is the most consequential invisible decision in data infrastructure. Every Parquet file in your lakehouse, every message crossing your network, every backup in your archive passed through a compressor, and the choice of which one, at which setting, ripples through everything downstream: bytes stored, bytes transferred, requests billed, CPU burned on every read for the life of the data. Yet most engineers' working knowledge of the topic amounts to a vague ranking, gzip is old, Snappy is fast, Zstandard is good, without the mechanics that would let them reason about a new situation.
This article fixes that. We will build the theory from the ground up, in plain language: why data compresses at all, why nothing compresses everything, and the two great families of technique that every modern codec combines. Then the codec lineup itself, gzip, bzip2, LZMA, Snappy, LZ4, Zstandard, Brotli, each with its design center and honest trade-offs, and why one of them effectively won the decade. Then the layer my readers live in: how compression actually works inside the lakehouse stack, Parquet pages, columnar encodings versus codecs, splittability history, hardware acceleration, and the economics on object storage. And finally the practical playbook: what to set, when to deviate, and how to measure. As always in this series, the goal is that the logic clicks, so the next codec announcement or benchmark chart explains itself.
