The lakehouse community spent this week deciding what belongs in the format and what belongs outside it. Iceberg contributors pushed to retire equality deletes in V4, Parquet voted on a new floating point encoding, and Arrow shipped its 25.0.0 release across every language it supports. Polaris debated the persistence layer that everything else sits on, DataFusion welcomed a new committer alongside a fresh release, and the incubating Ossie project fielded hard questions about maturity and identity. Read together, the dev lists tell a story about a stack that is growing up. The debates are less about whether features exist and more about what guarantees each layer owes the ones above it.
By the numbers, this was a heavy week. Polaris led with 89 messages, Iceberg posted 70, Parquet 65, Ossie 47, Arrow 36, and DataFusion 20. That is 327 messages across the six lists in seven days, spanning release votes, format proposals, persistence design, and governance. Every thread referenced below links to the full conversation on lists.apache.org, so treat this newsletter as a map and go read the primary sources on anything that touches your stack.
Apache Iceberg
The biggest structural conversation on the Iceberg list this week centered on the future of equality deletes. Huaxin Gao revived the long-running proposal to deprecate equality deletes in Iceberg V4, and the thread drew nine messages of substantive engagement. Maximilian Michels gave the streaming perspective that many practitioners will recognize. He called equality deletes the number one pain for streaming use cases and noted that many users give up when they see merge-on-read costs, or they build custom solutions that pull them away from core Iceberg. Michels also shared concrete progress from the Flink side. The index that powers ConvertEqualityDeletes persists in Flink's managed RocksDB state, updates as new data arrives, and checkpoints on a schedule. The conversion works for data written by any engine, but the conversion itself still requires Flink. Storing that index in Iceberg and opening it to all engines is the next step toward an engine-agnostic answer. Michels closed with a +1 for deprecation in V4, and he framed the plan as realistic given the progress since the first conversation in 2024. Watch this one. Removing equality deletes reshapes how every streaming writer targets the format.
