This week the lakehouse community spent most of its energy on two questions. The first is how the catalog grows from a table directory into a control plane that serves agents, BI tools, and query engines from one place. The second is how the file and table formats hold up when correctness gets tested at the bit level. Apache Polaris pushed hard on the first question with debates about non-Iceberg REST endpoints, runtime-activated datasources, and a semantic layer that lives in the catalog. Apache Iceberg and Apache Parquet pushed on the second with proposals about global snapshot consistency, primary keys, NaN ordering, and INT96 timestamps. Underneath all of it ran a quieter thread that touched several projects at once: the cost of shared continuous integration runners, the maturing of the C++ and Rust implementations, and a fresh round of release votes. Here is what the four core projects and DataFusion worked through over the past seven days.
Apache Iceberg
The most consequential design discussion of the week centered on data correctness across more than one table at a time. Xiening Dai's Global Snapshot Consistency proposal kept drawing replies from Russell Spitzer, Andrei Tserakhau, and Maninder Parmar. The core problem is real and getting harder. Today the Iceberg spec defines isolation at the level of a single write operation through table properties like write.delete.isolation-level. It says nothing about reading several tables as of one consistent point in time. Xiening framed two approaches that the thread treated as complements rather than rivals. One adds a batch LoadTables API that returns the metadata for many tables atomically as of a shared instant. The other introduces a commit sequence number, a monotonic value that lets a reader pin a consistent cut across tables. Maninder pointed out that the batch load path solves the read side, and the sequence number path gives the deeper guarantee. The reason this matters is the shape of modern workloads. An AI agent or a BI dashboard that joins five tables wants those five tables to reflect the same logical moment, not five independent and slightly skewed snapshots. Iceberg was built around single-table atomicity. This thread is the community starting to reason about multi-table atomicity as a first-class concern.
