How SQL analysts and analytics engineers can now leverage declarative flows for append, CDC, and batch ETL directly in their queries.
by Matt Jones and Shanelle Roman
Databricks is bringing declarative ETL to data warehousing workflows in Lakehouse, making it easier for SQL practitioners to simplify complex transformation logic in familiar places they already work.
This is part of a broader strategy to bring the declarative execution model behind Apache Spark™ Declarative Pipelines into more authoring experiences across Databricks. Instead of needing to work in a dedicated pipelines-oriented environment, SQL users can now define common ETL patterns directly within their SQL queries in Databricks Lakehouse.
Declarative SQL ETL on Databricks is not new. Today, thousands of SQL-first users already rely on declarative primitives like Materialized Views and Streaming Tables to simplify recurring transformations, keep downstream tables up to date, and accelerate BI workloads.
