A frequently encountered issue when exploring database optimizations is proving that a proposed query rewrite is actually faster than the existing query. Often, proving this isn’t just as simple as running both queries and comparing their execution times. When your hot queries are measured in milliseconds, factors such as cache states, concurrent queries competing for CPU and I/O, and noisy neighbors can mask the true performance difference between query versions.
Earlier this year, Datadog announced Bits Database Optimization, a Database Monitoring (DBM) feature that automatically surfaces slow or bottleneck queries and generates a suggested fix. One of the underlying principles behind the feature is to only recommend optimizations that we can prove to be more performant. Several key design choices enable Bits to tackle this challenge of proof using capabilities that distinguish it from other query optimization tools—most notably, its ability to simulate production datasets based on their schema and table and column statistics.
In this blog post, we’ll take a deeper look at how Bits Database Optimization works under the hood and the deciding factors it takes into account when determining whether a query rewrite is actually faster.
