When we started building more production services in Rust at Datadog, tracing was the part of our observability stack that needed to evolve alongside it the most. Different teams instrumented services with different libraries, context propagation varied depending on which crates were in the request path, and sampling decisions often depended more on local configuration than on end-to-end behavior.
These inconsistencies had a real cost during incidents. Engineers investigating a latency spike or error spike would pull up a trace only to find it truncated at a Rust service boundary—context lost, a new trace started, the downstream path invisible. Manually stitching together what happened from incomplete traces added time and uncertainty to every investigation. Over time, it became clear that solving this well meant investing in shared infrastructure. We needed a shared approach to Rust observability that worked across teams and held up under production load.
In this post, the Datadog APM team describes how we arrived at that approach by contributing upstream to OpenTelemetry and building an opinionated Rust tracer—dd-trace-rs—on top of it. We’ll walk through the design constraints we encountered, the trade-offs we made around propagation and sampling, and what we learned from operating Rust services with tracing in production.
