For a long time, Java engineers have had access to a powerful performance diagnostic tool: Java Flight Recorder (JFR). Built into the runtime, JFR gives engineers almost everything they need to diagnose performance issues—from garbage collection to algorithm optimization. But in some cases, it falls short.
As Datadog’s Java profiler matured, we repeatedly ran into CPU-bound investigations where existing Java Virtual Machine (JVM) sampling produced biased or incomplete results, especially in always-on production workloads at scale. In this context, biased means that samples were not collected in proportion to actual CPU time, making some hotspots appear more or less significant than they really were. As a result, the profiler output didn’t clearly explain where CPU time was actually being spent.
At the center of this is the JFR’s ExecutionSample event, which many profilers rely on. It makes specific decisions about when to capture stacks, which can make CPU-bound hotspots harder to identify. As a result, modern profilers, including ours, combine JFR with agents built on the Java Virtual Machine Tool Interface (JVMTI) and other low-level mechanisms such as AsyncGetCallTrace to produce more accurate results.
