If you're running agents in production, you already have a training dataset sitting in your traces and most teams never look. Here's the four-stage pipeline for turning that raw telemetry into something you can actually fine-tune on.
Originally published at overmindlab.ai.
Every agentic system can create its own training data.
Every agentic AI system you run is quietly writing its own training set, and most teams never use it.
When an agent runs, it emits a stream of inputs and outputs: the prompt, the tool calls, the reasoning and the final answer. That stream is a trace, and once you standardise how you capture it (the OpenTelemetry GenAI semantic conventions is the default schema here) you have raw telemetry. A training dataset is narrower: a curated set of runs, scored against a spec, designed for fine-tuning jobs.
