Amazon Bedrock AgentCore Evaluations solves a real problem: you build agents on LangGraph, your teammate uses LlamaIndex, and the platform team is experimenting with the OpenAI Agents SDK. Every framework has its own evaluation story, and none of them talk to each other.
AWS's answer is to treat OpenTelemetry telemetry as the evaluation contract. If your agent emits the right spans and attributes, AgentCore can score it without knowing which framework you used. The service works with LangGraph, LlamaIndex, OpenAI Agents SDK, Google ADK, Claude Agent SDK, and Strands Agents. It also works with custom stacks, as long as you instrument them correctly.
This is the first major cloud vendor to decouple agent evaluation from framework choice using a telemetry-based contract. Here's how the plumbing works.
The OpenTelemetry Contract
AgentCore Evaluations expects agents to emit structured telemetry in OpenTelemetry format. The service looks for specific span types and attributes:
