When evaluating agentic performance or security, developers typically focus on the underlying large language model. Benchmarks measure the reasoning capabilities of the LLMs, treating the agent harness as invisible wiring. This approach assumes the harness acts as a neutral substrate, serving merely to connect model reasoning to external tools and APIs.
A study by cybersecurity firm Lasso Security challenges this assumption. Researchers measured how much the agent harness alone influences performance under adversarial conditions. They constructed an autonomous attacker agent, held its model, system prompt, tool set, and targets constant, and varied only the runtime framework underneath it.
The results reveal that the harness is far from neutral. Simply swapping the harness redirects an attack’s strategy, determines whether network guardrails block or pass malicious payloads, and can quietly kill multi-turn execution loops. For engineers, this exposes a critical security blind spot: security is not a property of the model alone, but of the specific model-harness pair.
The test setup and the scoring problem
To isolate the harness from other variables, the Lasso Security team maintained strict control over the experimental environment. The test pitted an open-source framework, LangChain’s deepagents running on LangGraph, against a closed-source vendor tool, Anthropic’s Claude Agent SDK.
