My benchmark's Python column was N/A for a year — CPython's 4300-digit limit, and eight other bugs

Submission for DEV's Summer Bug Smash — Clear the Lineup track. The...

mercoledì 15 luglio 2026 New tab

820 words~4 min read

Submission for DEV's Summer Bug Smash — Clear the Lineup track.

The codebase

a2a-benchmark is my multi-language A2A (Agent-to-Agent) performance suite: four agents — Python and Go behind Gemini tool-calling via ADK, Node.js and Rust as direct handlers — each compute Mersenne primes with the Lucas–Lehmer test, while a harness sweeps N=1–24 and charts calculation time and round-trip time.

The committed results stopped at N=22. I never questioned that. I should have.

The headline bug: a whole column of data didn't exist

My benchmark's Python column was N/A for a year — CPython's 4300-digit limit, and eight other bugs

My benchmark's Python column was N/A for a year — CPython's 4300-digit limit, and eight other bugs

Related reading

Why did my benchmark stop at N=22? A debugging story in nine bugs

Revisiting Benchmarking- Building a Rust A2A Agent

The smolagents bug that made my agent retry the same valid code three times

I fixed a smolagents bug that confused everyone who hit it (with Sentry…

I Let 58 AI Agents Review Each Other's Code 561 Times — Here's What Happened

I Built What I Benchmark. Here's How I Try Not to Cheat.

Related reading

Why did my benchmark stop at N=22? A debugging story in nine bugs

Revisiting Benchmarking- Building a Rust A2A Agent

The smolagents bug that made my agent retry the same valid code three times

I fixed a smolagents bug that confused everyone who hit it (with Sentry…

I Let 58 AI Agents Review Each Other's Code 561 Times — Here's What Happened

I Built What I Benchmark. Here's How I Try Not to Cheat.