I benchmarked 5 managed graph databases — and the "obvious" winner changed depending on what I measured

I benchmarked 5 managed graph databases — and the "obvious" winner changed depending on what I measured

The setup

I was asked to benchmark CognoDB Cloud — a new Neo4j-compatible managed graph database — against four other players in the space: Neo4j AuraDB, Memgraph Cloud, FalkorDB Cloud, and ArangoDB Oasis. Same dataset, same logical queries, same client machine, five free-tier cloud instances.

The dataset was SNAP's cit-HepTh citation network: 27,770 physics papers, 352,807 citation edges. Simple schema — (:Paper)-[:CITES]->(:Paper) — nothing fancy, which turned out to be exactly the point. A boring, well-understood graph is a good stress test, because there's nowhere for a slow query to hide behind data complexity.

I measured five things on every platform: how fast data loads in, how fast you can walk 1/2/3 hops from a node, how fast you can look something up by key, how fast you can aggregate across the whole graph, and how the database holds up under concurrent load. Full methodology, raw numbers, and every caveat are in the repo — this post is the story of what surprised me.