A reproducible comparison of CognoDB Cloud, Neo4j AuraDB, Memgraph, FalkorDB and ArangoDB, and the two things that nearly made it lie.
I was asked to benchmark CognoDB Cloud against four other managed graph databases. Same data, same workloads, honest numbers. I figured the hard part would be writing fair queries across five different engines. It wasn't. The hard part was realising that my first clean looking results table was mostly a map of undersea internet cables, and that one of the databases was quietly throwing away my data while telling me everything was fine.
Here's what actually happened.
The number that looked great and meant nothing
My first latency table said FalkorDB was about 2,700 times faster than CognoDB at a single indexed lookup. 0.09 ms versus 239.61 ms. That number is real. I can reproduce it any time I want. It's also completely meaningless.
