Every few months at work I run into the same problem.
Two systems are supposed to produce identical data exports. A CSV from the old pipeline, a CSV from the new one. Simple enough to check — until the file has 500,000 rows, three sources to compare instead of two, and columns that were renamed somewhere along the way.
The usual tools fall apart fast:
Excel — opens maybe 100k rows before giving up
diff / fc — order-dependent, one mismatch per line, useless for structured data






