The code that processes a CSV in your tests looks reasonable. Read the file, parse the rows, aggregate the numbers. It passes review, it ships, and it runs fine for months. Then someone hands it a real export, the process dies with a heap-out-of-memory crash, and the fix is not a bigger server. It is a different approach to reading the file.
This article shows the failure with real output, explains exactly why it happens, and fixes it with streaming so the same job runs in constant memory no matter how large the file gets. Everything here runs on plain Node with a single dependency, the csv-parse package, and every number is captured from an actual run.
The setup
Here is a helper script that generates a CSV of order records. The rows are made up, so there is no real data involved. Nothing about the script matters except that it can produce a file large enough to be realistic:
// make-csv.js
