Messy CSV exports are a tax on every data task: stray whitespace, duplicate rows, inconsistent...
Three Python scripts demonstrate CSV cleaning (trim, dedupe, split, merge) using stdlib csv instead of pandas for minimal overhead. Relevant for data pipelines: no install friction, runs anywhere, readable code tweakable for custom rules without dependency bloat.
Messy CSV exports are a tax on every data task: stray whitespace, duplicate rows, inconsistent headers, files too big to open. You don't always need pandas for this — Python's built-in csv module handles most of it with zero dependencies and code you can drop on any machine.
Here are three small patterns I reach for constantly.
1. Clean: dedupe, trim, fix headers
import csv
def clean(path, out):
CSV files are everywhere: analytics exports, sales reports, product usage logs, survey results,...
Most spreadsheet cleanup work is not really an Excel problem. It is an extraction and review...
Why the obvious approach crashes, and how a few generator functions keep memory flat no matter how...
In the first article, I walked through a small Python data quality ETL starter that reads messy CSV,...
Every Lambda function I have written that touches S3 has the same three lines of...
Every other framework in this series leaked through identifiers. Pandas leaks through strings — and...
