By Michael Nocito, data analyst · Published August 8, 2026
By the end of this page you can tell the difference between a repeated row and a repeated key, count each of them without deleting anything, choose which of two conflicting rows survives, and know why the same table de-duplicates differently in pandas and in a spreadsheet. It is about twenty-five minutes, and every number below came out of running the code.
Here is what to do today, before any drop_duplicates. Run these two lines and compare them:
df.duplicated().sum() # whole rows repeated
df.duplicated(subset=["order_id"]).sum() # keys repeated
