Best way to find out where your scraper is fragile? Break it. On purpose. In a controlled way, in a test environment, with a checklist of failure modes you actively try to inject.
This is chaos engineering for scrapers. Most teams don't do it because they're convinced their scraper "works." Then they discover what doesn't work the hard way, in production, on a Sunday.
I ran the exercise on our image metadata scraper last week. Here's what I broke and what I found.
The 3-item attack list
Three categories of injected failure that catch most fragility:
