I Ship a New Data Scraper Every Few Days. Here Is What I Have Learned

A while back I kept hitting the same wall. I wanted some specific slice of public data, a list of local businesses, fresh SEC filings, app store reviews, and the options were always the same. Pay a bloated subscription for a dashboard I did not need, or write yet another scraper from scratch.

So I started building the scrapers anyway, and then I did the obvious thing. I put each one online as a small pay per use API. As I write this there are around 85 of them live, and I ship a new one every few days, so by the time you read this the count is higher.

Here is what the stack looks like and what I learned.

The stack

Every actor is Node with Crawlee and Playwright, deployed on Apify. ESM modules, a single src/main.js entry point, automated checks before every deploy. Apify handles the hosting and the pay per event billing, so a buyer is charged per result instead of a flat monthly fee. You run it, you pay for what comes out.