A lot of LLM workflows start with the same small problem: the model needs fresh data from a web page. Then the integration grows sideways. You add a scraper, a queue, a dataset store, polling logic, retries, and a parser. By the end, the code that moves data around is larger than the code that uses the data.
This is not because scraping platforms are bad. It is because they solve a broader problem than many LLM apps actually have.
The abstraction mismatch
Platforms like Apify are built around actors: reusable scraping or automation jobs with inputs, runs, logs, datasets, scheduling, and platform-managed execution. That model makes sense when you run recurring jobs across many targets, chain multiple scraping tasks, or need shared actors across a team.
For example, a batch pipeline might look like this:
