Firecrawl is a great tool. I want to say that upfront, because what I'm about to describe isn't really Firecrawl's fault. It's just not what it was built for.
I was using it as the data layer behind an AI agent, and the costs kept climbing for a simple reason: every time my agent needed fresh context, I paid to re-fetch the same sources again. Same pages, same data, paid for again and again, just to get a raw dump I then had to clean up myself before my agent could even use it.
It felt wasteful. Not because the tool was bad, but because I was using a scraper to do a knowledge base's job.
The Difference That Was Costing Me Money
A scraper gives you data right now: raw, unstructured, one-time. A knowledge base gives you data that's always current, structured, and queryable.
