Claude Code can read my files and run my shell, but out of the box it can't actually go get a page from the live web in a way that survives modern anti-bot defenses. A curl from the Bash tool gets you a 403 from anything behind Cloudflare. So I took an existing open-source scraper — D4Vinci/Scrapling — installed it locally, and registered its built-in MCP server with Claude Code. Now the agent has ten tools for pulling the real web: plain fetches, headless-browser fetches, stealth fetches that solve Cloudflare, and screenshots. The scraper isn't mine — I want to be clear about that. What I built is the local install and the MCP integration that hands those capabilities to the agent.
What it is
Scrapling is an adaptive web-scraping framework. The part I cared about for this is its fetchers: a fast HTTP client built on curl_cffi that impersonates real browser TLS fingerprints, a Playwright-backed fetcher for pages that need a real browser to render, and a "stealthy" fetcher built on patchright that bypasses anti-bot systems like Cloudflare Turnstile out of the box. It also ships a parser that "learns" a page's structure so your selectors keep working when a site redesigns. That whole engine is the maintainer's work, not mine.
