If you scrape the web at any real scale, a single proxy will not cut it. IPs get rate-limited, temporarily banned, or simply time out. The durable fix is a pool of proxies plus logic that rotates through them and retries failed requests on a fresh IP.
Here is how to build one in Python that does not fall over the first time a proxy misbehaves.
Why one proxy is never enough
A single IP hitting a site hundreds of times per minute looks nothing like a human. Target sites respond with 429 Too Many Requests, throw CAPTCHAs, or quietly start returning empty pages. Even a "good" residential IP has a bad day: the upstream node drops, the TCP connection hangs, DNS flakes.
So two things are non-negotiable in production:
