Up front: this is a personal/research tool for downloading from public Instagram profiles. Use it responsibly and within Instagram's Terms of Service and your local laws. This post is about the engineering — specifically, what it takes to make browser automation behave less like a bot — not about evading anyone.
Scraping any modern social platform is less a parsing problem and more a behavioral one. The HTML is the easy part. The hard part is that the site is actively watching how you act, and the moment you act like a script — instant scroll to the bottom, requests at machine speed, no pauses — you hit a challenge page and you're done.
I built InstagramWrapperPostScraper as a Python + Selenium tool that drives a real Microsoft Edge browser to download photos, videos, and captions from public profiles. The interesting engineering isn't "how do I find the image URL" — it's "how do I make a browser automation script move through a page the way a person would." MIT licensed, Python 3.10+.
Why a real browser instead of an API or HTTP
There are three broad ways to pull data off Instagram, and they fail differently:
