If you're feeding web content into an LLM — for RAG, for an agent's context window, for a summarization pipeline — raw HTML is the wrong input. This post covers why, and the specific extraction problems you'll hit once you try to fix it yourself.
Why raw HTML wastes your context window (and your accuracy)
A typical webpage's HTML is 70-90% markup, navigation, ads, cookie banners, and scripts that have nothing to do with the actual content. Dump that into an LLM prompt and you're paying token cost for <nav> boilerplate, burning context budget that could hold more actual retrieved content, and — worse — sometimes confusing the model with off-topic sidebar links and "related articles" widgets that get treated as part of the main content.
The fix is converting the page to clean Markdown: headings, paragraphs, lists, and links, with the chrome stripped out. This is a solved problem in principle (Readability-style content extraction has existed for years), but there are a few specific failure modes worth knowing about before you build it yourself.
Problem 1: content-extraction heuristics fail on modern SPA sites
