My Local LLM Was Running at 1.6% of Its Context. Here's the Setting That Fixed It

I run a content pipeline on a Mac mini (48GB unified memory) that splits long blog drafts into platform-specific short-form pieces. That job — read a 30-page document, hold the whole thing in mind, extract what matters for YouTube Shorts vs TikTok vs Reels — is exactly what long-context LLMs are supposed to be good at.

Mine wasn't. It kept "forgetting" the second half of every document, dropping key details, and producing shallow summaries no matter how I tuned the prompt.

Three weeks of tuning the wrong thing

I did what you'd do. Simplified the prompt. Rewrote the template. Swapped models. Re-downloaded them, twice. Spent entire evenings after work on this, convinced the model was the problem — a Q4_K_M quantized 13B–20B model should handle long documents, right? The symptoms said otherwise: solid on the first pages, incoherent by the end.

Classic context-window behavior. I just didn't see it yet.