I spent last weekend trying to build a simple document summarizer. Nothing fancy—just take a 50-page PDF, send it to an AI API, and get back three bullet points. But the API kept timing out. Every. Single. Time.
I went from frustration to a solid solution, and I want to walk you through my thought process, the dead ends, and what actually worked. No fluff, just the trade-offs I wish someone had told me about.
The Problem: Documents That Don't Fit in a Single Prompt
I had a bunch of long legal documents—think contracts, reports, proposals. The AI API I was using (let's call it the endpoint at https://ai.interwestinfo.com/v1/completions) had a context window of about 4,000 tokens. A single contract could easily be 15,000 tokens.
My first naive attempt? Just send the whole thing and hope for the best:
