How prompt caching actually works When an LLM processes your input, it doesn't just read...
How prompt caching actually works
When an LLM processes your input, it doesn't just read and forget. For tokens that appear in the same position across multiple requests, the model can reuse its previous computation. This is called prefix caching.
Request 1: [System Prompt] [Conversation Turn 1] [Turn 2]
└── 260K tokens computed from scratch ──┘
Cost: expensive
Learn how LLM tokenization works, why it drives cost and latency, and practical ways to reduce token usage in your AI apps with…
Learn how prefill and decode phases affect LLM app speed, what drives TTFT and inter-token latency, and which optimizations fix…
The problem isn't your prompts If you're running Claude Code, Codex, opencode, or openclaw and the...
I built an LLM caching library to test what AI-assisted development actually looks...
Serving long prompts doesn't have to mean slow responses. Learn how Together AI's CPD architecture separates warm and cold…
Most production AI features are still glueing prompts together with string concatenation in some random Node service. The prompt…
Cache-aware prefill–decode disaggregation (CPD) for up to 40% faster long-context LLM serving
Minions: embracing small LMs, shifting compute on-device, and cutting cloud costs in the process
LLM agent memory at 0.12% of model parameters
Subquadratic — Efficiency is Intelligence
Prompting just split into 4 different skills. You're probably practicing 1 of them (+ 7 prompts and a pre-flight to close the gap)
A guide to prompting Llama 2 – Replicate blog
