WISP — Stream What Shouldn't Run Last week JustVugg dropped Colibrì — a 2,400-line pure C...
WISP streams 744B models on consumer hardware with self-tuning disk cache reaching 85-92% hit rates automatically. CTO and AI teams can deploy foundation models locally without cloud APIs—reducing inference costs, avoiding vendor lock-in, and ensuring data residency.
WISP — Stream What Shouldn't Run
Last week JustVugg dropped Colibrì — a 2,400-line pure C
engine that proved a 744B parameter model could run on 25GB
of consumer RAM by streaming expert weights from disk.
It blew my mind. So I built on top of that concept.
AI developer runs 28.9-million-parameter model on $10 ESP32-S3 microcontroller — uses Google's Per-Layer Embeddings technique, stores table on 16MB Flash memory
AI enthusiast adds Nvidia Tesla V100 as loud as a lawnmower to gaming PC for $266 — 32GB of VRAM rig can run 27 billion parameter model at 32 tokens per second
It's slow, yes, but it's a start.
Build-in-public. This is the real architecture behind running ~50 local AI agents on 6GB of VRAM —...
Most people trying to run Mixtral or DeepSeek-V3 locally hit the same wall: they don't have 80GB of...
Building an AI Runtime Operating System for Commodity Hardware For the last few months...
NVIDIA Introduces SANA-WM: A 2.6B-Parameter Open-Source World Model That Generates Minute-Scale 720p Video on a Single GPU
Your AI agent just tried to run that 7B model you've been building. Error: "No GPU available." You...
