I spent a night trying to fit a dense 30B model, 256K context, vision, and speculative decoding onto...
I spent a night trying to fit a dense 30B model, 256K context, vision,
and speculative decoding onto one 24 GB GPU. The fastest quant lost. The
quant with the lowest perplexity lost too. What won was the
configuration that made the whole system useful, not any single number
impressive.
DFlash uses block diffusion drafting and KV injection to deliver up to 15x faster LLM inference on NVIDIA Blackwell
A 70B model generates one token per forward pass, and each pass reloads weights from VRAM, computes...
Qwen 3.6 35B-A3B for Local AI in 2026: The 24GB VRAM Line That Gets You 120 tok/s
Deploy Qwen3.6-35B-A3B-NVFP4 on vLLM: Hopper/Blackwell requirements, serve commands, DGX Spark flags, MTP setup, and what breaks…
Qwen 35B on an RTX 4070 left me 600 MiB of headroom and 4096 tokens of context. Two llama.cpp flags flipped that to 32,768 tokens…
How Flash Attention eliminates the HBM bottleneck in attention by tiling Q, K, V into SRAM blocks — IO complexity, v1→v2→v3…
