Chinese AI darling DeepSeek unveiled an updated version of its cost-and-latency-optimized Flash model on Thursday, with a new version 4.1 that includes architectural improvements more significant than you would expect in a point release because the changes might open the door to larger, smarter, and less resource-intensive models.At 763 billion parameters, the point release is more than 2.5x the size of the model it replaces. In fact, the model is larger than the V3 and R1 models that put DeepSeek on the map back in early 2025.Despite its ginormous parameter count, DeepSeek V4.1 Flash’s memory requirements aren’t nearly as high as you’d expect for a model of its size. Under the hood, DeepSeek's devs have made numerous architectural changes that see the LLM become smarter while dramatically reducing the resources necessary to serve it.
DeepSeek has managed this through two key improvements. First, it made significant changes to how the model handles the key-value (KV) caches used to track model state across multiple sessions. These so-called KV caches can be quite memory-hungry, particularly in high-throughput applications like chatbots.
Updates to the model’s various attention mechanisms and the introduction of a new causal encoder-decoder (CED) enabled the devs to improve prompt processing performance while cutting KV cache consumption to between 13 percent and 25 percent of DeepSeek V4 Flash's requirements.In other words, the V4.1 release can support four to eight times as many users in the same KV cache footprint. DeepSeek’s technical report goes into far greater detail on the architectural changes, but arguably the most interesting change is the introduction of a different kind of model weight.Of its 763 billion parameters, 196 billion are N-gram parameters that form what DeepSeek's developers refer to as a “conditional memory module.” The idea is that by decoupling memory from computation, DeepSeek can make its models smarter while also reducing the compute and memory resources required to serve them.What the heck is an N-gram?The big idea behind DeepSeek’s V4.1 Flash’s memory module is similar in many respects to Per-Layer Embedding (PLE) tech originally developed by Google’s Gemma team. The goal with PLE was to get LLMs to be smart enough to run usefully on devices with constrained bandwidth, memory, and compute – like smartphones.DeepSeek’s implementation, first detailed in a January research paper, trades PLE embeddings for N-grams. At a high level, N-grams are just groups of tokens. A three-gram would be three tokens in a row, a two-gram would be two, and so forth. As complicated as that might sound, it actually works a bit like word or phrase association.
