If you’ve ever wondered where all the data generated during an AI chatbot conversation actually goes, the answer is: it piles up fast, and the storage industry is scrambling to keep pace. SanDisk, freshly spun out from Western Digital as of February 2025, is placing a major bet that one specific piece of AI infrastructure, the KV cache, will account for 35% of all AI data center NAND flash workloads by 2030.
What KV cache actually does, and why it matters
KV cache stands for key-value cache. When an AI model processes your prompt, it generates key-value pairs at each layer of its neural network. Storing those pairs lets the model avoid redundant computation when generating subsequent tokens in a response. Without KV cache, every new word the model produces would require recalculating attention across the entire input sequence from scratch.
This cached data grows linearly with sequence length and session counts. As AI models handle longer context windows and serve more simultaneous users, KV cache storage needs balloon. Traditionally, that data lives in high-bandwidth memory (HBM) or DRAM. SanDisk’s pitch is to offload KV cache to NAND flash storage, enabling the scaling of context windows without a proportional increase in compute or power costs.
