For years, the AI hardware race has centered on one bottleneck: memory. Specifically, how fast you can feed data to GPUs that are increasingly starving for it. SanDisk, operating under the Western Digital umbrella, thinks the answer isn’t just faster DRAM. It’s flash memory that behaves like DRAM.
The company’s High Bandwidth Flash technology delivered 12.8 TB/s of memory bandwidth with 4 TB of capacity per GPU in simulation, a combination that would allow complex AI models to run on as few as one to four GPUs instead of entire racks of hardware. If those numbers hold up in production silicon, HBF could fundamentally change how companies deploy AI inference at scale.
What HBF actually does
High Bandwidth Memory, or HBM, is the current gold standard for feeding data-hungry AI accelerators. It’s fast, it sits close to the processor, and it’s expensive. The problem is capacity. Even the latest HBM stacks max out at relatively modest capacities, which means running large language models like Meta’s Llama 3.1 405B requires distributing the workload across many GPUs.
SanDisk’s HBF takes a different approach. Instead of stacking DRAM dies, it stacks NAND flash, the same type of memory in your phone and SSD, but engineered to deliver bandwidth that competes with HBM. Each GPU gets eight 512 GB stacks, totaling 4 TB of on-package memory. That’s 8 to 16 times the capacity of HBM at comparable cost, power consumption, and physical footprint.
