What: The OpenAI and Broadcom Jalapeño announcement (June 24, 2026) is OpenAI's first custom LLM-inference ASIC — a reticle-sized compute chiplet paired with HBM, built to run models rather than train them. The idea it makes concrete is an inference-optimized ASIC versus a general-purpose GPU.
Why: At decode time the bottleneck is usually moving data, not doing math, so a chip co-designed around that movement can serve the same tokens using far less power per token — early testing reports substantially better performance-per-watt (final numbers still being measured), which at OpenAI's scale materially changes serving cost.
vs prior: A general-purpose GPU runs anything — training, graphics, every model — and pays in silicon and power for that flexibility; Jalapeño is hard-wired for inference only, trading the GPU's versatility for a shorter, faster path between memory and compute.
Think of it as
A kitchen rebuilt to cook one dish, with the pantry moved beside the stove.
