If your AI Factory sells tokens, why not optimize token emission?
Last week, Nvidia invited a handful of journalists to a briefing about AI infrastructure to help convey the physical reality behind simulated intelligence."Infrastructure is physical, it's real," said Ian Buck, general manager of Nvidia's hyperscale and HPC computing business. "You can touch it, you can see it, and it's what helps bring AI to life. So that is part of the goal here."The briefing included a tour of a working Nvidia lab – a mini-datacenter – nestled in a residential district of Sunnyvale, a Silicon Valley suburb. Attendees were asked not to reveal the location – which is one of four such facilities – though they're easy enough to discover with a bit of online sleuthing.
AI infrastructure is indeed real – or at least realized as revenue when equipment is booked under a bill-and-hold arrangement – and you can touch it if your job involves handling tech gear.
It's also controversial as the AI tsunami crests over a society that's uncertain if the technology will bring bounty or harm. Just one day after the press event, Dutch activists peppered a data center serving Microsoft workloads with chemical-filled balloons to protest climate and political concerns. Nvidia presumably would prefer not to have neighbors grousing about power and water consumption at its facilities, though such concerns may be muted in an area so in thrall to the tech sector.In any event, Nvidia's latest data center hardware should require significantly less human intervention than its older kit. During the lab tour, Andrew Bell, a senior VP of hardware engineering, showed off how the company's Vera Rubin NVL72 compute tray is vastly faster to install than prior hardware. "With automation assembly, the compute tray inside the Vera Rubin NVL72 can be assembled in one minute, compared to GB200 compute tray assembly that typically takes 90 minutes. This represents a 90x improvement," he said.The Nvidia event focused on how the company's Vera Rubin platform, announced at Computex 2024 and detailed at CES in January, advances its vision of AI Factories, a type of computing infrastructure dedicated to running AI workloads.
