The usual story about Nvidia is a hardware story: the fastest chips, in the shortest supply, at the highest price. The more important part is software. For two decades the company’s real moat has been CUDA. It is the layer that turns its silicon into something developers can build AI on. That moat is now being tested.
CUDA, short for Compute Unified Device Architecture, took years to build. It bundles ready-made code and debugging tools. It also lets thousands of chips train a model together. The new threat is blunt: AI coding agents that can write that kind of low-level software themselves.
Jeremy Nixon is a former Google Brain researcher who founded the startup Infinity. He told Business Insider his team used agents to rebuild CUDA-like software for chip firm D-Matrix in about 10 hours. He framed it as proof that one of Nvidia’s biggest moats is being crossed.
Two moats, both under pressure
CUDA’s first advantage is the software. Its second is everything built on top of it. Millions of lines of company code and workflows make switching to a rival chip slow and costly. Amazon’s own documents once flagged CUDA as a major roadblock to adopting its in-house AI chips.
