vLLM hits 500K GPUs as co-founder Simon Mo makes the case for open models

Half a million GPUs are now running vLLM. That number, cited by the project’s own maintainers, is the kind of figure that stops a conversation cold.

What vLLM actually is, and why it matters

vLLM started at UC Berkeley’s Sky Computing Lab as a research project tackling one of inference’s most stubborn problems: GPU memory is expensive, and large language models waste a lot of it.

The core innovation is something called PagedAttention. Think of it like virtual memory management for your laptop, but applied to the KV cache that language models use to track context. It lets a GPU serve far more requests simultaneously by being smarter about how it allocates memory on the fly.

The project now supports over 500 model architectures and more than 200 accelerator types. In May 2025, it became a PyTorch Foundation project. The GitHub star count climbed into the tens of thousands rapidly.

vLLM hits 500K GPUs as co-founder Simon Mo makes the case for open models

Other newsrooms on this story

Related reading

Inside vLLM: How the World's Fastest LLM Inference Engine Works

AirLLM: Running 70B Parameter LLMs on a Single 4GB GPU

AirLLM Runs a 70B Model on a 4GB GPU. It's True, and That's Not the Interesting…

ZML releases free inference software that could reshape AI compute economics

Yann LeCun argues LLMs will drive real-world applications, but not human-level…

Virtuals integrates Leyten's distributed GPU inference engine to run GLM-5.2…