Running reasoning and agentic AI at the edge has been harder than it needs to be. Until recently, models capable of multi-step reasoning were too large to run locally on edge hardware. Developers building agents have had to route inference through a data center, adding network dependency, increasing costs, and exposing data that may need to stay on device.That constraint is lifting. Several model families released throughout the summer have collectively marked a turning point for edge AI. This new generation of compact open models now delivers reasoning and agentic capabilities that required large data center systems only a few months ago, and NVIDIA Jetson can run them today.
This could power in-cab assistants, real-time anomaly detection, and robots working in harsh or remote environments. On-site experts could spend less time troubleshooting, and critical systems could keep running when connectivity is limited or unavailable.This post walks through what you need to know to deploy this new generation of open models on Jetson, using Nemotron 3.5 Lightning and Qwen3.8-27B as examples. You’ll learn what to look for when comparing model architectures, how to apply inference optimization techniques to get the most out of your hardware, and how to validate the configuration for your workload.
