Nvidia's new open model trades reasoning depth for 2x agent speed
Nemotron 3.5 Lightning hits about 670 tokens per second, roughly double Gemini 3.5 Flash-Lite, by optimizing for the repetitive 90 percent of agent work instead of one-shot deep reasoning.
Nvidia trained a model to skip deep thinking on purpose. For agents, that might be the right call.
Nvidia released Nemotron 3.5 Lightning on August 11, 2026, an open-weight model built for one job: running the repetitive, high-volume slice of AI agent work fast, instead of reasoning deeply on hard one-off problems. It ships under the permissive OpenMDW-1.1 license with about 31.6 billion total parameters, only 3.6 billion of which activate on any given token, using a mixture-of-experts design. Nvidia's own benchmarks put its throughput at roughly 670 tokens per second, about twice Google's Gemini 3.5 Flash-Lite. That is the news: a major chipmaker is now shipping a model whose main selling point is that it is not trying to be the smartest one available.
built for the boring 90 percent of agent work
Long-running AI agents spend most of their time on high-volume execution: tool calls, result validation, and subagent delegation.
That is Nvidia's own framing for why Nemotron 3.5 Lightning exists. The backbone is a hybrid of Mamba and Transformer layers, with sparse mixture-of-experts activation and a context window that stretches to 1 million tokens. It ships in both BF16 and NVFP4 weight formats, and Nvidia added speculative decoding, multi-token prediction paired with dedicated draft models, to cut latency further.
the numbers Nvidia is showing
- About 670 tokens per second throughput, roughly double Gemini 3.5 Flash-Lite's 386 tokens per second
- An Artificial Analysis Intelligence Index score of 24, matching OpenAI's gpt-oss-120b
- A GDPval-AA v2 agentic Elo of 824, ahead of gpt-oss-120b's 800
- 86 percent accuracy on PinchBench, finishing tasks 30 percent faster than Qwen3.6 35B
Nvidia says the model runs anywhere from a DGX Spark workstation and Jetson or RTX 5090 hardware up to full datacenter clusters, through vLLM, TensorRT-LLM, Ollama, or llama.cpp. It also ships training recipes, so a team can fine-tune Nemotron 3.5 Lightning on its own agent traffic instead of running it as shipped.
Every one of those numbers comes from Nvidia or from Nvidia citing Artificial Analysis's public leaderboard, not from an independent lab running its own eval pass. That is normal for a same-day launch, but it is still a vendor grading its own homework until other labs reproduce the throughput and Elo numbers on separate hardware. An Artificial Analysis Intelligence Index score of 24 also puts it well below the top general-purpose reasoning models, and it was never built to compete there.
Why a build studio cares
This is a real architecture decision, not a hypothetical one. On the AI workflow builds we run, most agent turns are not hard reasoning steps: they are calling a tool, checking whether the tool's output looks right, and handing off to the next subagent, the same three tasks Nvidia named. Routing that volume to a fast, cheap model and saving a slower frontier model for the hard fraction of a pipeline is a real cost and latency lever, and Nemotron 3.5 Lightning is now a concrete option for the fast side of that split, provided a team is comfortable running open weights instead of an API.
Next step: read Nvidia's release notes for the full architecture writeup, then The Decoder's coverage for a second read on where the speed claims hold up. If your agent pipeline is burning frontier-model cost on tool calls and validation loops, write to us at hello@gattyworks.com.