What hardware do you need to run Nemotron 3.5 Bolt 74B-A9B?

Nemotron 3.5 Bolt 74B-A9B at IQ2 is 22 GB of weights, with a context ceiling of 256k tokens. NVIDIA quantised this one itself, to two bits with quantisation-aware training, so that a 74B model fits in 24 GB. It reads images and video as well as text, and the vision encoder is a separate 1.7 GB download this page does not price. On the cheapest machine that runs it, a Framework Desktop, 32GB, it generates 48 tok/s at 32k of context, worked out from memory bandwidth.

How good is Nemotron 3.5 Bolt 74B-A9B, really?

On the Artificial Analysis Intelligence Index v4.3 it scores 21 (very verbose: 230M output tokens over the index against a median of 100M), which puts it in the Haiku-class band. In the same band as Anthropic's cheap, fast tier. Every current OpenAI model scores above this band. See the whole table.

Head to head with its neighbours on the leaderboard: vs Qwen3.6 27B above it, vs Qwen3.6 35B-A3B below. Two more models need much the same memory at 32k of context: vs Qwen3 30B-A3B Instruct 2507, vs Qwen3-Coder 30B-A3B.

What Nemotron 3.5 Bolt 74B-A9B costs either way

Nobody rents Nemotron 3.5 Bolt 74B-A9B by the token. The closest hosted match, Qwen3.6 27B, costs $0.3 per million input tokens and $2 per million output (OpenRouter, cheapest active endpoint, checked 2026-10-21). Buying a machine only beats that if you use it hard enough, for long enough, that the hardware price divides down below the rental bill. What a million tokens costs each way puts the two prices side by side.

Machines that run Nemotron 3.5 Bolt 74B-A9B

Every machine here runs it, but not to the same length. The Strix Halo Framework Desktop, 32GB stops at 128k; the AMD Radeon AI PRO R9700, 32GB takes it to 256k, which is this model's own ceiling and the longest the calculator offers. The weights are the same size on every machine; what differs is the memory left for the key-value cache, which grows with every token you keep.

The table is the cheapest machine in each family, not the only one that runs it: 41 of the 56 machines on this site hold it at 32k. Within a family only memory decides, so that is every Mac Studio and Strix Halo box at any size, every NVIDIA card with 24GB or more, every Mac mini and MacBook Pro with 48GB or more, the Radeon AI PRO R9700, 32GB and the DGX Spark, 128GB. One table sets every machine against the others: price, the memory its GPU can use, what it holds and how long it takes to pay for itself.

One machine per family, cheapest first. Speeds are measured where a public benchmark exists and estimated from memory bandwidth otherwise; the calculator says which for any configuration. The longest context is the longest setting the calculator offers that the machine still holds this model at, cache included, and no machine is shown taking it past its own 256k limit. Each one opens the calculator on that machine at that length.

For scale, the calculator starts from 80 tok/s for a hosted API and times a local machine against it. 2 of the 7 above reach it, and the slowest is 32 tok/s.

The cheapest machine that runs it is sold with 32 GB. There is a page on what 32 GB runs, machine by machine: it names the 6 machines here sold at that size, and the 29 current models the roomiest of them holds at 32k, this one among them.

It fits in less memory than that: the smallest size here that runs it at 32k is 24 GB, on the GeForce RTX 3090, 24GB at $1,499, card only, which hands a model 23 GB and is no longer sold.

Run Nemotron 3.5 Bolt 74B-A9B at 256k on the AMD Radeon AI PRO R9700, 32GB

How much memory does Nemotron 3.5 Bolt 74B-A9B need at 256k?

Every figure above is priced at 32k of context. At 256k, which is this model's own ceiling and the longest the calculator offers, Nemotron 3.5 Bolt 74B-A9B needs 24.2 GB rather than 22.3 GB. The weights are the same either way; the key-value cache goes from 0.3 GB to 2.1 GB.

28 of the 37 machines here still sold at a published price hold Nemotron 3.5 Bolt 74B-A9B at 32k and 27 hold it at 256k. The cheapest still sold goes from the Framework Desktop, 32GB at $1,269 to the Radeon AI PRO R9700, 32GB at $1,299, card only. The Radeon AI PRO R9700, 32GB is priced as the card alone, so add the PC around it before comparing it with a complete computer.

On the Radeon AI PRO R9700, 32GB it runs 114 tok/s estimated at 32k and 69 tok/s estimated at 256k. Every token generated reads the whole cache back, so a longer window is slower as well as bigger.

The specifics

Parameters

74.9B, of which 9B are active per token

Quantisation

IQ2

Weights on disk

22 GB

KV cache

0.3 GB at 32k context — Only 8 of the 88 blocks are attention. Forty are Mamba-2, which keep a fixed-size state whatever the context, and forty are mixture-of-experts. The whole 256k window costs about 2.1 GB. How weights and cache add up.

Maximum context

256k tokens

Licence

OpenMDW 1.1

Sources

nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF, nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16