Can an NVIDIA GeForce RTX 5090, 32GB run local LLMs?

Yes — 33 of the 49 current models on this site fit in its 31 GB of usable memory. The strongest it holds, Qwen3.8 27B, runs at 66 tok/s with a 32k window, worked out from memory bandwidth. Whether that saves you money is a different question: at $1,999 and 500k tokens a day, it pays back in 123 years. Its price here is the card alone, so every figure below leaves out the PC you need to put it in. Every card on this site is set against the others here.

Run the numbers on this machine

What the GeForce RTX 5090, 32GB runs

They do not all hold the same window. The weights are a fixed size, but the key-value cache grows with every token you keep, so what is left of 31 GB after the weights is how far the context goes. On 4 of the 12 below, this machine's memory is what runs out first: Granite 4.2 30B stops soonest, at 32k. The rest reach 256k, where the calculator's list ends, or the longest setting their own context limit allows.

Speed and memory are at 32k context, the setting the calculator starts on; the memory column is the weights plus the cache for that much of it. Each speed says how it was arrived at, and every one here is estimated from this machine's 1792 GB/s of memory bandwidth rather than taken from a published benchmark. There is a page on how that sum works, and what each size needs, and one on what 32 GB runs, machine by machine. The longest context is the longest setting the calculator offers that this machine still holds the model at, cache included, and each one opens the calculator on that model at that length. A figure tagged memory is one this machine ran out of room for, and the rest are stopped by the model's own limit or by the end of the list.

For scale, the calculator starts from 80 tok/s for a hosted API and times a local machine against it. 7 of the 12 above reach it, and the slowest is 47 tok/s.

21 more fit. Ranked below the twelve above: MiniCPM5 2B, Qwen3.5 4B, Ling 3.0 tiny, Granite 4.2 8B, Qwen3-Coder 30B-A3B, gpt-oss-20b, Gemma 4 E4B, LFM2.5 2.6B, Devstral Small 2 24B, Ministral 3 14B, Ministral 3 8B. The other 10 have no intelligence-index score, so the ranking cannot place them: Laguna XS 2.1, LFM3 1.7B, Ornith 1.5 9B, Ornith 1.5 35B-A3B, Spark-X2.5 4B, Spark-X2.6 4B, KAT-Coder V2.5 Dev 35B-A3B, LFM2.5 24B-A2B, Flint 40B-A2.5B, Nanbeige4.3 33B. Their pages show what each one needs and what runs it.

The 49 counted above are the current models. This site prices 16 superseded ones as well, because people still run them, and this machine holds ten of those at 32k, strongest first: Qwen3 30B-A3B Instruct 2507, DeepSeek-R1-Distill-Qwen-32B, Llama 3.1 8B Instruct at Q4_K_M, Llama 3.1 8B Instruct at Q8_0, Mistral Small 3.2 24B Instruct, Qwen3 32B at Q4_K_M, Qwen3 14B, Qwen3 8B, Gemma 3 27B it, Gemma 3 12B it. Each of those pages asks whether the model is still worth running.

What has been measured on the GeForce RTX 5090, 32GB

Not one of the speeds above was measured on this machine, and that is not because nobody has run a model on one. This site holds four published speeds taken on this machine, three of them on superseded models the table above leaves out. They are below, as published and again at 32k of context, which is what every other speed on this page is taken at.

A published figure is what its source reports, at the window the model was run at. The 32k figure is that measurement read at 32k of context: every token generated reads the weights and the whole key-value cache, so the longer the window the slower it runs. The figures come from ggml-org/llama.cpp, hardware-corner.net.

Other machines to weigh against the GeForce RTX 5090, 32GB

Every row uses the same defaults as the figures above: 500k tokens a day at 15:1 input to output, 32k context, and each machine's strongest model that fits, counted against the same 49 current models. Graphics cards are priced as the card alone, so add the PC around one before comparing it with a complete computer.

Head to head with another card: RTX PRO 6000 Blackwell, 96GB · GeForce RTX 4090, 24GB · GeForce RTX 3090, 24GB · Radeon AI PRO R9700, 32GB · GeForce RTX 4080, 16GB · GeForce RTX 3060, 12GB. With a complete computer: Framework Desktop, 64GB · MacBook Pro M5 (14-inch), 16GB. There are 111 machine match-ups on the site.

The specifics

Chip

GeForce RTX 5090 — Blackwell · 32 GB GDDR7

Memory bandwidth

1792 GB/s — 28 Gbps × 512-bit bus ÷ 8 = 1,792 GB/s, which NVIDIA's compare page also publishes.

Usable by the GPU

31 GB — Usable memory is the VRAM less 1 GB: the margin llama.cpp's automatic fitting (--fit) leaves free on each device by default (common/common.h, fit_params_target = 1 GiB). Treated as nominal GB, like the other machines here.

Power under load

575 W (published) — 575 W is the maker's rated board power for the card alone. The rest of the PC adds more, and LLM decoding usually draws less than the full rating. NVIDIA recommends a 1000 W system power supply.

Availability

Launch price. Founders Edition out of stock at NVIDIA when checked 2026-09-16; partner cards sell for more.

Sources

NVIDIA, NVIDIA (comparison), NVIDIA Newsroom, NVIDIA Marketplace, TechPowerUp, ggml-org/llama.cpp