How much memory do you need to run a local LLM?

Two numbers, added together, held at the same time. The weights are a fixed download. The key-value cache grows with the context window you ask for, and on some models it ends up larger than the weights. Nemotron 3.5 Bolt 74B-A9B at IQ2 is 22 GB of weights plus 0.3 GB of cache at 32k context: 22.3 GB in one machine.

Run the numbers on that pairing

Jump to: Where the cache figure comes from · How much memory for a 7B or 8B model? · How much memory for a 14B to 32B model? · How much memory for a 70B model? · How much memory for a 100B model or larger? · Context is the part people miss · What can you run with the memory you already have?

Where the cache figure comes from

The weights are whatever the file weighs on disk. The cache is arithmetic. Every token in the context window leaves a key and a value behind in every layer, and all of it stays resident while the model is loaded.

Two things follow. The window you ask for is a hardware decision, not a setting: on most models the cache grows in step with it, and only the ones with sliding-window layers stop growing partway. And two models of the same size can want very different amounts, because the number of key-value heads and the width of each one are architecture choices.

Context is the part people miss

A longer window costs memory before it costs anything else. Here is how many of the 50 current models fit three machines as the window grows, counting a model only where its own context ceiling allows it.

Size is a poor guide to the cache. Two models here have the same 125 billion parameters: at 128k context Devstral 2 123B wants 47.2 GB of cache and Qwen3.5 122B-A10B wants 3.2 GB. Each model's page carries its own figure.

You can also make the cache smaller. llama.cpp keeps the key-value cache at 16 bits unless you pass --cache-type-k and --cache-type-v. q8_0 stores 32 values in 34 bytes and q4_0 in 18, so the cache shrinks to about a half or about a quarter. Every generated token reads the whole cache, so a smaller one also speeds up long contexts. Both are lossy; 4-bit loses more. (ggml-org/llama.cpp) The calculator has that switch, and every figure on this page is at the 16-bit default.

What can you run with the memory you already have?

Every size a machine on sale here comes in is below, with how much of it a model actually gets, how many of the 50 current models fit in that at 32k context, and the strongest of them. The number on the box is not the number a model gets: the system takes a share, and on a machine with unified memory the GPU is only allowed to address part of the rest. 32 GB appears 3 times below, handing a model anything from 21 GB on the Mac mini M6, 32GB to 31 GB on the AMD Radeon AI PRO R9700, 32GB. So find your size in the first column, then read down to the machine nearest yours. The cheapest machine is shown at each level, and each row opens the calculator on that machine and that model.

Read the strongest-model column before you spend anything. From 21 GB of usable memory up to 119.5 GB, the strongest model on this list does not change: it is Qwen3.8 27B the whole way. More memory across that stretch buys more models, more context and more room to work, not a cleverer one. The next step up is GLM-5.3-Flash, and the cheapest machine that holds it is the Mac Studio M5 Ultra, 256GB at $10,799.

Every size in that first column has a page of its own, and so does every other size a machine here is sold in: the machines you can buy at it, what each of them hands a model, every model that fits, and what the next size up buys. 12 GB · 16 GB · 24 GB · 32 GB · 36 GB · 48 GB · 64 GB · 96 GB · 128 GB · 192 GB · 256 GB · 512 GB.

Every figure is at 32k context unless the row says otherwise, with the cache at 16 bits, at the quantisation named against each model. Weights are the published file sizes on each model's page; the cache is worked out from the architecture recorded there. Machines are the current ones at list price, with the memory their maker publishes and the usable share on each machine's page; graphics cards are priced as the card alone, so add the PC around one before comparing one with a complete computer. All seven cards here are ranked by what each one holds. The tables by size cover all 66 models listed here, superseded ones included and marked, because people still run them. The two tables of what a machine holds count the 50 current ones instead, which is what every machine page and the calculator count. To change the context, the quantisation or the cache type, open the calculator.