How much memory do you need to run a local LLM?

Two numbers, added together, held at the same time. The weights are a fixed download. The key-value cache grows with the context window you ask for, and on some models it ends up larger than the weights. Llama 3.3 70B Instruct at Q4_K_M is 42.5 GB of weights plus 10.7 GB of cache at 32k context: 53.3 GB in one machine.

Run the numbers on that pairing

Where the cache figure comes from

The weights are whatever the file weighs on disk. The cache is arithmetic. Every token in the context window leaves a key and a value behind in every layer, and all of it stays resident while the model is loaded.

For Llama 3.3 70B Instruct: 2 (a key and a value) × 8 key-value heads × 128 numbers per head × 2 bytes = 4,096 bytes per token, per layer. Over 80 layers that is 327,680 bytes for every token in the window. Fill 32,768 tokens of context and the cache is 10.7 GB.

Two things follow. The window you ask for is a hardware decision, not a setting: on most models the cache grows in step with it, and only the ones with sliding-window layers stop growing partway. And two models of the same size can want very different amounts, because the number of key-value heads and the width of each one are architecture choices.

Context is the part people miss

A longer window costs memory before it costs anything else. Here is how many of the 39 current models fit three machines as the window grows, counting a model only where its own context ceiling allows it.

That is a shopping list, not a setting. Llama 3.3 70B Instruct needs 45.2 GB at 8k context, which the Strix Halo Corsair AI Workstation 300, 64GB at $1,700 holds. At 32k it needs 53.3 GB, and the cheapest machine that holds it becomes the Strix Halo Framework Desktop, 128GB at $3,449.

Size is a poor guide to the cache. Two models here have the same 125 billion parameters: at 128k context Devstral 2 123B wants 47.2 GB of cache and Qwen3.5 122B-A10B wants 3.2 GB. Each model's page carries its own figure.

You can also make the cache smaller. llama.cpp keeps the key-value cache at 16 bits unless you pass --cache-type-k and --cache-type-v. q8_0 stores 32 values in 34 bytes and q4_0 in 18, so the cache shrinks to about a half or about a quarter. Every generated token reads the whole cache, so a smaller one also speeds up long contexts. Both are lossy; 4-bit loses more. (source) The calculator has that switch, and every figure on this page is at the 16-bit default.

Installed memory is not usable memory

The number on the box is not the number a model gets. The system takes a share, and on a machine with unified memory the GPU is only allowed to address part of the rest. This is what each machine can actually hand a model, cheapest machine shown at each level, counted against the 39 current models.

Read the last column before you spend anything. From 21 GB of usable memory up to 119.5 GB, the strongest model on this list does not change: it is Qwen3.8 27B the whole way. More memory across that stretch buys more models, more context and more room to work, not a cleverer one. The next step up is GLM-5.3-Flash, and the cheapest machine that holds it is the Mac Studio M5 Ultra, 256GB at $10,799.

Every figure is at 32k context unless the row says otherwise, with the cache at 16 bits, at the quantisation named against each model. Weights are the published file sizes on each model's page; the cache is worked out from the architecture recorded there. Machines are the current ones at list price, with the memory their maker publishes and the usable share on each machine's page; graphics cards are priced as the card alone, so add the PC around one before comparing one with a complete computer. The tables by size cover all 55 models listed here, superseded ones included and marked, because people still run them. The two tables of what a machine holds count the 39 current ones instead, which is what every machine page and the calculator count. To change the context, the quantisation or the cache type, open the calculator.