What hardware do you need to run Llama 3.1 8B Instruct?

Llama 3.1 8B Instruct at Q8_0 is 8.5 GB of weights, with a context ceiling of 128k tokens. Same model at higher precision. Marginal quality gain; twice the memory. On the cheapest machine that runs it, a Mac mini M6, 24GB, it generates 9.9 tok/s at 32k of context, worked out from memory bandwidth.

How good is Llama 3.1 8B Instruct, really?

On the Artificial Analysis Intelligence Index v4.3 it scores 7, which puts it in the Below every hosted tier band. Fine for simple, well-specified tasks. Noticeably less capable than anything the big labs sell today. See the whole table.

• Summarising — good
• Translation — usable
• Everyday coding — usable
• Reasoning & maths — don’t
• Agentic work — don’t

Is Llama 3.1 8B Instruct still worth running?

Not on the two figures this site can compare. The intelligence index puts eight of the current models here above Llama 3.1 8B Instruct, and not one of them asks more of a machine: at 32k of context it needs 12.8 GB and scores 7, where the strongest of the eight, Qwen3.5 9B, needs 6.8 GB and scores 14 (reasoning mode; 13 without).

The cheapest machine still sold that holds Llama 3.1 8B Instruct at 32k is the Mac mini M6, 24GB at $1,099, which holds Qwen3.5 9B as well. This site quotes Qwen3.5 9B at 19 tok/s estimated there and Llama 3.1 8B Instruct at 9.9 tok/s estimated. Qwen3.5 9B itself runs on the Mac mini M6, 16GB at $899, $200 less.

Both scores above are Artificial Analysis's own estimate rather than a measured run of the full v4.3 suite.

What Llama 3.1 8B Instruct costs either way

Renting the same model costs $0.02 per million input tokens and $0.04 per million output (OpenRouter, cheapest active endpoint, checked 2026-10-29). Buying a machine only beats that if you use it hard enough, for long enough, that the hardware price divides down below the rental bill. What a million tokens costs each way puts the two prices side by side.

Machines that run Llama 3.1 8B Instruct

Every machine here runs it, but not to the same length. The Mac mini M6, 24GB stops at 32k; the AMD Radeon AI PRO R9700, 32GB takes it to 128k, this model's own ceiling. The weights are the same size on every machine; what differs is the memory left for the key-value cache, which grows with every token you keep.

The table is the cheapest machine in each family, not the only one that runs it: 50 of the 56 machines on this site hold it at 32k. Within a family only memory decides, so that is every Mac Studio and Strix Halo box at any size, every NVIDIA card with 16GB or more, every Mac mini and MacBook Pro with 24GB or more, the Radeon AI PRO R9700, 32GB and the DGX Spark, 128GB. One table sets every machine against the others: price, the memory its GPU can use, what it holds and how long it takes to pay for itself.

One machine per family, cheapest first. Speeds are measured where a public benchmark exists and estimated from memory bandwidth otherwise; the calculator says which for any configuration. The longest context is the longest setting the calculator offers that the machine still holds this model at, cache included, and no machine is shown taking it past its own 128k limit. Each one opens the calculator on that machine at that length.

For scale, the calculator starts from 80 tok/s for a hosted API and times a local machine against it. 1 of the 7 above reaches it, and the slowest is 8.9 tok/s.

The cheapest machine that runs it is sold with 24 GB. There is a page on what 24 GB runs, machine by machine: it names the 7 machines here sold at that size, and the 28 current models the roomiest of them holds at 32k.

It fits in less memory than that: the smallest size here that runs it at 32k is 16 GB, on the GeForce RTX 4080, 16GB at $1,199, card only, which hands a model 15 GB and is no longer sold.

Run Llama 3.1 8B Instruct at 128k on the AMD Radeon AI PRO R9700, 32GB

How much memory does Llama 3.1 8B Instruct need at 128k?

Every figure above is priced at 32k of context. At 128k, this model's own ceiling, Llama 3.1 8B Instruct needs 25.7 GB rather than 12.8 GB. The weights are the same either way; the key-value cache goes from 4.3 GB to 17.2 GB.

33 of the 37 machines here still sold at a published price hold Llama 3.1 8B Instruct at 32k and 27 hold it at 128k. The cheapest still sold goes from the Mac mini M6, 24GB at $1,099 to the Radeon AI PRO R9700, 32GB at $1,299, card only. The Radeon AI PRO R9700, 32GB is priced as the card alone, so add the PC around it before comparing it with a complete computer.

On the Radeon AI PRO R9700, 32GB it runs 43 tok/s estimated at 32k and 22 tok/s estimated at 128k. Every token generated reads the whole cache back, so a longer window is slower as well as bigger.

Three more machines, at a shorter window

The table above lists the machines that run it at 32k, the context the calculator starts on, where Llama 3.1 8B Instruct needs 13 GB. Three more machines hold it at a shorter window, because the weights are the same size either way and the key-value cache is not. The cheapest of those is the Mac mini M6, 16GB at $899. The table above starts at $1,099.

One machine per family, cheapest first, from the same list as the table above. Each window is the longest setting the calculator offers that the machine still holds this model at, and it opens the calculator on that machine at that length. What it needs there is the weights plus the key-value cache at that window, measured against the memory each machine's GPU can address, which that machine's own page gives.

Llama 3.1 8B Instruct at Q4_K_M or Q8_0?

Llama 3.1 8B Instruct is one of two models this site lists at two precisions, and the heavier build is the subject of this page. The weights go from 4.9 GB at Q4_K_M to 8.5 GB at Q8_0. The key-value cache is the same size either way, 4.3 GB at 32k of context, because both builds share an architecture, so what a machine has to hold goes from 9.2 GB to 12.8 GB. All 56 machines on this site hold the lighter build at 32k; 50 hold the heavier. The cheapest one still sold is the Mac mini M6, 16GB at $899 for the lighter build and the Mac mini M6, 24GB at $1,099 for the heavier.

The heavier build costs speed as well, because every token generated reads the weights and the whole cache. On the Mac mini M6, 24GB, the cheapest machine still sold that holds both, this site quotes 14 tok/s at Q4_K_M and 9.9 tok/s at Q8_0, both worked out from memory bandwidth.

What the extra precision buys is the half this site has no figure for. The intelligence index has one score for Llama 3.1 8B Instruct, 7, which both builds here carry, and the five ratings above are the same five on both. The memory and the speed are the whole measured difference between them.

The specifics

Parameters

8B

Quantisation

Q8_0 — also listed here at Q4_K_M, which is 4.9 GB

Weights on disk

8.5 GB

KV cache

4.3 GB at 32k context. How weights and cache add up.

Maximum context

128k tokens

Licence

Llama 3.1 Community License

Sources

bartowski/Meta-Llama-3.1-8B-Instruct-GGUF, unsloth/Llama-3.1-8B-Instruct