What hardware do you need to run Gemma 3 12B it?

Gemma 3 12B it at Q4_K_M is 7.3 GB of weights, with a context ceiling of 128k tokens. Strong multilingual for its size and light on KV cache. Not a coding model. On the cheapest machine that runs it, a Mac mini M6, 16GB, it generates 12 tok/s at 32k of context, worked out from memory bandwidth.

How good is Gemma 3 12B it, really?

On the Artificial Analysis Intelligence Index v4.3 it scores 4, which puts it in the Below every hosted tier band. Fine for simple, well-specified tasks. Noticeably less capable than anything the big labs sell today. See the whole table.

• Summarising — good
• Translation — good
• Everyday coding — usable
• Reasoning & maths — don’t
• Agentic work — don’t

Head to head: vs Ministral 3 8B, the next model up the leaderboard. Also head to head: vs Gemma 4 12B, the current Gemma nearest it in size. One more model needs much the same memory at 32k of context: vs Qwen3 8B.

Is Gemma 3 12B it still worth running?

Not on the two figures this site can compare. The intelligence index puts eight of the current models here above Gemma 3 12B it, and not one of them asks more of a machine: at 32k of context it needs 9.8 GB and scores 4, where the strongest of the eight, Qwen3.5 9B, needs 6.8 GB and scores 14 (reasoning mode; 13 without).

The cheapest machine still sold that holds Gemma 3 12B it at 32k is the Mac mini M6, 16GB at $899, which holds Qwen3.5 9B as well. This site quotes Qwen3.5 9B at 17 tok/s estimated there and Gemma 3 12B it at 12 tok/s estimated.

Qwen3.5 9B's score is Artificial Analysis's own estimate rather than a measured run of the full v4.3 suite.

What Gemma 3 12B it costs either way

Renting the same model costs $0.05 per million input tokens and $0.15 per million output (OpenRouter, cheapest active endpoint, checked 2026-10-29). Buying a machine only beats that if you use it hard enough, for long enough, that the hardware price divides down below the rental bill. What a million tokens costs each way puts the two prices side by side.

Machines that run Gemma 3 12B it

Every machine here runs it, but not to the same length. The Mac mini M6, 16GB stops at 32k; the Strix Halo Framework Desktop, 32GB takes it to 128k, this model's own ceiling. The weights are the same size on every machine; what differs is the memory left for the key-value cache, which grows with every token you keep.

All 56 machines on this site run it at 32k, not just the eight in the table. One table sets every machine against the others: price, the memory its GPU can use, what it holds and how long it takes to pay for itself.

One machine per family, cheapest first. Speeds are measured where a public benchmark exists and estimated from memory bandwidth otherwise; the calculator says which for any configuration. The longest context is the longest setting the calculator offers that the machine still holds this model at, cache included, and no machine is shown taking it past its own 128k limit. Each one opens the calculator on that machine at that length.

For scale, the calculator starts from 80 tok/s for a hosted API and times a local machine against it. 1 of the 8 above reaches it, and the slowest is 12 tok/s.

The cheapest machine that runs it is sold with 16 GB. There is a page on what 16 GB runs, machine by machine: it names the 6 machines here sold at that size, and the 15 current models the roomiest of them holds at 32k.

It fits in less memory than that: the smallest size here that runs it at 32k is 12 GB, on the GeForce RTX 3060, 12GB at $329, card only, which hands a model 11 GB and is no longer sold.

Run Gemma 3 12B it at 128k on the Strix Halo Framework Desktop, 32GB

How much memory does Gemma 3 12B it need at 128k?

Every figure above is priced at 32k of context. At 128k, this model's own ceiling, Gemma 3 12B it needs 16.2 GB rather than 9.8 GB. The weights are the same either way; the key-value cache goes from 2.5 GB to 8.9 GB.

All 37 machines here still sold at a published price hold Gemma 3 12B it at 32k and 30 hold it at 128k. The cheapest still sold goes from the Mac mini M6, 16GB at $899 to the Framework Desktop, 32GB at $1,269.

On the Framework Desktop, 32GB it runs 20 tok/s estimated at 32k and 12 tok/s estimated at 128k. Every token generated reads the whole cache back, so a longer window is slower as well as bigger.

The specifics

Parameters

12.2B

Quantisation

Q4_K_M

Weights on disk

7.3 GB

KV cache

2.5 GB at 32k context — 8 of the 48 layers grow with the context and the other 40 stop at a 1,024-token window. All 48 carry eight KV heads at 256 wide, 8,192 bytes a token, so the windowing slows the growth rather than stopping it. How weights and cache add up.

Maximum context

128k tokens (input)

Licence

Gemma Terms of Use

Sources

bartowski/google_gemma-3-12b-it-GGUF, unsloth/gemma-3-12b-it