What hardware do you need to run Inkling Small?

Inkling Small at UD-Q4_K_M is 163 GB of weights, with a context ceiling of 1M tokens. Thinking Machines' open weights. 'Small' is relative to a 975B sibling — this still needs 256 GB. On the cheapest machine that runs it, a Mac Studio M5 Ultra, 256GB, it generates 43 tok/s at 32k of context, worked out from memory bandwidth.

How good is Inkling Small, really?

On the Artificial Analysis Intelligence Index v4.3 it scores 26, which puts it in the Haiku-class band. In the same band as Anthropic's cheap, fast tier. Every current OpenAI model scores above this band. See the whole table.

Head to head with its neighbours on the leaderboard: vs Beam 1.0 501B-A23B above it, vs Tencent Hy3 below. One more model needs much the same memory at 32k of context: vs DeepSeek V4-Flash.

What Inkling Small costs either way

Renting the same model costs $0.45 per million input tokens and $1.2 per million output (OpenRouter, cheapest active endpoint, checked 2026-10-29). Buying a machine only beats that if you use it hard enough, for long enough, that the hardware price divides down below the rental bill. What a million tokens costs each way puts the two prices side by side.

Machines that run Inkling Small

The Mac Studio M5 Ultra, 256GB holds it to 256k, the longest the calculator offers.

The table is the cheapest machine in each family, not the only one that runs it: 4 of the 56 machines on this site hold it at 32k. Within a family only memory decides, so that is every Mac Studio with 256GB or more. One table sets every machine against the others: price, the memory its GPU can use, what it holds and how long it takes to pay for itself.

One machine per family, cheapest first. Speeds are measured where a public benchmark exists and estimated from memory bandwidth otherwise; the calculator says which for any configuration. The longest context is the longest setting the calculator offers that the machine still holds this model at, cache included, and no machine is shown taking it past its own 1M limit. Each one opens the calculator on that machine at that length.

For scale, the calculator starts from 80 tok/s for a hosted API and times a local machine against it. The one above does not reach it, at 43 tok/s.

The cheapest machine that runs it is sold with 256 GB. There is a page on what 256 GB runs, machine by machine: it names the 2 machines here sold at that size, and the 44 current models the roomiest of them holds at 32k, this one among them.

Run Inkling Small at 256k on the Mac Studio M5 Ultra, 256GB

How much memory does Inkling Small need at 256k?

Every figure above is priced at 32k of context. At 256k, the longest the calculator offers, Inkling Small needs 170.1 GB rather than 163.6 GB. The weights are the same either way; the key-value cache goes from 1 GB to 7.6 GB.

The window costs nothing in hardware: the one machine here still sold at a published price that holds Inkling Small at 32k, the Mac Studio M5 Ultra, 256GB at $10,799, holds it at 256k as well.

On the Mac Studio M5 Ultra, 256GB it runs 43 tok/s estimated at 32k and 24 tok/s estimated at 256k. Every token generated reads the whole cache back, so a longer window is slower as well as bigger.

The machines that miss Inkling Small, and what they run

Of the machines here with a published price that are still sold, every one but the Mac Studio M5 Ultra, 256GB misses Inkling Small, and none of them misses by a window: its 163 GB of weights are larger than the usable memory in every one of them, before a single token of key-value cache. The DGX Spark GB10 Grace Blackwell, 128GB comes nearest, 43.2 GB short at 4k, the shortest window on the list. Qwen3.8 27B is the strongest model seven of the eight hold, and it scores 34 on the intelligence index against this model's 26.

Three machines here do hold Inkling Small at 32k, and none of them is an answer to what to buy, which is the question the rest of this page asks: the Mac Studio M3 Ultra, 256GB and the Mac Studio M3 Ultra, 512GB, no longer sold, at the $7,099 and $9,499 they launched at, and the Mac Studio M5 Ultra, 512GB, on sale with no published price. They are sold with 256 GB and 512 GB of memory and hand a model 192 GB to 384 GB of it, against the 164 GB Inkling Small needs at 32k.

One machine per family, the largest configuration in each that still misses, nearest first. Short by is the weights plus the key-value cache at 4k measured against the memory that machine's GPU can address, which that machine's own page gives. The last column is the highest-scoring current model the machine holds at 32k.

Price the DGX Spark GB10 Grace Blackwell, 128GB on Qwen3.8 27B instead

The specifics

Parameters

266B, of which 12B are active per token

Quantisation

UD-Q4_K_M

Weights on disk

163 GB

KV cache

1.0 GB at 32k context — 35 of 42 layers use a 512-token window. Published only as an unsloth dynamic quant, which allocates bits differently from a plain Q4_K_M. How weights and cache add up.

Maximum context

1M tokens (model_max_length; the config states no max_position_embeddings)

Licence

Apache 2.0

Sources

thinkingmachines/Inkling-Small, unsloth/Inkling-Small-GGUF