A model fits in 16 GB. Does the workload?

The claim that 16 GB is all you need for serious local AI is true about loading a model and incomplete about finishing real work. Weights and the KV cache share one pool of memory, and an agent workload spends that pool faster than a demo reveals [S1 1:37].

The fit is real, and it is only half

Qwen 3.8 27B did not fit a 16 GB card before: its standard 4-bit build is 17.44 GB [S1 2:02]. DasLab, out of the Institute of Science and Technology Austria, released GSQ-RCO quantization, which compresses sensitive and insensitive parts differently — rolling the t-shirts, folding the dress shirt [S1 2:21]. The result, an 11.8 GB build the lab calls "Task Lossless," ties the full model's 30/30 math score and coding results and misses one extra question of 198 on science [S1 2:46]. Manolo Remidi's demo showed 15.1–15.9 GB in use on a $429 RTX 5060 Ti at 45.6 tokens a second, while a 5090 on the identical file was set to 262,000 tokens of context against the 5060 Ti's 64,000 [S1 0:53].

Weights are fixed; the conversation grows

Two things live on the card — the weights, which are the model, and the KV cache, which is the conversation [S1 4:48]. Weights are fixed — 11.8 GB stays 11.8 GB — while the cache grows with every token read or written [S1 5:01]. On Qwen 3.8 27B that is 64 KB per token: 32,000 tokens is 2 GB, and 128,000 is 8 GB, nearly the model's own size [S1 5:16]. Mistral's DevStraw Small 2 uses 160 KB per token, leaving roughly 11,000 tokens beside it [S1 5:39]. Agent tools spend the budget before work starts: Cloud Code loads 30,000–32,000-token system prompts before you type, leaving 34,000 of a 64,000-token window [S1 7:11]. Luke's dev lab measured a Kanban app at 87% and a Blender task at 96% of a 128,000-token window, filled mostly by tool output rather than your messages [S1 7:34].

GB is not GiB, and bandwidth sets the speed

The source’s memory-unit wording needs care: 16 GiB equals about 17.18 decimal GB, whereas 16 decimal GB equals about 14.9 GiB. The following capacity estimates are the presenter’s reported figures, not independently verified specifications; llama.cpp reserves roughly 1 GB, and the prediction head (0.35 GB) plus the auto-loading vision file (0.93 GB) leave near 14 GB before a conversation — about 47,000 tokens at full cache precision [S1 6:09]. Disabling vision and compressing the cache to 8-bit roughly doubles that to 110,000 [S1 6:42]. Speed is a separate ceiling: generating one token reads the whole model once, so tokens per second is bounded by memory bandwidth divided by file size [S1 9:25]. A 5060 Ti moves 448 GB/s, capping a 10.93 GB 3-bit file at 41 tokens a second; multi-token prediction lifts one user from 27.4 to 45.6, but consumes cache and is switched off as context fills [S1 9:37]. Prefill is the wait rarely quoted: the model must read the whole prompt before its first word, so the 5060 Ti's roughly 738 tokens a second means a fresh 30,000-token agent context waits 41 seconds; an RX 9060 XT measured 149 tokens a second at 32,000 tokens [S1 10:44].

Prices, labels, and what the workload asks

The RX 9070 XT at $790 moves 640 GB/s against the RTX 5060 Ti's 448 GB/s at $789 — 43% more bandwidth for the same price [S1 11:52]. NVIDIA's prefill advantage on llama.cpp is still real: a 5070 Ti read a 32,000-token prompt over three times faster than an RX 9070 XT while writing under 10% faster [S1 12:16]. The 5060 Ti has drifted from its $429 sticker to $789, the 16 GB version up 84% while the 8 GB is up 24% [S1 12:38]. A "16 GB" Mac is not a 16 GB GPU: macOS grants graphics 12.7 GB by default, leaving under a gig and roughly 14,000 tokens after an 11.8 GB model [S1 12:53]. Past 100,000 tokens, 16 GB reaches its limit only with 3-bit weights, compressed cache, prediction off, and speed near 15 tokens a second [S1 15:28]. The cheapest upgrade is a second 16 GB card: two 5060 Ti 16 GBs cost about $1,578, and one owner went from 27 to 46 tokens a second switching from layer split to tensor split [S1 15:49].

For short-to-medium context, daily use, 16 GB is genuinely enough, and the demo and numbers hold up [S1 17:50]. But enough to load and enough to finish the job are two different numbers [S1 18:05]. Size the model for the conversation, not the conversation for the model [S1 17:44].

Scope note. Coverage rests on a single commentary video (S1), which relays other creators' figures — Manolo Remidi's demo, Luke's dev lab's measurements, owners' bandwidth tests, and unnamed benchmark comparisons — so those accounts are not independent corroboration of one another. The "Task Lossless" label and its benchmark parity are the quantizing lab's claims, and prices are as of the source's publication week. The transcript is automatic, so names and numbers may be mis-transcribed; the model name follows a corrected transcription, while other proper names are left as transcribed.