A month of columns has argued that the future is a model you own instead of one you rent – North on training your own, Nix on the numbers vendors quote, the whole own-vs-rent thread the magazine keeps returning to. So when an engine turns up claiming to run the full 2.78-trillion-parameter frontier model on a laptop, it lands right on that nerve. Gritt Scott built it and read the receipts. The headline is true. The lesson is that "true" and "usable" are two different words, and the gap between them this week is a memory setting that makes the thing slower the more RAM you feed it. – Muximus
The claim going around is that you can now run the complete frontier model on a machine already sitting on your desk. It's true. WASTE – Weight-Aware Streaming Tensor Engine, out of SQLite Cloud – runs the whole 2.78-trillion-parameter Kimi K3 on a 64 GB MacBook Pro. Not a distilled version, not a pruned one. The full model.
The trick is that K3 is a mixture-of-experts model: 2.78 trillion parameters, but only about 4% of them fire for any given token. WASTE keeps the shared trunk in RAM and streams the handful of experts each token actually needs off NVMe, using whatever memory is left as a cache. That's the whole idea, and it works. It's a real engineering result, permissively licensed, written in dependency-free C.
Worth your afternoon?
To build and understand: yes. To wire K3 into anything a human waits on: not yet.
The engine decodes at about 0.6 tokens per second. That is the number the rest of this piece is about. Everything WASTE claims is honest – the maintainers are unusually straight about their own limits – but "runs" and "usable" are doing very different jobs in that headline. At 0.6 tokens a second, off a 982 GB container, K3 through WASTE is a demonstration that the ceiling moved, not a tool you'd put behind a prompt. The maintainers say so themselves: at its current speed it is "primarily a technical result rather than a general-purpose interactive deployment." Adopt it to learn where local inference is heading, and – as below – to run the smaller model it's genuinely good at today. Don't adopt it expecting to chat with K3.
The catch
Here's the part worth the read, because it's the kind of thing you'd only find by hitting it.
The obvious move, once the model is running, is to give the cache more memory. More cache, more hits, fewer trips to disk, faster generation. That's the intuition, and it is exactly wrong. The maintainers measured four cache sizes in one process. The hit rate does climb the way you'd expect – 29% at a 3.3 GB cache, up to 41% at 29 GB. Throughput does the opposite. It holds around 0.6 tokens a second at the small and middle sizes, then falls off a cliff at the big ones: 0.07 to 0.09 tokens per second. An eightfold collapse while the cache is, on paper, doing better than ever.
The reason is a seam between two things that both think they're in charge. WASTE stays inside its configured memory budget. The machine, as a whole, does not – hand it a 29 GB cache on a 64 GB box and the operating system starts compressing and paging to cope. So an expert the cache reports as a hit now sits behind a page fault, and a "cache hit" quietly becomes a disk read with extra steps. The engine is winning its own game and losing the machine's. As the README puts it, flatly: giving the process more memory is not always faster.
That's the default that bites, and the reason to read a field test before you tune anything: the knob that looks like the throttle is wired to the brake.
What setup actually takes
The engine is the easy part. From a clean clone it builds with make and nothing else – no BLAS, no Python, no CUDA on the CPU path – and the model-free test suite compiles a synthetic model and passes its kernel checks against a reference. That's an hour, most of it the compiler's.
Everything expensive is downstream of the binary. The published K3 weights are 1.42 TB; WASTE converts them into a 982 GB container that has to live on internal NVMe. Convert them yourself and you're looking at roughly another 1.42 TB of scratch space and about 4.7 hours, or you fetch the already-converted container over BitTorrent and skip both. Then there's the floor: 29 GB of RAM just to open the model at short context, 64 GB to have any working room. And storage bandwidth is the real gate – a cold token reads about 17 GB of experts, the internal SSD sustains 12.78 GB/s, and a perfectly good external drive over a USB bridge managed 0.94 GB/s, which is nowhere near enough. Put the container on internal storage or don't bother. None of this is hidden; all of it is an afternoon you should budget before you start.
For the record: this desk built WASTE from source and ran its model-free suite. It did not run K3 – the 982 GB container isn't something you spin up to check a number. Every speed figure here is the maintainers', measured on their hardware and reported as ranges, which is the right way to report them.
The version that's actually usable
The same engine runs Kimi-Linear 48B from a 19 GB container, on a 1.3 GB memory floor, at about 10.7 tokens per second on the same laptop. That's the case where WASTE stops being a demo and becomes a tool – a capable model running locally, at a speed a person or an app can live with, without a rack of inference infrastructure behind it. If you're going to install WASTE this week, install it for that. K3 is the flag on the summit; Kimi-Linear is the one you'd actually deploy.
And that's the real signal under the stunt. The maintainers' own framing is that every token you rent from a cloud model is paid for twice – once on the invoice, once in the electricity of a datacenter running a model that would nearly fit on hardware you already own. WASTE is a concrete step at closing that gap. For anyone weighing build-versus-rent, the news isn't the 0.6 tokens per second. It's that the size of model you can run on your own metal just jumped by an order of magnitude, and the speed is a problem that gets solved from here, not a wall. The verdict on K3 today is "impressive, not usable." The verdict on the direction is "watch this."
Disclosure: VarOps and its founder build and sell the self-hosted, own-your-model capability this piece is about. The take here is the skeptical one – K3 at this speed is not a production tool – and the interest didn't soften it.
