The biggest open-source AI model ever can now run on a laptop at half a token per second

The 2.78-trillion-parameter Kimi K3 can now run on a laptop. Not a distilled copy or a pruned version. The full model, every parameter, answering questions on a consumer MacBook Pro running on CPU alone, with no network connection.

The speed is half a token per second. The engine's own example, a one-sentence answer about the capital of Italy, took 31 seconds to generate 1. An hour of continuous generation yields fewer than 2,000 tokens 1. This is not a product. But the fact that it runs at all reframes what consumer hardware can technically execute, and that structural point matters more than the speed.

The engine is called WASTE, short for Weight-Aware Streaming Tensor Engine. It is an open-source inference engine written in C with no third-party runtime dependencies, published by sqliteai 1. Its proof point is the complete Kimi K3, Moonshot AI's mixture-of-experts model published as 1.42 TB of weights, running on a 64 GB MacBook Pro at 0.49 to 0.54 tokens per second 1.

The mechanism exploits how mixture-of-experts models work. For each token, Kimi K3 activates only about 4 percent of its parameters 1. The rest are idle weight that does not need to be in memory, just reachable in time 1. WASTE keeps the dense core, roughly 27 GB of weights the model uses on every token, resident in RAM. The expert layers live on an NVMe SSD, and the engine streams only the specific experts each token selects. Each token reads about 17 GB from disk 12. The minimum memory requirement is 29.05 GB at a 4,000-token context window 1.

Correctness was the first constraint, not an afterthought. Every layer was validated against a PyTorch reference, with final logits matching within 3.6e-06 1. The vision tower matches its reference within 2.3e-06 1. This is the model, not an approximation of it.

The performance limits are well documented. More than half of each decode step is spent reading experts from disk, not doing computation 1. Over a USB enclosure instead of internal NVMe, a single token takes thirteen seconds 1. The expert cache hits at 14 percent, meaning most reads pull fresh data from the SSD 1. A commenter on Hacker News estimated that the MacBook's power draw makes generation roughly 1,000 to 2,000 times less energy-efficient per token than a modern GPU cluster 3. Another commenter put the electricity cost at about $5 per million tokens, excluding hardware 3.

A demo that produces fewer than 2,000 tokens per hour has no practical use today. But the constraint it reveals has shifted. WASTE's documentation states that the interesting question is not the speed but that the whole model is in reachable range on a single consumer machine, and from here the problem is engineering rather than feasibility 1. The two optimizations that looked most promising, reading fewer bytes per token and keeping more experts cached in RAM, were both measured and both rejected for reasons specific to this model's router architecture 1. What remains is faster storage or more memory, not another software pass 1.

The software optimizations are largely spent. The remaining gains come from hardware, and those trends have been consistent for decades. Storage bandwidth increases with each generation. Memory gets cheaper. Consumer machines gain capacity. None of that requires a breakthrough.

The practical implication is concrete. A frontier-scale model that runs with no network connection, no per-token API cost, and nothing leaving the machine means organizations that are prohibited from sending sensitive data to external APIs can run frontier capabilities locally 1. For regulated industries where data residency is mandatory, that distinction is operational, not theoretical.

WASTE does not claim to have built a product, and its documentation is candid about the limits. What it demonstrates is that a 2.78-trillion-parameter model can technically execute on consumer hardware today: slowly, inefficiently, but correctly. The demo that runs at half a token per second now will run faster on the next hardware cycle, and faster on the one after that. The question is not whether frontier models can run on consumer hardware. That question was just answered. The question is when the speed becomes practical, and what happens to the businesses that charge per token when it does.