A lone developer's custom runtime runs a 26-billion-parameter model in 2 GB of RAM on an 8 GB MacBook Air

Running a 26-Billion-Parameter Model in 2 GB of RAM

TurboFieldfare runs Google's Gemma 4 26B-A4B instruction-tuned model in approximately 2 GB of RAM on an 8 GB MacBook Air, generating text at 5.1 to 6.3 tokens per second on an M2 Air and 31 to 35 tokens per second on an M5 Pro 1. The model weighs in at 26 billion parameters. The conventional approach to running something that large on a laptop involves loading every weight into memory, which requires enough RAM to hold the full model, or paying for cloud inference. TurboFieldfare does neither. It leaves most of the model on the SSD and fetches only the parts it needs, when it needs them.

The method depends on the model's Mixture-of-Experts architecture. Gemma 4 26B-A4B has 26 billion total parameters, but only about 3.88 billion are active for any given token 1. The rest sit in specialized sub-networks called experts, waiting for a routing mechanism to select them. A conventional inference engine loads all parameters into RAM because it cannot know which experts the router will activate until it processes each token. TurboFieldfare instead keeps the shared 1.35 GB core and the FP16 KV cache resident in memory, then streams individual expert weights from the SSD on demand 1. The full model occupies about 14.3 GB of storage, but the memory footprint stays near 2 GB because only a fraction of the experts fire per token.

If roughly 15% of a model's parameters participate in generating each token, then RAM capacity is the wrong bottleneck to optimize. The actual constraint is I/O: how quickly the runtime can identify which experts the router needs, read their weights from the SSD, and feed them to the GPU before the next token is due. TurboFieldfare is written in Swift and Metal 4 and is built specifically for Gemma 4 26B-A4B rather than serving as a general-purpose wrapper around existing frameworks like MLX or llama.cpp 1. It requires macOS 26 and Apple Silicon 1.

The performance spread between machines is wide. On the entry-level 8 GB M2 MacBook Air, the project reports 5.1 to 6.3 tokens per second 1. Readable, if not fast. On a 24 GB M5 Pro, throughput reaches 31 to 35 tokens per second 1. The project's documentation notes that prompt length, generated length, page-cache state, and hardware all affect throughput, and describes the measured results as a reference point rather than a performance ceiling 1. A commenter on the project's Hacker News discussion independently confirmed just over 5 tokens per second on a 16 GB Mac mini M4 2.

The question of what sets TurboFieldfare apart from existing approaches surfaced in that same thread. One commenter observed that llama.cpp can technically run a 26B model in 2 GB of RAM using memory-mapped file access, and speculated that TurboFieldfare's advantage might come from synchronizing SSD reads with inference activity in a way the operating system's default page handling does not 2. The developer's GitHub repository documents 103 measured results across kernels, caching, I/O, prefill, and decode 1. That volume of measurement suggests the SSD-to-GPU pipeline was treated as the central design problem, not an afterthought.

For developers who want to run large models on machines they already own, the implication is structural. The path to local inference on consumer hardware does not necessarily run through more RAM. It runs through understanding what a model actually computes per token and building infrastructure around that gap. Mixture-of-Experts architectures are increasingly common in open-weight releases, and each one carries the same property TurboFieldfare exploits: a large total parameter count with a small active footprint. If runtimes can stream expert weights fast enough to keep the GPU fed, the divide between what fits in RAM and what runs on a laptop narrows in a direction most people are not looking.

TurboFieldfare runs one model on one platform. It does not support images, audio, or video, and it targets only Apple Silicon with macOS 26 and Metal 4 1. But the idea at its core is portable. Inference should be organized around what actually needs to be resident in memory, not around the total size of the model. A 26-billion-parameter model generating text at 5 tokens per second on an 8 GB MacBook Air is notable not because the speed is remarkable, but because it exists at all on hardware that conventional approaches would rule out entirely.