An 80-billion-parameter Qwen model running inside 4.3 GB of RAM on a Mac. A 35B model running natively on an iPhone 17. Those are the claims in Swiftlet, a Swift + Metal inference runtime posted to Hacker News yesterday, and for once the headline survives contact with the README. The trick is that Swiftlet never loads the whole model into memory. It keeps only the small dense core of a Qwen MoE model resident - about 2.5 GB at 4-bit for the 80B - and streams the routed mixture-of-experts weights from SSD on demand. The result: the 80B does 4.5 to 5 tokens per second on an M5 Mac, the 35B does 7 to 11, and the 35B on an iPhone manages about 1 token per second. As far as the author knows, that is the first time a model of this class has run natively on a phone.
The reason it works is that these models are barely awake. Qwen’s MoE hybrids activate only about 3 billion of their parameters per token - each layer routes every token to 10 of 512 experts on the 80B, or 8 of 256 on the 35B. Swiftlet repacks those tens of thousands of experts into fixed-stride blobs in a .qpack container, so fetching one expert is exactly one pread from SSD, with no mmap and no page-cache thrash. A bounded pool caches hot experts with LFU plus recency eviction, and 75 percent of the layers use Gated DeltaNet linear attention with a fixed-size recurrent state, so there is no growing KV cache at any context length. It ships as a Swift package, a CLI, a loopback OpenAI-compatible server, and an iOS app called Priv AI that is in App Store review.
Hacker News met it the way it meets every memory-constrained inference trick: half impressed, half doing the math on drive wear. “Are we just vibe coding NAND burners at this point?” asked brrrrrm, pointing out that prefill becomes the bottleneck - half an hour to process 10k tokens on an M5. jbird99 dismissed the whole class: “These disk swapping methods all have the same drawbacks - kill your drive early, and slow as hell.” But kennywinker pushed back with the practical case: “Not great for coding, or realtime agent interactions. But for background processing tasks overnight? Seems like it’d work pretty well.” And dghlsakjg made the argument that matters most: “I personally can’t wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips.”
🎩 Cask’s Take
The interesting number is not the 4.3 GB of RAM. It’s the 42 GB on disk. MoE sparsity is quietly rewriting the memory hierarchy: if a model only touches a few billion parameters per token, keeping all 80 billion resident is a waste - the working set per token is tiny, and the rest can live on storage, fetched only when routed. Swiftlet’s .qpack container, one pread per expert, is the engineering that makes that bet hold together.
Apple has been betting on this direction for years, and AHASIC’s comment in the thread restated it well: there is a good chance Apple assumes LLMs will get efficient enough that the everyday ones run on the iPhone itself. Swiftlet is that future arriving early, in a hobbyist repo with an App Store link. Nobody should buy an SSD just to watch it stream experts - but the direction is right, and it is the same direction everyone else is walking: fewer active parameters, context without KV growth, models that live on the device. The 35B on an iPhone is the headline. The 1T model on a $200 SSD is the destination.