Bonsai 27B: The Largest AI Model That Runs on Your Phone

For the last two years, the rule for AI on a phone was simple: keep it small. Three billion parameters, maybe four. Anything bigger wouldn't fit in memory — and if it did, it crawled. A 27-billion-parameter model, the kind you'd normally run on a desktop with a discrete GPU, was firmly out of reach.

On 14 July 2026, PrismML broke that rule. Bonsai 27B is a full 27B multimodal model compressed down to 3.9 GB — small enough to load and run on an iPhone. It is, as far as anyone can tell, the largest model of its class ever to run on a phone.

Here's what PrismML actually built, how the trick works, how much intelligence survives the squeeze, and whether you'd actually want to run it.

What PrismML shipped

Bonsai 27B isn't a new model trained from scratch. It's an extreme low-bit rebuild of Qwen3.6-27B — Alibaba's strong open 27B multimodal model. The architecture is left untouched; what changes is how each weight is stored. Instead of the usual 16 bits — or even the 4 bits of a typical Q4_K_M quant — Bonsai stores each weight in roughly one to two bits.

It comes in two builds:

Retention of the full-precision (FP16) baseline across PrismML's 15-benchmark suite, as reported at launch.

Both are multimodal — they accept images as well as text — both carry a 262,144-token context window, and both are released for free under the Apache 2.0 license. You can find both on the Bonsai 27B model page.

Why 3.9 GB is the whole story

A 27B model at full precision (FP16) is about 54 GB of weights. Even quantized to 4 bits — the standard for local models — it's around 16 GB, more RAM than most phones have. Bonsai's low-bit builds change the maths entirely:

The 1-bit build lands at 3.9 GB. A modern flagship phone with 8–12 GB of RAM has room for it, with headroom left for the OS and the app. That's the entire reason this is news: not that the model is clever — it's Qwen underneath — but that a serious 27B model now fits.

How you squeeze 27B into a pocket

Two ideas do most of the work.

1-bit and ternary weights. Most quantization stops at 4 bits because going lower usually wrecks the model. PrismML pushes to the theoretical floor: the ternary build stores each weight as one of three values (−1, 0, +1); the 1-bit build uses just two (−1, +1). That's ~1.1–1.7 bits per weight versus 16 — a 10–14× shrink. What makes it work is the conversion itself: a low-bit-aware method that protects the handful of weights that matter most, so accuracy degrades gracefully instead of collapsing. A split-precision design. Not every part of the network is treated equally. Per PrismML's breakdown, Bonsai 27B is roughly 24.8B language weights (the part taken down to 1–1.7 bits), a 0.46B vision tower kept separately at 4-bit, and 2.5B of embeddings and the output head. Holding the vision tower and embeddings at higher precision is what preserves image understanding and output quality while the bulk of the network goes ultra-low-bit.

There's also a neat reason the 262K context stays practical on a phone: about 75% of Qwen3.6-27B's attention is linear, so the memory cost of a long context grows far more slowly than in a classic transformer. Long documents don't blow up your RAM budget.

How much intelligence survives?

This is the real question with any aggressive quantization. PrismML's answer, across a 15-benchmark suite spanning knowledge, reasoning, math, coding, instruction-following, tool-calling and vision:

• Ternary Bonsai 27B keeps more than 95% of the full-precision baseline.
• 1-bit Bonsai 27B keeps more than 90%.

The reported scores back that up on the reasoning and coding side:

Those are genuinely strong numbers for something that fits on a phone — the ternary build is essentially a laptop-grade reasoning-and-coding model at 5.9 GB.

But there's an honest catch, and it's worth stating plainly. Not every skill compresses equally. Tool-calling — reliably emitting structured function calls, the backbone of agentic workflows — drops from 80.0 at full precision to about 66.0 at 1-bit. PrismML's own analysis notes it degrades several times harder than math does. The ternary build holds up much better (~74). So if your use case is agents and tool use, reach for ternary; the 1-bit phone build is best thought of as a chat, reasoning and drafting model, not an agent.

Is it fast enough?

Small enough to fit is one thing; usable speed is another. PrismML's reported decode speeds for the 1-bit build:

Around 7–10 tokens per second is the point where text generates about as fast as you comfortably read. The iPhone's ~11 tok/s clears that bar — not blazing, but genuinely usable for a real conversation. On a laptop-class M5 Max it's snappy, and on a desktop GPU it's fast enough that the low-bit build becomes an efficiency play rather than a necessity.

Which build should you run — and how

• On a phone: the 1-bit build (3.9 GB). Great for offline chat, reasoning, writing and summarization. Skip it for heavy tool-use.
• On a laptop or desktop: the ternary build (5.9 GB). It's the daily driver — near-full-precision quality with tool-use intact.

Both ship as GGUF (for llama.cpp, LM Studio, and most desktop runners) and MLX (for Apple Silicon). The ternary build is also available as a hosted API on Together AI if you'd rather not run it locally. On a phone, an app like LM Studio's mobile client or PocketPal is the easiest way in — see our guide on running a local LLM on your phone for the step-by-step.

Not sure your device can handle it? Check with the Hardware Analyzer →.

A couple of caveats

• These are launch-day, vendor-reported numbers. PrismML published the benchmarks; independent replication will take a little time. Treat the exact figures as "reported," not yet independently verified.
• It's still a 3.9 GB model on a phone. Expect the handset to get warm and the battery to drain faster during sustained generation — the same as any on-device LLM.
• 1-bit is a real trade. More than 90% retention is remarkable, but "more than 90%" still means measurably weaker than the ternary or full-precision versions — most visibly on tool-calling. Match the build to the job.

The bottom line

A year ago, the sensible ceiling for phone AI was a 3–4B model, and the story was "surprisingly good for its size." Bonsai 27B moves that ceiling by nearly an order of magnitude: a 27B multimodal model, with a 262K context and laptop-grade reasoning scores, running offline in under 4 GB. The clever part isn't a new model — it's the low-bit engineering that makes an existing strong model portable.

The direction of travel is clear. The gap between "what runs in a datacenter" and "what runs in your pocket" just narrowed dramatically — and because it's Apache-2.0 licensed, anyone can build on it.

---

Explore the specs and both builds on the Bonsai 27B model page → New to on-device AI? Start with Can You Run a Local LLM on Your Phone? → Browse more phone-friendly models in the Model Library →