Ternary quantization: 5x speed increase
Reducing model weights from 16-bit to 3-bit (ternary) results in a 5x improvement in inference speed, in addition to the massive size reduction.
Snapshot · Moonshots with Peter Diamandis
Urgent Update- AI Sputnik Moment: Kimi K3 Released w/ Emad Mostaque | Ep. 272
Explore episode
Jul 19, 2026
Where this was said
Bonsai 27B and On-Device AI: Frontier Intelligence in Your Pocket
At 1:03:40 · chapter starts 1:02:05
Emad Mostaque presents the counterpart to KIMI K3's datacenter scale: PrismML's Bonsai 27B, a US startup from Caltech backed by Khosla Ventures that achieved what was previously considered impossible — running a 27-billion-parameter, GPT-5-class model entirely on a smartphone [1] — Emad Mostaque "PrismML's Bonsai 27B compresses a GPT-5-class 27-billion-parameter model to just 6 gigabytes via ternary quantization — a 5% accuracy drop …" 1:00:40 . The technique is ternary quantization: reducing model weights from 16-bit floating point to three values (approximately 1.58 bits), shrinking the model to 6 gigabytes with only a 5% accuracy drop and 4 gigabytes with a 15% drop. The speed bonus is equally remarkable — ternary is 5x faster than 16-bit. Dave Blundin and Emad Mostaque excitedly note that binary and ternary computation opens the door to entirely new computing substrates beyond CMOS silicon — crystals, photonic chips, anything capable of representing -1, 0, or 1. Tencent's parallel achievement with the HiDream-3 team (formerly WizardLM at Microsoft) achieves binary compression of a 300-billion-parameter model with a 5% performance drop. The endpoint: a device the size of a smartphone running KIMI K3-class intelligence by end of 2027.
27B model runs on smartphone at 6GB
PrismML's Bonsai 27B achieved ternary quantization to compress a 27-billion-parameter model to just 6 gigabytes with only a 5% accuracy drop, enabling it to run entirely on a smartphone.
Similar snapshots
MVP built in one week
Sam's initial MVP was coded in approximately one week using ChatGPT voice mode and copy-pasting code, with no prior technical experience.
Discord vs email for young users
Sam argues Discord is 10x better than email for building relationships with younger users who rarely check their inbox.
Monthly infra costs ~$780–$980
Sam's monthly operating costs include Cursor ($200), AI image generation ($100), AI video generation ($200), hosting ($100), email marketing ($80), and AI compute ($300–$500).
ChatGPT-powered pain point research
Sam recommends copying days of Discord chat history into ChatGPT and prompting it to list recurring pain points as a fast, free market research technique.
50 free tools built total
Bhanu and his team built approximately 50 free tools to attract search traffic, each linked back to SiteGPT.
New tool built in under 5 minutes
With AI coding tools like Cursor, Bhanu can now create a new free marketing tool in less than 5 minutes by referencing existing tools.
KD filter < 10 for rankable keywords
Bhanu filters Ahrefs keyword results to show only those with a keyword difficulty below 10, making them realistic ranking targets for any decent website.
Minimum 1,000 monthly search volume
Bhanu sets a minimum search volume of 1,000 monthly searches when selecting keywords to target with free tools.
20 downloads/day at launch
PropGPT averaged 20 downloads per day right after launching on the App Store through influencer marketing.
