Quote · The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
20VC: Will OpenRouter Sell for $10BN to Stripe? | Why Chinese Open Models Are Beating America—and What Happens Next | Why Enterprises Are More Fearful of Anthropic and OpenAI Than China | Is the Routing Layer Becoming a Commodity with Alex Atallah
Where this was said
Is Distillation Wrong—and How Should We Look at It?
At 49:20 · chapter starts 48:57
The distillation debate has generated significant controversy in AI circles, with critics dismissing distilled models as derivative. Alex's response cuts through the noise: distillation is a fundamental model-building technique, not a shortcut or a form of IP theft. [1] — Alex Atallah "Distillation is vilified in public discourse, but Claude Sonnet is a distilled version of Opus. All major labs do it. The real question isn…" 48:17 The closed-weight labs do it constantly — Sonnet is literally a distilled Opus. The legitimate concern is when a company distills a competitor's model to build a directly competitive product, which is why labs have the right to prohibit it in their terms of service. OpenRouter actively helps model labs enforce those terms. For everything else — building smaller, specialised models, doing RL rollouts on open-weight outputs — distillation is not only acceptable but practically superior, because it allows builders to inspect the teacher model's outputs and catch alignment issues before they propagate.
Claude Sonnet is a partially distilled version of Opus, illustrating that even closed-weight frontier labs routinely use distillation to create smaller, cheaper models.
