Soft label distillation entropy advantage
Distillation from soft targets provides far more bits per sample than one-hot labels because the entropy of a soft distribution is vastly higher.
Where this was said
Automated AI researchers
At 2:18:16 · chapter starts 2:16:16
Eric reports from his AlphaGo project: LLMs excel at hyperparameter tuning and executing experiments, but fail at lateral thinking — choosing the right question and recognizing dead-end research tracks. [1] — Eric Jang "LLMs can autonomously run experiments, tune hyperparameters, and optimize code. What they can't do: decide which question to investigate ne…" 2:14:00
Go as a Research Environment for AI
Go has a fast, unambiguous outer loop (win rate vs. KataGo) and contains subtasks that overlap with LLMs and robotics — distributed systems, hyperparameter search, hypothesis generation. It's a cheap, verifiable testbed for automated AI research.
Similar snapshots
MVP built in one week
Sam's initial MVP was coded in approximately one week using ChatGPT voice mode and copy-pasting code, with no prior technical experience.
Discord vs email for young users
Sam argues Discord is 10x better than email for building relationships with younger users who rarely check their inbox.
Monthly infra costs ~$780–$980
Sam's monthly operating costs include Cursor ($200), AI image generation ($100), AI video generation ($200), hosting ($100), email marketing ($80), and AI compute ($300–$500).
ChatGPT-powered pain point research
Sam recommends copying days of Discord chat history into ChatGPT and prompting it to list recurring pain points as a fast, free market research technique.
50 free tools built total
Bhanu and his team built approximately 50 free tools to attract search traffic, each linked back to SiteGPT.
New tool built in under 5 minutes
With AI coding tools like Cursor, Bhanu can now create a new free marketing tool in less than 5 minutes by referencing existing tools.
KD filter < 10 for rankable keywords
Bhanu filters Ahrefs keyword results to show only those with a keyword difficulty below 10, making them realistic ranking targets for any decent website.
Minimum 1,000 monthly search volume
Bhanu sets a minimum search volume of 1,000 monthly searches when selecting keywords to target with free tools.
20 downloads/day at launch
PropGPT averaged 20 downloads per day right after launching on the App Store through influencer marketing.
