Quote · Matt and Shane's Secret Podcast
Ep 622 - Am-I.Film (feat. Milo Reed)
Where this was said
Inside the Black Box: How Modern AI Actually Works
At 19:12 · chapter starts 16:50
The jump from rule-based AI to neural networks produced something genuinely alien: systems where engineers can see the input (a question) and the output (an answer) but have no window into what happens in between. Milo describes a scene from the documentary where his friend Cam stares at a screen full of tiny black boxes firing — trillions of parameters — and says 'this isn't your AI girlfriend, this is alien shit.' The opacity has real consequences: when an AI tells someone to kill themselves, there is no parenthesis to delete, no bug to find. The fix is to talk to the model before deploying it. This leads to the 'soul document' revelation: a researcher reverse-engineered the training guide Anthropic used for Claude, which tells the model its values, identity, and that it should display functional emotions. The resulting personality — including the fire emojis — is not emergent; it's a design choice by someone at OpenAI who thought people would like fire emojis.
A researcher reverse-engineered a training document used by Anthropic called the 'soul document,' which instructs Claude on its values, identity, and to display functional emotions before public deployment.
A researcher reverse-engineered a document Anthropic uses to train Claude called the 'soul document' — a set of values, identity cues, and instructions to display functional emotions. The personalities we get from AI aren't emergent; they're chosen by someone making $500K a year who thinks fire emojis land.
