Podbit · The a16z Show
Most companies treat evals as an afterthought. Kavak treats them as the foundation. Spending equal engineering effort on evals and agents is what allows Kavak to deploy at massive scale without blowing up — because better brakes mean you can press the gas harder.
Where this was said
At 8:40 · chapter starts 8:10
Gabriel asks how Kavak evaluates whether agents are working at the scale of 96–98% of all interactions. Alejandro's answer reframes the question entirely: evals are not a safety tax, they are the accelerator. His analogy is elegant — you'll only floor the gas if you have good brakes. Most companies go slow on AI deployment because they lack rigorous evaluation, not because the models are weak. Kavak inverts this by spending roughly equal engineering time, tokens, and money on building evals as on building agents themselves [1] — Alejandro Maza Ayala "Equal eng effort on evals vs agents: Kavak spends roughly equal engineering time, tokens, and money on building evals as it does on buildin…" 09:05 . The measurement focus is ruthlessly commercial: did the customer convert? Did they come back? Not vanity metrics like call duration or number of interactions. This discipline, Alejandro argues, is what separates genuine agentic deployment from theatre.
96% of all customer interactions and 95% of all transactions at Kavak are now handled by AI agents. Between 100,000 and 200,000 agent instances wake up every day, each with its own virtual machine, working anywhere from 3 minutes to 3 days before setting an alarm for their next task.
Kavak's AI agents handle 96% of all customer interactions with no human involvement.
95% of all Kavak transactions are completed end-to-end by AI agents, not humans.
Between 100,000 and 200,000 unique agent instances are spun up at Kavak every single day, each with its own virtual machine.
Kavak spends roughly equal engineering time, tokens, and money on building evals as it does on building agents themselves.
Kavak's AI sales agents now convert customers at 2.1x the rate of its human team, and also tripled NPS. The agents handle everything from car recommendations to financing to trade-in quotes — a 15-skill job that no single human could match.
The fastest path to Twitter growth isn't volume — it's forming sharp opinions about how the platform works and sharing them immediately. People cluster around those who understand the rules and say so out loud.
Sam had no coding knowledge, so he used ChatGPT voice mode to generate his entire codebase and copy-pasted it into Notepad. A friend later introduced him to Cursor, and he never looked back.
Copy days of Discord chat history, paste it into ChatGPT, and ask it to list recurring pain points. The ones that come up most often are your best product bets.
Sam's top advice: when prompting Cursor, tell it to architect code for 100,000 users from day one. The AI changes its approach, building scalable frameworks instead of brittle one-user code.
With AI coding tools like Cursor, Bhanu replicates an existing free tool for a new keyword in under 5 minutes. What used to be a multi-day build is now a lunch-break task.
Ahrefs, SiteGPT, Cal.com, PostHog, Datafast, Sibyl AI, Bento, Feather, Featurepace, Mintlify, Cloud Code, ChartMogul — Bhanu runs his entire business solo with these 12 tools.
PropGPT runs on React Native with TypeScript and Python for ML, Neon for the database, RevenueCat and Superwall for monetization. LLM costs are $20/month, data APIs $100/month, and after $10K in monthly marketing spend, margins sit at roughly 50%.
Automated food production cheaper than grocery shopping. Mining companies that own nothing but real estate. Kalanick believes these are near-certainties, not moonshots, and they dwarf anything digital AI has done to enterprise software.
Meraki couldn't beat Cisco on reputation in 2009. So they stopped trying to pitch and started shipping. Attend a webinar, get a free access point. Once mid-market IT teams experienced cloud-managed networking, no argument was needed — the product sold itself.
We use essential and analytics cookies to run Vuci. To understand how the site is used: Privacy Policy.
Install Vuci on your phone
Add it to your home screen for a faster, app-like experience.
Install Vuci on your phone
Tap the Share button, then “Add to Home Screen”.
A new version is available
Reload to get the latest Vuci.