Where this was said
Evals: The Brakes That Let You Go Fast
At 8:47 · chapter starts 8:10
Gabriel asks how Kavak evaluates whether agents are working at the scale of 96–98% of all interactions. Alejandro's answer reframes the question entirely: evals are not a safety tax, they are the accelerator. His analogy is elegant — you'll only floor the gas if you have good brakes. Most companies go slow on AI deployment because they lack rigorous evaluation, not because the models are weak. Kavak inverts this by spending roughly equal engineering time, tokens, and money on building evals as on building agents themselves [1] — Alejandro Maza Ayala "Equal eng effort on evals vs agents: Kavak spends roughly equal engineering time, tokens, and money on building evals as it does on buildin…" 09:05 . The measurement focus is ruthlessly commercial: did the customer convert? Did they come back? Not vanity metrics like call duration or number of interactions. This discipline, Alejandro argues, is what separates genuine agentic deployment from theatre.
96% of all customer interactions and 95% of all transactions at Kavak are now handled by AI agents. Between 100,000 and 200,000 agent instances wake up every day, each with its own virtual machine, working anywhere from 3 minutes to 3 days before setting an alarm for their next task.
Kavak's AI agents handle 96% of all customer interactions with no human involvement.
95% of all Kavak transactions are completed end-to-end by AI agents, not humans.
Between 100,000 and 200,000 unique agent instances are spun up at Kavak every single day, each with its own virtual machine.
Most companies treat evals as an afterthought. Kavak treats them as the foundation. Spending equal engineering effort on evals and agents is what allows Kavak to deploy at massive scale without blowing up — because better brakes mean you can press the gas harder.
Kavak spends roughly equal engineering time, tokens, and money on building evals as it does on building agents themselves.
Kavak's AI sales agents now convert customers at 2.1x the rate of its human team, and also tripled NPS. The agents handle everything from car recommendations to financing to trade-in quotes — a 15-skill job that no single human could match.
