Quote · The MongoDB Podcast
Modern AIOps: What It Takes to Build Reliable AI Products
Where this was said
The Core Problem: Why LLM Products Are Hard to Build Reliably
At 12:30 · chapter starts 10:40
This is where the conversation moves from biography to substance. Karthik systematically dismantles the assumption that AI product development resembles traditional software engineering. Before LLMs, observability was reactive — you built the product, shipped it, and monitored for failures. Errors had logical causes you could reason through. LLMs break this model entirely: the same prompt can yield different outputs at any time, and even an overnight model update can silently degrade a product the developer last tested hours before. [1] — Karthik Kalyanamaran "Before LLMs, software bugs were logical — a missing condition, a stack overflow. You could reason your way to the fix. LLMs are non-determi…" 11:50 Traditional unit tests are useless in this environment because they assume deterministic outputs. Karthik argues that the right mental model is an ongoing operational journey: develop with Langtrace tracing every API call, manually score traces to establish a baseline, ship to production when you have reasonable confidence (not 99% certainty — that's an illusion), then continue monitoring in production and iterating as real users expose edge cases your tests never anticipated. The tooling to support this loop is precisely what Langtrace is designed to provide.
Before LLMs, software bugs were logical — a missing condition, a stack overflow. You could reason your way to the fix. LLMs are non-deterministic: the same input can produce a different output every time, and traditional unit tests are useless against that. Observability is no longer reactive maintenance; it's a required part of the build process from day one.
Achieving 99% accuracy in internal testing does not guarantee the same performance in production because real users will surface edge cases and scenarios internal tests never covered.
Token count, cost per model call, time-to-first-token, and tokens-per-second for streaming — these are the core metrics Langtrace surfaces for every LLM invocation. For vector databases like MongoDB Atlas, Langtrace traces pipeline settings, retrieved results, and even embeddings, enabling replay analysis when something goes wrong in retrieval.
