The MongoDB Podcast

Podbit · The MongoDB Podcast

Modern AIOps: What It Takes to Build Reliable AI Products

Explore episode Jun 5, 2026
Technology
What Is an AI Engineer? Context Engineering Beats Prompt Engineering

Modern AIOps: What It Takes to Build Reliable AI Products · Jun 5, 2026 Technology

An AI engineer isn't just someone who writes good prompts. The more important skill is context engineering — knowing precisely how much information to feed a model, when to feed it, and how to structure it. Beyond that, AI engineers must constantly choose between single-agent and multi-agent architectures, with no established right answer yet.

Where this was said

Key Metrics: What Langtrace Surfaces for LLMs and Vector Databases

At 19:55 · chapter starts 18:02

The exchange here is tightly practical. Jesse prompts Karthik to enumerate what Langtrace actually shows you beyond accuracy scores, and Karthik delivers a comprehensive inventory. For LLM calls: token counts (crucial for cost management), model cost (a major operational concern), time-to-first-token (the perceived latency metric for streaming UIs), and tokens-generated-per-second (the throughput metric for streaming). Model metadata — vendor, name, temperature, top-p, top-k — is captured automatically. For vector databases, including MongoDB Atlas Vector Search, Langtrace traces the pipeline configuration, retrieved results, and optionally the raw embeddings themselves. The embeddings capture enables replay analysis: if a retrieval produced wrong results, developers can re-run the embedding against a modified index without having to reconstruct the original query. Scores on individual traces round out the picture, giving teams a trend line of retrieval quality over time.

Technology
What Langtrace Actually Measures: Tokens, Cost, Latency, and More

Modern AIOps: What It Takes to Build Reliable AI Products · Jun 5, 2026 Technology

Token count, cost per model call, time-to-first-token, and tokens-per-second for streaming — these are the core metrics Langtrace surfaces for every LLM invocation. For vector databases like MongoDB Atlas, Langtrace traces pipeline settings, retrieved results, and even embeddings, enabling replay analysis when something goes wrong in retrieval.

Similar podbits