Quote · The MongoDB Podcast
Don't Build Your Own AI (Unless You Have To)
Where this was said
Architectural Pillars for Production-Grade AI Products
At 39:20 · chapter starts 38:30
Moving from strategy to implementation, Akshaya gives his clearest technical prescription of the episode. Clean data plumbing comes first: pipelines, a single source of truth, and PII scrubbing — because 'garbage in, hype out' and even the perfect transformer model will break on bad data. [1] — Akshaya Murthy "I say garbage in, hype out with AI. So bad data even breaks the perfect models. The transformer works, but your data is terrible." 38:50 Second, build a modular API-first model layer. Just as you wouldn't lock customers into a single ISP, you cannot lock them into a single LLM vendor. Containerised, microservice-sized model infrastructure lets you hot-swap models as the landscape evolves, giving both you and your customers optionality. Third, bring CI/CD discipline to model management: automated regression tests when you swap a model, one-click rollbacks when behaviour deviates from baselines — treating model changes with the same rigour as code releases. Fourth, invest in monitoring and drift detection so that six months of reinforcement learning doesn't silently push the model into a problem space it was never meant to solve. And throughout all of it, keep a human in the loop: the best learnings from model outputs should feed back into continuous improvement. Shane draws a parallel to MongoDB's MAP (MongoDB AI Applications Program) composable reference architecture, which follows the same hot-swap logic across LLMs and orchestration frameworks.
The architecture decisions that will make or break your AI product: clean data pipelines, a modular API-first model layer that lets you hot-swap LLMs, CI/CD for models themselves, and continuous monitoring with human review. These aren't nice-to-haves — they're the difference between a product and a liability.
Responsible AI isn't a checkbox at the end of the build cycle. The NIST govern-map-measure-manage framework needs to run through every phase: pre-training, training, post-training, delivery, and application. Bias and privacy must be by design, not bolt-ons.
