The a16z Show

Podbit · The a16z Show

The CISO Playbook for AI Agents | Datadog

Explore episode Aug 11, 2026

Where this was said

The AI Judge: Scanning Skills and Code for Malicious Intent

At 10:25 · chapter starts 8:10

The problem of uncontrolled agent skill marketplaces forced Datadog's security team to build something new. The 'judge' — an LLM-based system that evaluates the intent behind a piece of code — grew out of an existing need to review third-party code contributions to the Datadog agent. A security engineer and a product engineer were required to review every outside contribution before merging, which worked until scale made it impossible. The AI judge replaced that bottleneck. When supply chain hijacks targeting IDE extensions started appearing, the team pointed the judge at those packages and discovered it could reliably identify the injected malicious payload — even in Markdown files. Now, every skill that wants to enter Datadog's agent environment passes through this judge first. It's not a blocklist approach; it's an intent evaluation layer. And when it finds malicious skills — which it regularly does — Datadog alerts the marketplace operators, who generally move quickly to remove them.

Similar podbits