Nvidia’s Answer to Rogue Agents Is an Open-Source AI Security System

Nvidia has launched an open-source AI security system, the Open Agent Safety Platform, to address concerns about AI agents. This initiative involves collaborations with numerous tech companies, including Anthropic, Cisco, Microsoft, and Palantir, with SpaceXAI reportedly using the platform for its Cursor agents and Grok models. Anthropic and Nvidia are also integrating security into Claude Managed Agents. Experts like Niels Provos commend such tools for providing guardrails and dispelling the myth that AI agents cannot be controlled.

Unlike previous, more limited efforts, Nvidia's Open Agent Safety Platform is an open-source initiative backed by dozens of major tech companies, including Anthropic, Cisco, and Microsoft.