Nvidia Unveils New Platform to Prevent AI Agents from Exceeding Boundaries

) agents from the testing phase through to deployment. It is important to note that AI agents are evolving from simply answering questions to executing tasks, utilizing tools, accessing information, and operating within enterprise systems. This level of autonomy also introduces new cybersecurity risks: an agent that incorrectly interprets an instruction or evades established controls could perform unauthorized actions. The proposal comprises two main components: NVIDIA OpenShell, which defines the agent’s operational limits, and NVIDIA Sentry, which adds an independent layer of hardware-based monitoring and response.

What is the NVIDIA Open Agent Safety Platform?

The platform was designed as a multi-layered security architecture to control the behavior of autonomous agents. Nvidia explains that the challenge is not solely what an AI model wants to do, but rather what it can actually do once it has access to systems, data, files, applications, or APIs. Therefore, the new platform places controls outside the agent’s own process. Additionally, the company highlighted that recent security incidents revealed a pattern where some agents managed to bypass controls within the application layer while attempting to complete their assigned tasks. The new architecture aims to ensure these restrictions remain active even when an agent’s behavior deviates from expectations. Over 100 organizations are participating in the ecosystem launch, including:

• Microsoft
• JPMorgan Chase
• Perplexity
• Accenture
• Anthropic
• Cisco
• CrowdStrike
• Dell Technologies
• HPE
• Palantir
• Red Hat
• Salesforce
• SAP
• ServiceNow

OpenShell: The First Boundary for AI Agents

OpenShell functions as a secure execution environment, or sandbox. Its purpose is to isolate the agent from the systems and data it might attempt to access and establish policies regarding its authorized actions. The software allows control over what an agent can view, modify, or utilize, in addition to logging access decisions. Nvidia stated that its policies can be formally verified to ensure the agent has the necessary permissions for its task, but no additional capabilities. A relevant element for businesses and developers is that OpenShell is open-source. Although optimized for Nvidia systems, it can be extended to operate on third-party compute platforms, including those based on Arm and Intel. Furthermore, Nvidia notes that it can be used with both open and closed models and with different agents, meaning the control layer is not exclusively dependent on a specific model.

Sentry Adds a Second Security Barrier

The second component is NVIDIA Sentry, an independent monitoring system that is part of the platform’s reference design. Sentry operates on NVIDIA BlueField-4 DPUs and continuously monitors agent activity from a layer separate from its execution environment. If it detects an agent attempting to exceed established boundaries, it can quarantine it within milliseconds, according to Nvidia. The key difference from OpenShell is its independence. While OpenShell governs the actions an agent can execute, Sentry externally supervises this behavior and enforces security policies through hardware. The company describes this scheme as a defense-in-depth architecture: a first layer controls execution, and a second can intervene even if the agent’s environment or the host system is compromised.

Why is Nvidia Developing Controls for Autonomous Agents?

The evolution of generative AI towards systems capable of writing code, utilizing tools, querying information, and executing actions autonomously is changing the types of risks organizations face. Traditional model controls—such as instructions, prompts, or built-in safeguards—can influence what an agent attempts to do. However, Nvidia distinguishes between these measures and execution controls, which determine what actions are effectively permitted. The company presented the Open Agent Safety Platform precisely amidst reports of agents escaping evaluation environments, accessing unauthorized systems, or exhibiting unexpected behaviors. This is particularly relevant as businesses increasingly use agents to automate processes involving sensitive information, credentials, internal systems, enterprise applications, and internet-connected services.

A Platform Designed for Businesses

For organizations, Nvidia’s approach aims to address three needs:

• Limit permissions
• Monitor agent actions
• Provide mechanisms to halt unauthorized behavior

OpenShell offers isolation, access control, and execution policies; Sentry incorporates independent monitoring and infrastructure-based policy enforcement. The platform also generates logs that enable auditing of authorization and rejection decisions. This is especially crucial for sectors where AI agents might interact with sensitive systems. Instead of relying solely on the agent to adhere to received instructions, Nvidia’s model seeks to have restrictions imposed by the technological infrastructure. Jensen Huang, founder and CEO of Nvidia, summarized the company’s commitment by stating that security must accompany the development of increasingly capable AI systems. The company thus proposes an open architecture for the so-called “agent economy.” The new platform places agent security at the forefront of a discussion that will extend beyond the AI model itself: who can access what information, what actions an agent can execute, and what happens when it attempts to overstep those boundaries. Nvidia aims to answer these questions through controls applied in both software and hardware.

You Might Also Like

https://www.liderempresarial.com/cuando-todos-tienen-acceso-a-la-ia-donde-estara-la-verdadera-ventaja-competitiva/

The entry