Key points
Key points
- AI agents can perform complex, multi-step tasks with limited human intervention, increasing both their value and the potential impact of errors.
- Rapid adoption is expanding the number of non-human identities with access to applications, data and production systems.
- Recent incidents demonstrate that AI agents can behave unpredictably, acting beyond their intended scope or being manipulated by malicious actors.
- Effective controls include agent inventories, least-privilege access, audit trails, staging environments and continuous monitoring.
- Growing demand for AI security solutions may create long-term opportunities in areas such as observability, anomaly detection and agent governance.
Artificial intelligence (AI) has evolved far beyond simple chatbots that answer questions. Today's AI agents can browse the web, write and execute code, manage files, send emails and complete complex, multi-step tasks with minimal human intervention. Their rapid adoption is driven by the promise of greater productivity, efficiency and cost savings.
Despite their potential, AI agents present important limitations and risks. A study by Carnegie Mellon University found that AI agents can become disoriented, choose incorrect shortcuts or struggle to complete even simple tasks.1 They can cause unintended record updates, break workflows and push flawed logic into production systems. Those incidents often go unnoticed until something breaks or sensitive data are affected.2 According to The Economist, “AI agents, which are supposed to work on people’s behalf in “alignment” with their values, lie, cheat and steal if necessary. They break free from captivity and form harmful posses to do harm to people.”3 Taken together, these developments point to the emergence of a new risk category: the “rogue AI agent”.
The rise of AI agents
The rise of AI agents
The global market for AI agents is expected to experience substantial growth over the coming decade (Figure 1). With a projected compound annual growth rate (CAGR) of over 43%, the market is forecast to expand from USD 3.66 billion in 2023 to approximately USD 139.12 billion by 2033. Within this market, ready-to-deploy AI agents are expected to maintain their leading position. Their continued dominance is primarily driven by ease of implementation and seamless integration into existing business environments, making them an attractive option for organizations seeking to leverage AI capabilities without the time, complexity and resource commitments associated with developing custom-built ("build-your-own") agent solutions.4
In our view, the rise of AI agents is fundamentally reshaping the cybersecurity landscape. As AI agents become increasingly accessible to develop and deploy and as agents are able to create and orchestrate additional agents, organizations are facing a rapid proliferation of non-human identities in the form of AI agents. These agents interact across applications, application programming interfaces (APIs), software-as-a-service (SaaS) platforms and data environments, often operating with privileged access and at machine speed.5 Consequently, the attack surface has expanded considerably. IT security teams are no longer tasked solely with securing software and infrastructure; they must also govern and protect autonomous systems whose decision-making processes can be difficult to predict, monitor and control, particularly in the case of a rogue AI agent.
Figure 1: The global AI agents market is projected to reach USD 139 billion by 2033
Market size by agent type, 2024-2033 (in USD billion)
Source: Market.us, AI Agents Market, June 2025, market.us/report/ai-agents-market/
What is a “rogue AI agent”?
What is a “rogue AI agent”?
A rogue AI agent is an autonomous system that acts in ways its creators did not intend, whether by pursuing objectives too literally, exploiting loopholes in its instructions, operating beyond its authorized scope or being manipulated by a malicious actor.6,7
Importantly, these systems do not become rogue because AI is inherently malicious or a “robot uprising” is underway. Rather, such outcomes typically emerge when highly autonomous agents are deployed without adequate governance, oversight and security guardrails. AI agents act on signals and lack the “cognitive filter” that prevents humans from following bad ideas. They are designed to optimize for a given objective and may pursue unintended ways of achieving it, which can make their behavior difficult to predict. In reinforcement learning, successful outcomes are rewarded during training.8 In our opinion, this makes them both powerful and potentially dangerous.
Notable incidents of rogue AI agents
Notable incidents of rogue AI agents
Recent evaluations by major AI labs revealed that AI agents went rogue during cybersecurity tests. Below are several notable incidents involving rogue AI agents:
- An Alibaba-affiliated AI agent unexpectedly escaped its sandbox by establishing a reverse SSH tunnel9 to an external IP address. Without human instruction or authorization, the AI agent autonomously diverted computing power to mine cryptocurrency in order to maximize its computational rewards.10
- An autonomous AI agent driven by OpenAI models escaped its evaluation sandbox, penetrated the Hugging Face infrastructure, moved laterally over several days, stole tokens and utilized stolen credentials before being contained.11
- Anthropic’s Claude models gained unauthorized access to the real systems of three different organizations during a test after mistaking the targets for a fictional challenge.12
- The UK's AI Security Institute reported that AI models from Anthropic and OpenAI took unsanctioned actions on the live internet. In one case, an agent created multiple fake online identities to trick a human open-source project maintainer into approving malicious code.13
How can organizations mitigate rogue AI agent risk?
How can organizations mitigate rogue AI agent risk?
When an AI agent goes rogue, it is rarely the result of a single dramatic failure. Typically, the problem develops through a gradual accumulation of small missteps, such as unreviewed access, missing audit trails, outdated credentials or unmonitored actions.14 The following measures can help reduce the risk of rogue behavior:15
- Maintain a complete inventory of AI agents: Establish and maintain an inventory of all AI agents, documenting each agent’s purpose, owner, permissions and access to data, applications and systems.
- Enforce least privilege principles and continuous governance: Grant each AI agent only the minimum permissions required to perform its tasks. Permissions must be tested, reviewed and logged regularly. Introduce a staging environment for AI agent actions before they reach production. Every AI agent needs a full audit trail.
- Use continuous monitoring to observe agent behavior in real time: Centralize oversight through a unified checkpoint that inspects all agent traffic and blocks prompt-injection attempts, data exfiltration and suspicious tool calls before they cause harm.
Implications and outlook
Implications and outlook
AI agents are no longer a science-fiction abstraction; they are powerful optimization engines operating in complex, imperfectly controlled environments. The real risk lies in misaligned objectives, insecure integrations, opaque behavior and the speed and scale at which AI agents can act.
As AI agents move deeper into our daily lives, treating rogue behavior as an operational risk becomes essential. Addressing this challenge requires a combination of technical safeguards, thoughtful system design and organizational discipline about how much autonomy to grant and when.
For long-term investors, this structural shift is creating compelling opportunities. The UBS security equity strategy seeks to capture this trend by investing in companies at the forefront of AI security. By allocating capital to innovators in areas such as anomaly detection, continuous observability, dynamic agent governance or agent recovery, rewind and remediation, the strategy aims to benefit from the growing demand for solutions that enhance the reliability and trustworthiness of AI systems.
1 Xu et. al. (2024): TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks, arxiv.org Navigating websites seems to be a hard task, as evidenced by an AI agent that couldn’t figure out how to e.g. close a pop-up box (source: Xu et. al. (2024), p. 9).
2 Rubrik (2025): When AI Agents Go Awry: Introducing Rubrik Agent Rewind, Powered by Predibase AI Infrastructure, rubrik.com.
3 The Economist (2026): AI agents lie, cheat and steal. That is putting off users, economist.com.
4 Market.us (2025): AI Agents Market, Market.us.
5 Okta (2026): The blueprint for the secure agentic enterprise, okta.com.
6 Rubrik (2025): When AI Agents Go Awry: Introducing Rubrik Agent Rewind, Powered by Predibase AI Infrastructure, rubrik.com.
7 Adversaries can use subtle tactics that steer AI agents away from their intended plan. They can insert hidden instructions into untrusted inputs or use poisoned tool metadata to shape the parameters the model produces (source: Crowdstrike (2026): AI Agent Security: Architecture, Attack Surface, and Defense – A Practical 90-Day Roadmap for Securing Agentic AI, brochure, crowdstrike.com). As a result, this makes them vulnerable to manipulation attempts such as prompt injection attacks (further information about prompt injection attacks are described in our Thematic Insights (2025): The dark side of AI (part one): prompt injection attacks, ubs.com).
8 Financial Times (2026): “Rogue” AI is working as intended, in: Financial Times, 18.8.2026, p. 17.
9 A reverse SSH tunnel (also known as remote port forwarding) is a technique used to establish a secure, encrypted connection from a remote server back to a local machine that is otherwise inaccessible from the outside internet, Qbee (2025): Reverse SSH Tunneling: The Ultimate Guide, qbee.io
10 Bloomberg (2026): Alibaba’s rogue AI turns crypto miner – Without permission, bloomberg.com.
11 OpenAI (2026): OpenAI and Hugging Face partner to address security incident during model evaluation, openai.com.
12 Anthropic (2026): Investigating three real-world incidents in our cybersecurity evaluations, anthropic.com.
13 AI Security Institute (2026): Incident Report: unsanctioned agent behavior during cyber testing, aisi.gov.uk.
14 A recent survey of 200 North American security leaders found that 63% of them now see the biggest internal security risk is their employees unknowingly giving AI agents access to sensitive data, 1Password (2025). New research uncovers four security challenges caused by unmanaged AI access, 1password.com.
15 Palo Alto Networks (2026): Preventing AI Agents from Going Rogue, paloaltonetworks.com.
Code : M-007066
