Artificial intelligence agents have moved from theoretical entities to real-world digital workers that schedule meetings, write code, analyse markets, and execute actions across systems! But as these independent agents become more autonomous and intelligent, they will be attacked in new ways using new, and in some cases. A few of the most important security vulnerabilities today are prompt injection and the wider scope of autonomous risk.
In this blog, we will unpack both of these emerging threats and discuss what developers, enterprises, and cyber security teams can do to mitigate their risk in AI ecosystems.
Here is a two-part risk breakdown:
Prompt injection is the AI equivalent of SQL injection. Instead of targeting a database, it exploits the interaction between the user and the AI model through the input prompt designed for the application you are using. In simple terms, a user can maliciously embed instructions, distractions, or misleading content to divert the AI from its intended purpose. This is done by hiding the harmful text as a normal entry, such as a calendar event, a page, or a chatbot interaction.
For example, an AI assistant inputting a summary of a webpage with user data could have hidden instructions via hidden text that reveal confidential user information, completely change the content of the description, or send emails with fraudulent attachments. These hidden instructions would look like normal unmarked text, graphics or links.
As generative models (i.e. OpenAI's GPT or Anthropic's Claude) are integrated into more software, this attack surface increases exponentially. And since LLMs don't 'detect' intentions like a person can, they are highly vulnerable to these types of nuanced attacks.
Autonomous agents are engineered to be given a goal (book a flight, for example, or perform a code audit) and then autonomously decompose the goal into steps, decide what tools to use, and carry out tasks in real time. This is efficient, but raises an important question: what happens when the agent misunderstands a task or acts outside of its intended parameters?
This is where autonomous risks emerge.
An agent can:
Unlike static interactions where an LLM is hosted in a chat-like interface, autonomous agents have persistent states, memory, and often access to additional external tools (browsers, shells, CRMs, etc.). When poorly scoped, mis-specified, or leveraged incorrectly, they can act more like a rogue script running in your infrastructure.
With generative AI entering finance, healthcare, customer support, and software development, these risks aren't just theoretical anymore.
A 2024 report from the UK's NCSC states, “Cyber criminals are adapting their business models to embrace this rapidly developing technology - using AI to increase the volume and impact of cyber attacks against citizens and businesses”. Similarly, OpenAI has warned about the “emergent capabilities” of agents and the challenges in containing autonomous behaviour once multiple tools and APIs are in play.
The concern isn't just about data theft; it's also about reliability, liability, and loss of control.
As always, do not forget that security is a concern of design, not an afterthought. Here are ways to keep your data secure while using AI:
The AI agents are here, and so are the vectors of opportunity for a new attack surface. Prompt injection and autonomous risk are not bugs to fix; they are systemic challenges in how we build intelligent systems.
If we gain an understanding of the nature of these threat vectors and create appropriate guardrails, then we have the potential for innovation without leaving the backdoor open. At DCG, we provide expert-led cyber training, from covering the basics to live simulations – we've got it all covered.
Ready to start your team's threat preparedness training? Contact us today!
AI‑Powered Threat Intelligence: Defending Against Next‑Gen Attacks
The Future of Cyber Insurance: How Compliance Affects Coverage in the UK
AI, Privacy, and Compliance: Adapting to New Global Regulations
Cyber Security Trends 2025: Navigating The Evolving Threat Landscape
Remote work is no longer a trend, it’s the norm. But while the workplace has evolved, many organisations are still clinging to legacy technologies like VPNs to secure a fundamentally different environment.
With the growing complexity of cyber threats, existing defence measures can no longer keep up. Threat actors are leveraging automation,
Zero Trust, once simply a buzzword, is now a foundational principle of cyber security for modern organisations.
Given the increased frequency and seriousness of cyber attacks, cyber insurance has gone from a luxury to a necessity for business risk management.
Join industry leaders on cyber projects to create a global impact and ensure a secure digital future.
