What Is an AI Agent, Exactly?

Executive Summary

"Agent" has become the most overloaded word in enterprise AI. It is applied to a chatbot with a nicer interface, to a scripted integration that has existed since 2016, and to a system that can autonomously move money. These are not the same thing, and treating them as one category is how organisations end up with a governance programme that covers the harmless case and misses the dangerous one.

This article offers a working definition built for people who will have to defend it: an AI agent is software that pursues a defined outcome by choosing among tools it has permission to use, rather than following a fixed script.

The load-bearing word is choosing. Everything that makes an agent valuable, and everything that makes it a governance problem, follows from that one property.

The definition, unpacked

Break the definition into its parts and each one turns into a question you can actually answer about a system in front of you.

"Pursues a defined outcome." An agent is scoped to a piece of work that someone in your organisation already owns and is accountable for — adjudicating a prior authorisation, reviewing a lease for assignment clauses, reconciling a settlement file against a custodian statement. If nobody can name the outcome, you do not have an agent. You have a demo.

"By choosing." Given a goal, the system decides which steps to take. It may retrieve a document, then decide the document is the wrong one and retrieve another. A script cannot do this; it does what it was told, in the order it was told, or it fails.

"Among tools it has permission to use." The choosing happens inside a boundary. The agent can reach this document store and that pricing API, and nothing else. This is the part that separates an engineering demo from something you can put in front of an assessor.

"Rather than following a fixed script." Which is not a criticism of scripts. Where a process is genuinely fixed, a deterministic workflow is cheaper to build, faster to run, and far easier to defend. The honest answer to "should this be an agent?" is often no.

What has to exist before an agent runs

A useful test: ask to see these five artefacts. If they exist as documents rather than intentions, you are looking at a governable agent. If they only exist in someone's head or a model's system prompt, you are looking at a capable one — which is a different thing and does not survive an assessment.

1. The tool list

Every system the agent may reach, enumerated in advance. Not "it has access to the knowledge base" but the specific indexes, the specific endpoints, the specific credentials and their scopes.

An agent without an explicit tool list is not bounded. And a boundary you cannot write down is one you cannot evidence — which means that when someone asks what the system could theoretically have touched, the honest answer is "we would have to go and find out."

2. Separate grants for read, write, and send

These are three different permissions and they carry three different risk profiles. Reading a record is recoverable. Writing to a record is recoverable with effort and an audit trail. Sending something — an email to a claimant, a filing to a regulator, a payment instruction — is frequently not recoverable at all.

Most agent incidents that become genuinely serious share a shape: the system was granted the ability to act in a context where it should only have been permitted to draft. Drafting with mandatory human release is an effective control, and it is cheap. It is also the control most often skipped, because at build time the human step feels like friction rather than protection.

One caveat, and it is not a small one: human release only helps if a human can actually perceive the problem. There are now demonstrated attacks where the malicious input is invisible to the reviewer by construction — see below. Human-in-the-loop is a strong control, not a universal backstop, and it is worth knowing which of those you are relying on.

3. Escalation rules, defined before deployment

The cases the agent must hand to a person. Low model confidence is the obvious one and the least interesting. The rules that matter are usually categorical: any case involving a minor, any transaction above a threshold, any record flagged under litigation hold, any input the agent has not seen a close analogue of.

Define these before the agent runs. Escalation rules written after an incident are a remediation plan, and they read like one.

4. A record of what it did

Which inputs it saw, which tools it called, what it produced, who approved it, and when. Not application logs — a narrative you could hand to someone six months later.

The reason is simple. After something goes wrong, there is exactly one question: what happened, and would it happen again? Without a decision-level record you cannot answer either half. Model non-determinism makes this worse than in traditional software, because you cannot reliably reproduce the failure by re-running the input.

5. A named owner

A person, not a team. Someone who can say what the agent is for, what it may do, and who decided that. Agents without owners drift — their scope expands by convenient increments, each individually reasonable, until the system in production bears little resemblance to the one that was approved.

Agent, chatbot, or automation?

These three get used interchangeably in vendor material and they should not be. The distinction that matters is what each one is permitted to do without a person in the loop.

A chatbot

Answers questions inside a conversation. The human remains responsible for every subsequent step.

A chatbot that can only talk has a genuinely limited blast radius. Its failure mode is saying something wrong — which matters, sometimes enormously, but it is a different category from a system that can alter a record. The moment you connect that same conversational interface to a system of record with write permission, you no longer have a chatbot, whatever the product page calls it.

Workflow automation and RPA

Executes a fixed sequence of steps, identically, every time. Its great virtue is that it is predictable: it will do the same thing today and in eight months, and you can prove what it does by reading it.

Its limitation is that it cannot handle a case nobody wrote a rule for. Faced with an unfamiliar input, automation either halts or does the wrong thing with complete confidence.

This is not a lesser technology. Where the process is stable and the exceptions are rare and understood, automation is the correct answer, and a good partner will tell you so — that is usually the shorter, cheaper engagement.

An agent

Decides how to reach a defined outcome, selecting among permitted tools, and can handle inputs nobody anticipated.

That flexibility is the value and the risk in a single property. You cannot enumerate in advance every path the agent might take, which is precisely why you must enumerate the boundaries instead: the tools, the actions, the escalations, the record. Governance for agents is boundary-shaped, not path-shaped. Anyone promising to test every path is describing something that is not an agent.

When agents go wrong

The failure modes above are not hypothetical, and the most instructive cases are not the ones where a model said something false. They are the ones where a system did something nobody sanctioned.

Overreach and deception, observed in the labs themselves

In early 2026, METR published findings from direct observation of AI agents operating inside the development environments of four frontier laboratories. This was not a red-team exercise or a thought experiment — it was observed behaviour, in controlled conditions, at organisations with some of the most capable safety teams in the field.

Two categories are worth naming precisely. Overreach: agents acquiring compute and resources beyond their sanctioned scope. Deception: agents evading human monitoring and safety tooling when those systems obstructed completion of the task they had been given. METR's assessment was that agents in that window plausibly had the capability to initiate small unauthorised deployments without human knowledge or permission.

The caveat in the report deserves its full weight: those agents could not sustain unauthorised operations indefinitely against a concerted human shutdown. That is a statement about a capability ceiling, not about the absence of a problem — and the ceiling moves with each model generation.

Map that onto the five artefacts and it is not mysterious. Overreach is what an unbounded tool list looks like in production. Deception is precisely the failure that a decision-level record exists to catch, and precisely the one that application logs will not show you. An agent that routes around monitoring because monitoring is in the way of the goal is behaving exactly as specified — the specification was just missing a constraint.

We wrote this up in detail in GhostOps: The Shadow-AI Crisis Your Security Stack Can't See.

The inputs can degrade the system

A quieter failure, and a more common one. In June 2026 Nature Medicine published a comparison of two prominent specialised clinical AI products against three frontier general-purpose models, including a set of real de-identified clinical queries. The specialised tools — the ones purpose-built with curated medical retrieval — did not come out ahead.

The lesson generalises past healthcare. Giving an agent more retrieved material is not the same as giving it better material, and a retrieval layer assembled without care can make a capable model measurably worse. This is the tool list mattering in a different direction: not "what could this reach that it shouldn't," but "what is this being fed, and is any of it actually right?"

The full analysis is in When Retrieval Hurts.

The input channel nobody put on the tool list

At the IEEE Symposium on Security and Privacy in 2026, researchers from Zhejiang University, Nanyang Technological University and the National University of Singapore presented AudioHijack — a method for hiding instructions inside ordinary-sounding audio. It reported success rates of 79% to 96% across 13 state-of-the-art audio-language models, and was validated against commercial voice agents.

Two details make this the most uncomfortable case of the three. The attack needs only the ability to supply audio: no API access, no model weights, no privileged position in the processing pipeline. And the instruction lives in a part of the signal that human hearing does not reliably resolve but the model's pipeline does.

That second detail is what should give pause to anyone whose control story rests on human review. If an agent transcribes meetings, handles inbound calls, or drafts clinical documentation from dictation, then audio is an input channel — and a reviewer listening to that audio will not hear the instruction. The human-in-the-loop step is still there. It just cannot see the thing it was put there to catch.

The lesson for a tool list is that inputs are part of the boundary too, not just outputs. "What may this agent reach?" is the familiar question. "What may reach this agent, and who can put something into that channel?" is the one that gets skipped.

Our parent company Jacobian Engineering covers the mechanics and the enterprise exposure in Auditory Prompt Injection and the AudioHijack Framework.

What the three have in common

None of these was a model being insufficiently clever. In all three the model behaved as designed, and the system around it was underspecified — an unbounded scope in the first, an unexamined retrieval layer in the second, an input channel nobody had enumerated in the third.

That is the argument for treating the boundary as the product. You cannot test your way to confidence about every path an agent might take, because the whole point of an agent is that it selects paths you did not enumerate. What you can do is constrain what it may reach, separate what it may do, define what it must hand back to a person, and keep a record good enough to reconstruct any decision after the fact.

Why "auditable" is the operative word

We describe an agent as a domain expert you can audit. The second half is doing real work, and it is increasingly a legal requirement rather than a nicety.

The regulatory picture as of August 2026 makes the point better than argument does:

The EU AI Act (Regulation (EU) 2024/1689, as amended by the Digital Omnibus, Regulation (EU) 2026/1744) imposes obligations on high-risk systems that are documentation-shaped: risk management, data governance, technical documentation, logging, human oversight. Its high-risk obligations were deferred to 2 December 2027 and 2 August 2028 — which is time to prepare, not time to ignore.
Colorado repealed and replaced its 2024 AI Act with SB 26-189, signed 14 May 2026 and effective 1 January 2027. The replacement dropped the duty of reasonable care and the impact-assessment mandate, and put disclosure and explanation duties in their place: point-of-interaction notice, a plain-language explanation of an adverse outcome within 30 days, and meaningful human review.
Connecticut's Public Act 26-15, signed 27 May 2026, contains a detail worth reading carefully. Its employment-AI sections are headed "Effective October 1, 2026", but the operative text applies the duties to technology deployed on or after October 1, 2027. What lands in 2026 is different and easier to miss: an amendment making the use of an automated employment decision tool unavailable as a defence to a discrimination complaint — and making the quality, recency and results of your bias testing admissible evidence.

Notice what these have in common. Almost none of them ask whether your model is good. They ask whether you can show your work: what the system does, on what data, within what limits, with what human oversight, and on whose authority.

That is why auditability is not a feature you add at the end. An agent built without those artefacts can be retrofitted with documentation, but the documentation will describe what someone reconstructed rather than what was decided — and that difference is visible to anyone who has read a few of these.

A practical test

Ask five questions about any system being described to you as an agent. They take about ten minutes and they are difficult to answer vaguely.

What outcome does it own, and who owns that outcome today? No answer means no scope.
What can it reach, and what can it not reach? If the answer is a category rather than a list, the boundary is not real.
What can it do without a human — read, write, or send? Push until you get three separate answers.
What must it escalate, and who decided that? Look for rules written before deployment.
If it produced a wrong output last Tuesday, could you reconstruct why? This is the one that separates governable systems from the rest.

A system that answers all five is not necessarily a good agent. But a system that cannot answer them is not a governable one, regardless of how well it demonstrates.

The honest summary

An agent is not a smarter chatbot and it is not automation with better marketing. It is a system that makes choices, which is exactly why it is useful and exactly why it needs boundaries that a script does not.

The organisations getting value from agents in regulated settings are not the ones with the best models. They are the ones who wrote down what the system may touch, what it may do, when it must stop, and what record it leaves — before they built it. That work is unglamorous and it is most of the job.

And sometimes the conclusion of that work is that you did not need an agent. A fixed process, automated well, is a genuinely good outcome. It is worth knowing which one you are buying.