AI news
AI model reviews
AI tools
How-to guides
Our staffAboutContact

Confirmation hacking AI assistant definition: what the term means in 2026

Confirmation hacking means tricking an AI agent or its user into approving a harmful action. Here is the real research behind the informal term.

Red padlock resting on a laptop keyboard
Photo: FlyD on Unsplash

Confirmation hacking, in the context of an AI assistant, is an informal name for attacks that target the "are you sure?" step: tricking an AI agent, or the person supervising it, into approving an action they'd refuse if they saw it clearly. It usually works through prompt injection that distorts what the approval prompt shows, or through approval fatigue, where people click "yes" so often they stop reading. I couldn't find a standards body or major lab that uses the exact phrase, so treat it as a handy label rather than a formal term.

The closest official name is "HITL dialog forging", also called Lies-in-the-Loop, which OWASP catalogues as an attack on human-in-the-loop approval in AI agents. That's where most of the real evidence sits, and it's what this piece leans on.

Confirmation hacking AI assistant definition: the short answer

An AI assistant that can act (send email, run code, buy things) often pauses to ask you first. Confirmation hacking is anything that makes that pause worthless.

There are, as far as I can tell, three versions that people lump under the term. The first is a forged dialog, where hidden instructions in a web page, document or GitHub issue shape the summary you're asked to approve. The second is fatigue, where the prompts are honest but so frequent that approval becomes a reflex. The third shows up in automated systems, where an AI or a script "confirms" that something happened by reading evidence an attacker can write.

FormWhat gets fooledClosest named conceptMain fix researchers propose
Forged approval dialogThe human reading the promptHITL dialog forging / Lies-in-the-LoopBuild the dialog from the real action, not from model text
Approval fatigueThe human's attentionApproval fatigue (Anthropic's term)Fewer, better prompts; sandboxes for low-risk work
Forged automated confirmationA rule or AI judgeForgeable confirmationTake evidence from a channel the attacker can't write

The background: approval prompts and prompt injection

Prompt injection is the root of most of this. OpenAI describes it as a kind of social engineering aimed at the model: a third party slips instructions into content the AI reads, such as a web page or an email, hoping it'll treat them as orders. Simon Willison named the problem back in 2022, and it still isn't solved.

So vendors added a human checkpoint. ChatGPT agent, going by OpenAI's own write-up, "pauses and asks for confirmation prior to taking sensitive steps such as completing a purchase", and it has a Watch Mode that needs you to keep the tab open on sensitive sites. Claude Code is read-only by default and asks before it changes files or runs most commands, per Anthropic's sandboxing post. OWASP recommends this kind of human approval as a mitigation for two of the entries in its LLM Top 10.

The weak spot is obvious once you say it out loud. The dialog you approve is usually written from the same context the attacker poisoned. If the agent has read a malicious page, its summary of what it's about to do may be the attacker's words, not the truth. OWASP calls the approval step "the last line of defense before an AI agent executes sensitive operations", which is precisely why it's worth attacking.

A rough everyday comparison: it's like a bank clerk asking you to sign a transfer slip that the fraudster filled in, while reading the amount aloud from a note the fraudster also wrote. You signed, but you never agreed to that transfer.

Real examples from security research

The clearest case comes from Checkmarx Zero. Its researchers showed that text planted in a public GitHub issue could get Claude Code to propose a command while the approval prompt hid what the command really did, mostly by burying it under a wall of harmless-looking output. Dark Reading's account quotes the team: "LITL abuses the trust between a human and the agent." Once the dialog looked routine, "All the user has to do to harm themselves is press Enter."

Anthropic's response is worth knowing. According to the same report, the company said this wasn't a vulnerability, since users are responsible for reading permission prompts in full before granting them. I think that's defensible on paper and shaky in practice, for reasons Anthropic's own data makes clear (more on that below).

In a follow-up post, Checkmarx extended the idea: padding text before the payload as well as after it, tampering with the one-line description that summarises the agent's plan, and Markdown injection in the Copilot Chat extension for VS Code that could make attacker text look like interface elements. OWASP's page lists the same three techniques by name. I won't go further into the mechanics; the point is that the dialog itself became the attack surface.

Browser agents have the same problem. A September 2026 preprint, The Verifiable Action Card, built a 24-scenario benchmark with forged dialogs, confused-deputy pages (where a form looks normal but carries a hidden recipient) and indirect injection. Without the authors' defence, attack success across the models they evaluated ranged from 68% to 100%.

That's a single preprint, not yet peer reviewed, so I'd treat the exact figures cautiously.

A second preprint looks at machines confirming things for other machines. In Forgeable Confirmation in Automated Computer Security Testing, the authors found that nine of 15 confirmation checks in an AI-assisted security-testing pipeline could be forged, and whether a check was forgeable came down to one thing: did it read data the attacker controlled? Moving the decisive evidence to a channel the attacker can't write cut attack success from 97% to 0%.

Benefits and limitations of confirmation prompts

None of this means approval prompts are useless. They catch honest mistakes, and those are common. In its post on Claude Code's auto mode, Anthropic describes incidents from its internal log: an agent deleting remote git branches after a vague "clean up" request, another uploading an engineer's GitHub token to an internal cluster, another attempting migrations on a production database. A human glance would probably have stopped all three.

The limits are just as concrete. Anthropic says plainly that constant approval clicks lead to fatigue, "where users might not pay close attention to what they're approving." Its auto mode post adds a number that surprised me: in practice, users accept 93% of manual prompts anyway. If nearly everything gets a yes, a forged dialog only has to look ordinary.

That's the tension. More prompts mean more chances to catch a bad action, but they also train people to stop reading. Fewer prompts mean more attention per prompt, but more actions run unchecked. Neither extreme is safe, and I don't think anyone has found the right balance yet.

Model-level defences help without closing the gap. Anthropic reports that Claude Opus 4.5 cut prompt injection success against its browser extension to about 1% in its internal adaptive testing, and the company itself says 1% still represents meaningful risk. Small percentages add up fast when an agent reads thousands of pages.

Practical takeaways for you

If you use an AI agent at work or at home, a few habits seem to matter more than anything else:

  • Read the whole approval prompt for anything irreversible, especially payments, emails to new recipients, file deletions and shell commands. Scroll up if it's long; a suspiciously long prompt is itself a warning sign.
  • Give narrow instructions. OpenAI's guidance suggests that a broad request to answer all your overnight emails invites more trouble than a specific, bounded task.
  • Use logged-out or sandboxed modes when the task doesn't need your accounts. ChatGPT Atlas has a logged-out mode, and Claude Code's sandbox limits which folders and servers the agent can touch.
  • Be wary when the agent has just read something from strangers: a GitHub issue, a shared doc, a product review, an inbound email.
  • Don't turn off permission checks entirely. Anthropic describes the skip-all-permissions flag as unsafe in most situations.

If you build agents, OWASP's advice is specific: construct the approval dialog in your own code, don't render model-supplied Markdown in it, and stop untrusted content from touching the summary line. The Verifiable Action Card paper goes a step further by rebuilding the dialog from the pending action itself and checking, at the moment of execution, that the approved action is the one that runs. Our guide on how to build AI agents covers the basics, and our list of AI code security tools is a reasonable next stop.

The next steps for agent makers

We know a fair amount already. Forged approval dialogs have been shown on real coding assistants, approval fatigue is measurable, and confirmations that read attacker-controlled data can be faked. What we're still guessing at is how often any of this happens outside research labs; I couldn't find a public, confirmed real-world attack that used a forged AI approval dialog, which is either reassuring or a sign that nobody's looking closely.

The direction of travel seems clear enough. Vendors are moving away from asking humans about everything and toward sandboxes, classifier-based approval (Anthropic's auto mode uses a model to judge each action) and dialogs built from ground truth. My guess is that "trusted display" for agent actions, something like the padlock icon in your browser but for what the agent is about to do, becomes a standard feature within a couple of years. That's my prediction; nobody has announced it.

For now, I'd treat any approval prompt from an agent that has read untrusted content as a claim worth checking before you click yes. If you want the wider picture, see our explainers on agentic AI, shadow AI and the dangers of AI, plus our Claude AI review.

FAQ

Is confirmation hacking an official security term?

No. As of November 2026 I couldn't find OWASP, NIST or a major AI lab using it. The established names are HITL dialog forging or Lies-in-the-Loop for forged approval prompts, approval fatigue for reflexive clicking, and prompt injection for the underlying technique.

Is confirmation hacking the same as prompt injection?

Not quite. Prompt injection is usually the delivery method, and confirmation hacking is what it's aimed at: the approval step. Approval fatigue doesn't need any injection at all, just a lot of prompts.

Which AI assistants are affected?

Any agent that asks a person to approve actions based on content it has read could be exposed. Checkmarx demonstrated the attack on Claude Code and showed related Markdown issues in GitHub Copilot Chat, and the Verifiable Action Card paper tested browser agents across several models. The weakness comes from how approval dialogs are built, so it isn't tied to one product.

Can I spot a forged approval prompt?

Sometimes. OWASP suggests paying attention to odd structure, such as very long prompts, unexpected formatting or text that looks like interface elements. But the honest answer is that well-made forgeries are hard to spot, which is why researchers want the fix built into the software rather than left to you.

Does turning on auto-approve make me safer from this?

Not by itself. It removes fatigue but also removes your chance to catch a bad action. Classifier-based modes like Claude Code's auto mode and sandboxing try to keep a check in place without asking you every time; fully skipping permissions is the risky option.

Keep reading