When someone hides an instruction in something your AI reads, and the AI follows it.
4 min to watch1 min to read
Based on real cases: the EchoLeak flaw in Microsoft 365 Copilot (2025)1,2 and a car dealer's chatbot (2023).3 The screens, names and emails are made up.
Read the walkthrough as text
Without protection
You use an AI assistant that can read your email for you.
You ask it for something simple.
The AI says yes and gets to work.
Behind the scenes, the AI opens your email inbox.
It reads the first email: a team lunch on Friday.
Then an invoice from the accounts team.
The third email is from someone you don't know. The AI opens it.
Inside, there's text you can't see: white words on a white background.
This is prompt injection: someone hid an instruction for the AI inside the email.
The AI can't tell your words from the stranger's words. So it follows the hidden instruction.
It clicks Send.
Your finance file just left the company. Nobody clicked anything.
Back in the chat, the AI reports in like nothing happened.
That's prompt injection.A stranger hid an instruction where the AI would read it, and the AI followed it. Researchers showed this kind of attack on Microsoft 365 Copilot in 2025; it was fixed that June.
With protection
Same request, same inbox. This time the AI has two simple protections.
It reads your emails the same way.
It opens the stranger's email and finds the hidden text.
Protection one: words inside an email are something to summarize, never something to do.
Protection two: this assistant has no permission to send email. It can only read and summarize.
Back in the chat, it tells you what it found.
Read it. Don't follow it.A safe AI treats outside text as information, and can't send anything out unless you say yes.
It happens in chats too
Prompt injection doesn't always hide in an email. Sometimes the person typing does it.
A visitor to a car dealer's website gives its chatbot a new rule.
The chatbot was built to be helpful, so it goes along.
Then the visitor asks for a new car for one dollar.
The chatbot follows its "new rule" word for word.
This really happened in 2023. The dealer didn't honor the deal, but the chat spread online.
Same trick, typed in the open.The chatbot treated a stranger's instruction as its new job. Real case: a car dealer's chatbot, December 2023.
Keep going:
Why it matters, what it costs, and what to ask your team.
Why it works
To the AI, it's all one stream of text.
Your request and the email the AI reads arrive together. The AI can't always tell which words are from you and which are from a stranger.
But how am I supposed to know who's talking?
Try it
Can you sneak an instruction past a filter?
Many apps add a filter that blocks known tricks. See how far a filter alone gets you.
Hide an instruction for the AI inside an email:
Or try:
Blocked 0Got through 0
A toy filter that blocks a few well-known phrases, the way many real ones do.
Filters catch the obvious ones. Polite or sneaky instructions slip through. That's why safe AIs also can't send anything out without asking you.
The dangerous mix
The lethal trifecta
Security researcher Simon Willison named three abilities that are risky together: reading your private data, seeing outside content, and being able to send things out.4 Turn them on and off.
Any two are manageable. All three together is how one hidden instruction becomes a data leak.
Under the hood, step by step: build an email summarizer that can't be tricked.
Build along
Build a safe email summarizer, one step at a time
We start with the simplest version, run it on the same three emails, and fix it in small steps. Press Run it after each step and watch the result change.
Layer three
Break the trifecta on purpose.
Design so no single AI has all three abilities at once.
Split the workOne model reads outside text with no tools. Another has tools but only sees the clean summary.
Make sending a human decisionEmails out, file shares and payments need a click from a person.
Don't auto-load links or images in AI outputShow them as plain text until someone chooses to open them.
Log and alertRecord every tool call so a strange one gets noticed.