Every CEO who tells me proudly that they rolled out an AI assistant gets the same question from me: who tested what happens when an outsider plants an instruction inside a document that assistant reads? Usually the answer is silence.
Not negligence. Nobody pitched the project that way. It was sold as better service, or as saved headcount. But the moment you connected a language model to your company data, you opened an attack surface that the defenses you spent the last fifteen years building were never designed for.
I do not work in cybersecurity. I am a CEO who builds systems himself, and when you build, you see up close where things are brittle. What I am describing here is not a future scenario. It is documented, it is mapped, and it has a name.
The instruction hiding inside the data
The central problem in language model security is prompt injection. It is hard for an architectural reason: your instructions and the data the model reads arrive through the exact same channel. The model cannot really tell the difference between "this is an order from my employer" and "this is text I was only asked to summarize." An attacker who plants one sentence inside an incoming email, a CV, a supplier invoice or a web page can get the model to treat that sentence as a command.
In the OWASP Top 10 for LLM Applications 2025, prompt injection ranks first, for the second edition running. Second, after a jump from sixth, is sensitive information disclosure.
None of this is theoretical. In June 2025 the research firm Aim Security disclosed a flaw in Microsoft 365 Copilot, named EchoLeak and tracked as CVE-2025-32711, with a severity score of 9.3. The attacker sends an ordinary-looking email with instructions hidden inside an HTML comment or in white text. The user clicks nothing. The assistant reads that email on its way to answering an entirely different question, and leaks internal data out. It was documented as the first case in which prompt injection was used to pull data out of an AI system running in production.
Why "we will close that in the next release" is not an answer
I would like to write that there is a patch. There is not. At Infosecurity Europe in 2026, a contributor to the OWASP project said plainly that prompt injection remains an unsolved architectural problem, for the simple reason that a language model processes all input as one stream and nothing reliably enforces a permissions boundary between the system instruction, the user question and content pulled in from outside. A Help Net Security review from June 2026 found that prompt injection still leads most security failures of AI agents in production.
The practical conclusion is not to give up on AI. It is to stop designing as if you can prevent this, and start designing permissions. If the assistant can read only what the user in front of it is allowed to read, and cannot take any action with financial or legal consequence without human approval, the worst case from a successful injection shrinks dramatically.
That is exactly the logic the Israel National Cyber Directorate put to Israeli executives in a report it published in May 2026: move from prevention and patching holes to containment and operational resilience, on the assumption that the internal network will be breached. The report also recommended a binding date, 31 May 2026, for presenting the threat to boards and audit committees. The background is not reassuring. According to figures presented by the head of the Israel National Cyber Directorate, Yossi Karadi, at Cybertech Global in Tel Aviv in January 2026, Israel handled more than 26,000 cyber incidents in 2025, a 55% rise over 2024.
Seven questions for the vendor, before you sign
These are the questions I ask every CEO to put to their AI vendor in writing. Not because I expect the CEO to understand every answer in depth, but because the quality of the answer teaches you more than the answer itself.
- What company data can the model reach, and under what permission. Does it see exactly what the user sees, or more.
- Which actions may the system take without human approval. Payment, outbound mail, updating a record, deleting.
- Where are the prompts and the outputs stored, for how long, and in which country.
- Is our input used for training. In writing, not in conversation.
- What happens to content pulled from an outside source. Is it tagged and separated from the instructions.
- How do you detect and log an injection attempt, and who gets the alert.
- Give me the log of three real conversations from the past month. No log, no control.
A vendor spooked by these questions is not a vendor. And if the answers turn too technical, that is exactly the moment to bring a security professional into the room. I know how to build the system and ask the right questions. A penetration test on a critical system belongs to someone who does it for a living.
What I actually do
Two principles repeat in the systems I build. First: sensitive data does not leave the perimeter when there is no reason for it to. At a large medical institution I worked with, transcription of patient calls runs on a local server rather than in the cloud, for exactly this reason. Second: every meaningful action has a human gate. The model proposes, a person approves.
It does not impress anyone in a slide deck. It is what holds when something goes wrong.
If you are in the middle of an AI project and want to run these seven questions past your vendor, I am happy to sit on it for an hour. No slides.
