The Pandora's box of data compliance and privacy: agentic AI and the Model Context Protocol (MCP)

The question changed and the rules did not

Every AI policy written between 2023 and 2026 — the journal declarations, the HTA position statements, the firm-level acceptable-use rules — rests on the same mental model. A person has a document. The person decides what to put in a prompt. The confidentiality question is therefore a question about that decision, and the remedy is to decide better: redact first, use the enterprise tier — with zero retention, a compliance API, a BAA, and residency where PII or PHI is involved — and file the paperwork that goes with it.

Tool-using AI, and specifically agentic AI, breaks that model. Under the Model Context Protocol and similar arrangements, a model does not wait to be handed text. It reaches: opening files, running queries, calling services, passing results between steps. Nobody chooses what goes in, because choosing is the part that was automated. The confidentiality question stops being what did you send and becomes what could it reach — and that question has a different shape, a different answer, and no rulebook behind it.

Scope, not intent

This is the distinction the whole subject turns on. Pasting is bounded by intent: what you deliberately put in the box is what leaves, so the failure mode requires someone to be careless or dishonest, and the fix is a rule telling them not to be. An agent is bounded by scope — a configuration written once, usually generously, so that the thing works. From then on it applies to every request.

Nobody has to behave badly for this to go wrong. Consider the most ordinary possible instruction: pull together the background for this submission. A well-built agent will do exactly that, thoroughly, across everything it can see. The thoroughness is the product working. The problem is what "everything it can see" contains.

What is actually on the machine

A general audience reads "an agent might read your files" as a mild risk, because the picture in mind is a laptop with a few documents on it. That is not what a working machine in this field looks like, and the gap between the two pictures is where the argument lives.

You already know what is on yours. Pricing models and strategies, net-price assumptions, country-level discounts. Market-access discussions and negotiations on submission status. Target product profiles. Unpublished trial data. Draft dossiers for products that never launched. Competitive analyses that name the competitor. Patient-level files from an analysis three years ago that was never formally closed out. And the part people forget when they think about "files" at all — the mailbox, where identifiable information arrives constantly, unsorted, in attachments nobody filed and threads nobody closed.

This is not one sensitive document. It is hundreds, accumulated across years and belonging to several clients who have never met, whose agreements each assume their material is held apart from the others. The folder structure is usually the only thing enforcing that separation, and a mailbox does not even have that.

An agent with filesystem access does not see a folder structure. It sees a root. An agent with a mail connector does not see a filing system at all — it sees an account, and everything that has ever arrived in it.

The obvious objection is that a strict IT policy should prevent this accumulation, and a strict policy will certainly discourage it. It is very hard to enforce. People travel. They work from a conference, finish something at home, pull a file down because the hotel wifi was failing, and the document stays. Over years of engagements the residue builds up on exactly the machines doing the most sensitive work. This is precisely why laptop encryption is mandated in regulated firms that already have strict policies — not because the policy is ignored, but because everyone involved knows what is really on the disk.

Why reaching becomes disclosing

Reading a file is not, by itself, a confidentiality event — the material was already on the machine, lawfully. The event is what happens next. Whatever the agent gathers is assembled into context and sent to a model, which for most deployments means an API call to infrastructure in another jurisdiction, and unless something specifically intervened, with the real values intact — and with no audit record surviving the trip.

Three properties make that worse than the equivalent paste. There is no moment of decision, so nobody can later say what they intended to disclose. There is usually no record of which files were opened, so the scope of a disclosure cannot be reconstructed even in good faith. And the volume is whatever the traversal found, rather than whatever a person thought was relevant.

A consultant who pasted a paragraph into a chatbot has done something bounded and describable, and if a client asks, they can answer. A consultant whose agent read a working drive has done something they cannot characterise at all. In a dispute, the second position is far worse — not because more was necessarily disclosed, but because nothing can be excluded.

What the rulebook says about this: nothing

We went looking, across the bodies that actually govern this work: the journals and publishers, the scientific societies, the HTA agencies, and the regulators. As of September 2026, no professional or regulatory body has published a rule addressing tool-using AI as a distinct category. The closest four approaches are worth stating precisely, because each one demonstrates the gap rather than filling it.

• ESMO names agentic systems in its clinical guidance specifically to exclude them, noting that systems which take action autonomously are outside its scope and that future guidance will be needed.
• Elsevier's June 2026 policy names "AI agents and deep research tools" — and then folds them into the same disclosure regime written for chatbots. Naming a thing is not governing it.
• NICE asks submitters to evidence measures against prompt injection and data poisoning. That is the closest any HTA text comes to the attack surface agents create, and it never uses the word.
• The EU's HTA Coordination Group requires that all prompts be recorded and produced on request — a rule that quietly assumes prompts are things a person wrote, rather than context a tool assembled.

Meanwhile the duties that already exist do not pause while the guidance catches up — and the catch-up required is substantial, because most regulated-industry guidance addresses early generative AI and is being read in the era of agentic systems. An NDA asks whether confidential information reached a third party, not whether a human chose to send it. PIPEDA treats a transfer to a processor as a use for which the organisation remains accountable. And Quebec's Law 25 attaches transparency duties to decisions based exclusively on automated processing — a test that ordinary AI drafting escapes only because a human reviews the output, and that agentic pipelines are built to stop escaping. That last point is our reading rather than settled interpretation, and we set it out at greater length in the PIPEDA and Law 25 guide.

Our position: today's MCP does not belong on a regulated machine

It would be easy to end this with a configuration checklist — narrow the scope, review the grants, be careful. We do not think that is the honest conclusion, and it is not the one we act on ourselves.

In regulated work the obligation is not to take reasonable care once the tool is in place. It is to take all necessary measures to protect the information you hold — an obligation that runs to your clients through their agreements, to patients through privacy law, and to regulators through the undertakings your firm has already signed. Against that standard, a tool that can traverse a working machine, pass what it finds to a third-party model — or to several tools in turn, each calling its own uncloaked API — and leave no reliable account of what it read or where that material went is not a thing to be configured carefully. On the evidence available today, the risk outweighs the benefit by a wide margin, and running it on a machine holding client and patient material is difficult to reconcile with the duty itself.

That is a statement of our position, not of law. Reasonable people are deploying these tools in good faith, and the capability is genuinely valuable — which is exactly why the conclusion is uncomfortable rather than obvious. But the question a regulator or a client asks after an incident is not whether the configuration was thoughtful. It is what the tool could reach, what left the building, and what you can show. Most current deployments cannot answer the third question at all.

What would have to be true instead

In the long term the capability is not the problem; the absence of accountability and guardrails around it is. A tool of this class belongs on a regulated machine when it can demonstrate three things, and demonstrated is the operative word — each can be tested without reference to a vendor assurance, which in our view does not by itself meet the “all necessary measures” standard the law intends.

• Limit the tool’s reach at the root, not per installation. The scope should be built for the job in front of you and expire when that job does. What usually happens instead is that someone grants a whole drive on day one because it made setup easier, and nobody revisits it.
• Remove the values before the AI sees the data. Secrets, sensitive values, confidential business information, PII and PHI should be gone before anything is transmitted. Traversal can still happen and the model can still reason, provided what crosses the boundary is cloaked labels rather than a client’s pricing model or a patient’s record. This is the difference between an incident and a non-event.
• Deep traceability and auditability, recorded as it happens.Every path reached, every value transmitted, every human approval — written at the time, not reconstructed under questioning. That has to extend to the reasoning itself, including the chain of thought: an agent’s intermediate steps are where it decides what to open next, so a record that omits them cannot explain how the run reached the files it reached. Without this, the honest answer to “what did it see?” is “we cannot say”, and that answer is the one that ends engagements.

Tooling meeting that bar is largely not on the market yet, and we would rather say so than pretend the gap is smaller than it is. It is the standard we hold our own work to: cloaking exists to make the second property true at the boundary, and the Untraceable Protocol specifies the third as an auditable record rather than a promise. Until a tool can show all three, our advice to a regulated team is the unglamorous one: keep it off the machine that holds the confidential material.

Related reading

• Do you have to disclose that AI wrote it? — who actually requires a declaration, by destination, and what each document says.
• PIPEDA, Law 25 and generative AI — the Canadian analysis, including where s. 12.1 starts to bite for autonomous systems.
• Does using AI break your NDA? — the clause-by-clause framework this page assumes.