Privacy before procurement

The dreaded communication, "Can privacy review this AI contract before we go live on Friday?"

For many healthcare privacy professionals, this is becoming a familiar request. The organization has identified a promising tool, watched the vendor demonstration and negotiated pricing. Information security may already have completed its assessment. Operational leaders are eager to begin implementation, and the go-live date has been set.

Privacy is then asked whether the organization can move forward.

The request is usually framed as an AI review. But the hardest questions are rarely about the algorithm itself. They are about the data: what information the tool will receive, where that information will go, how it will be used and whether those uses will expand over time.

In that sense, nearly every healthcare AI project is really a data-sharing project.

Recognizing that point can help privacy professionals move the discussion away from abstract concerns about AI and toward the operational decisions that will determine whether the technology is governed responsibly.

AI changes more than the workflow

Consider a background documentation tool that listens to a patient encounter and generates a draft clinical note.

At first glance, the purpose seems straightforward. The tool reduces the amount of time a clinician spends documenting the visit. The organization may treat it as another software implementation involving protected health information and focus primarily on whether the vendor will sign a business associate agreement.

But the data-sharing questions begin almost immediately.

Is the entire conversation transmitted to the vendor, or only a transcript? Is the audio temporarily processed or stored? Can vendor personnel access recordings for quality assurance? Are the data used to improve the product or train another model? Which subcontractors support the service? What happens when a patient discusses substance use, reproductive health, behavioral health or another category of particularly sensitive information?

The same pattern appears in less obvious AI uses. A patient messaging tool may draft responses using information from the medical record. A revenue cycle system may analyze documentation to recommend billing codes. An analytics platform may combine clinical, demographic and utilization data to identify patients for outreach.

Each use creates a new pathway through which health information is accessed, transformed or disclosed. The AI may be new, but the privacy problem is the same one we’ve been trying to address: organizations need to understand where the data travels and what happens along the way.

The business associate agreement is a starting point

Healthcare organizations are accustomed to asking whether an AI vendor is a business associate and whether a business associate agreement is required. Those are necessary questions, but they are not sufficient.

A business associate agreement establishes important legal responsibilities. It does not, by itself, explain how an AI product works or resolve every question about its use.

For example, a contract may permit a vendor to use information for management, administration or data aggregation. Product terms may separately describe the use of customer data to maintain or improve the service. Technical documentation may reveal that information is retained in system logs or reviewed by personnel when troubleshooting.

None of those practices is necessarily improper. But the organization cannot evaluate them responsibly if it is not fully aware of them.

Privacy teams should therefore review the proposed data use, the product configuration and the operational workflow, not merely the agreement attached to the procurement ticket. The goal is to develop one coherent picture of what the organization is authorizing.

That picture should answer several basic questions: What data does the tool need? Who will receive the data? What new information will the tool create? How long will the information remain available? Can it be used for purposes beyond delivering the contracted service?

When the answers differ between the sales presentation, technical documentation and contract, that is not a minor drafting issue. It is a governance and risk issue.

Data minimization must happen before implementation

AI systems tend to benefit from more data. Privacy programs tend to ask whether all that data is necessary. The tension is not new, but AI makes it easier for organizations to accept broad access without examining the underlying need.

A vendor may request access to an entire clinical record because its product can process it, even though the specific use case requires only a small portion of that record. An operational team may seek historical data to improve model performance without first determining whether identifiers are needed. A pilot may quietly become a production deployment before anyone revisits the original data scope.

Privacy professionals can create significant value by asking a deceptively simple question early: What is the minimum information required for this particular use?

That discussion may lead to limiting data fields, excluding certain records, shortening retention periods or separating a pilot environment from production systems. It may also reveal that the organization has not clearly defined what it expects the AI to accomplish.

Data minimization is not merely a compliance exercise. It forces the organization to connect the information being shared to a defined purpose.

New outputs create new obligations

AI does not only consume information. It creates information.

A summary, risk score, recommended response or predicted outcome may become part of a clinical or administrative workflow. It may influence a decision even if it is never formally added to the medical record.

Privacy governance should account for those outputs.

Who may access them? How are they validated? Can patients request access to them? Are they retained in an auditable system? What happens when an output is wrong, incomplete or based on information that should not have been included?

These questions should not be left entirely to the vendor. The healthcare organization remains responsible for deciding how the output will be used and what role human judgment will play.

An AI-generated result can look authoritative even when it is only a recommendation. A mature governance process makes that limitation visible rather than relying on individual users to remember it during a busy workday.

Governance cannot end when the contract is signed

One of the most difficult features of AI governance is that the product approved today may not be the product used six months from now.

Vendors add capabilities. Models change. Integrations expand. Staff discover uses that were not considered during the original review. A tool purchased to summarize internal documents may later be connected to patient communications or clinical systems.

Traditional vendor reviews often occur at contracting and renewal. AI requires a more active approach, as its rapidly evolving. 

Organizations should establish responsibility for reviewing material product changes, expanded data uses and new integrations. They should also create a practical way for workforce members to raise concerns when a tool behaves unexpectedly or is used beyond its approved purpose.

This does not require a committee meeting every time a vendor updates a feature. It does require a clear owner, defined escalation points and enough visibility to know when the organization’s risk has changed.

Privacy should shape the data strategy

Privacy professionals do not need to become data scientists to contribute meaningfully to AI governance.

Their value lies in understanding how information moves through the organization, how legal permissions differ from patient expectations and how a narrow operational decision can create a much broader pattern of data use.

That perspective is most useful before a vendor is selected and the implementation timeline is announced. By the time privacy receives a contract for final review, the organization may still be able to reduce risk, but its choices are narrower and more expensive.

Healthcare organizations will continue adopting AI. The privacy profession should not respond by treating every project as an unfamiliar technological puzzle.

Start with the data.

Determine what is being shared, what is being created, who will use it and how those practices may change. Those questions are not separate from AI governance.

They are the foundation of it.