How to Build Generative AI on Your Company’s Data Without Leaking It

The most useful generative AI tools inside a company are the ones that know the company: its contracts, product specs, support history and policies. They are also the ones that make security teams nervous, and for good reason.

A public chatbot that knows nothing about your business can’t leak much. An assistant connected to your file shares, CRM and ticketing system can, if it is built carelessly. The good news is that the risks are well understood, and so are the controls. Here is where data actually leaks in generative AI projects, and how to build so it doesn’t.

Where the leaks actually happen

Most people picture a model “memorizing” secrets and repeating them to strangers. That is rarely the real problem. In practice, data leaks through five routes.

1.      The AI provider keeps your data. Prompts and documents are sent to a model provider whose terms allow storage or training, or nobody checked the terms.

2.      The assistant ignores permissions. A search index is built from every document in a shared drive, so an intern can ask about salaries and get an answer.

3.      Prompt injection. A document or email contains hidden instructions, and the assistant follows them, for example by sending data to an outside address.

4.      Over-broad tools. An assistant that can send email, update records or call APIs does more than it should because its permissions were never narrowed.

5.      Logs nobody thinks about. Conversations, retrieved passages and debug traces sit in logs with weaker access controls than the source systems.

Notice that only the first is about the model itself. The rest are about how the system around it was designed.

Control 1: Choose where the model runs

You have three broad options: a commercial model through an enterprise agreement, the same model through your own cloud account (for example on AWS, Azure or Google Cloud), or an open-source model you host yourself.

For most companies, a commercial model in their own cloud account is the practical middle ground. Data stays inside an environment your security team already governs, and enterprise terms typically exclude your data from training. Self-hosting gives the most control but adds real operational work. Whatever you choose, read the data retention and training terms before the first document is sent.

Control 2: Retrieve, don’t retrain

Many teams assume they must fine-tune a model on their documents. Usually they shouldn’t. Retrieval-augmented generation, often called RAG, keeps documents in a search index and passes only the relevant passages to the model at the moment of the question.

This matters for security. Data you never put into model weights can’t come back out of them, and you can delete or update a document in the index at any time. Fine-tuning still has a place, mainly for style and specialized formats, but it is rarely the right way to give a model access to sensitive facts.

Control 3: Carry permissions through to every answer

The single most common mistake in internal assistants is a search index that ignores who is asking. The fix is to store access rights with each document chunk and filter results by the user’s permissions before anything reaches the model.

If a person can’t open a file in your document system, the assistant shouldn’t quote it to them. Test this with real accounts at different access levels, not just an admin login.

Control 4: Treat every document as untrusted input

Prompt injection works because the model can’t reliably tell the difference between your instructions and text inside a document. Reduce the damage it can do:

•       Give the assistant the smallest set of tools it needs, with read-only access wherever possible.

•       Require a person to confirm any action that sends data outside the company or changes a record.

•       Strip or flag hidden text and unusual instructions in retrieved content.

•       Log tool calls separately so unusual behavior is easy to spot.

Control 5: Design logging and retention on purpose

Decide what you log, who can read it and how long you keep it. Retrieved passages in logs should carry the same access restrictions as the source documents. For regulated data, such as patient records under HIPAA or personal data under GDPR, agree these rules with your compliance team before launch, not after an audit.

Questions to ask any vendor

If you are working with a partner on a generative AI project, ask them to answer these in writing before any data leaves your systems:

1.      Which model provider will we use, and what are its data retention and training terms?

2.      Where will our data, indexes and logs be stored?

3.      How do you enforce our existing document permissions?

4.      What can the assistant do on its own, and what needs human approval?

5.      How will you test for prompt injection before launch?

6.      Who owns the code, prompts and configuration when the project ends?

A vendor who struggles with these questions has not thought hard about your data. You can find our own checklist and approach on building generative AI applications.

Start with one contained use case

The safest way to begin is with a narrow use case and a small group of users. An internal assistant over product documentation, for example, carries less risk than one connected to HR files and email. Prove the controls work there, then widen access.

We see the same pattern across projects at aidevelopmentcompany.com: the teams that move fastest are the ones that agree on data rules first. Once security knows exactly where data goes and who can see what, approvals stop being the bottleneck.

The bottom line

Generative AI on your own data is not inherently risky. Careless architecture is. Keep data in an environment you control, retrieve instead of retraining, enforce permissions on every answer, limit what the assistant can do and design logging on purpose. Do those five things and your security team becomes a partner in the project instead of the reason it stalls.

About the Contributor

Nishkam Batta, Editor-in-Chief, HonestAI Magazine | AI Consultant, GrayCyan AI Solutions

Nish leads an applied AI company helping manufacturing and related companies automate operations with human-in-the-loop AI that integrates into ERPs, WMS, CRMs, and other enterprise tools, with an emphasis on explainable AI, clear audit trails, and measurable outcomes. His team builds agentic ERP systems that execute multi-step tasks inside approved guardrails so humans keep accountability, approvals, and override control.

Book a Consultation  |  LinkedIn

- Advertisement -
- Advertisement -

MTA Announces Festive Light Trains and Extra Railroad Service for Holiday Season

rainTime App Will Display Holiday Lights Around Decorated Trains MTA...

Yonkers Resident Shoots and Kills Man, Flees Upstate, Arrested

On Tuesday, November 17th, 2026, at approximately 1:35 AM,...

How to Choose the Best Transaction Monitoring Solution

Learn how to choose the best transaction monitoring solution...

On this day in Yonkers history…

Lewis Yancey and Roger Williams 1929 by International News...
- Advertisement -
- Advertisement -

Related Articles