By Alfie Lambert · · 9 min read
Private AI for business: how to give your AI everything your company knows, without it leaving the building
In the late 1990s, Hewlett-Packard's chief executive Lew Platt summed up a problem every large company recognises: "If only HP knew what HP knows."
HP had tens of thousands of clever people. Somewhere in the building, someone had already solved most of the problems the rest of the company was stuck on, but nobody could find them.
Nearly thirty years later, most companies are trying to use AI to fix exactly this. And most of that AI still doesn't know what the company knows.
The smartest new joiner you'll ever hire
Here's how I think about company AI.
You've hired the most capable new joiner in history. They've read most of the internet, they write beautifully, and they answer anything in seconds without ever needing a coffee.
Then nobody gives them an employee handbook.
They don't know your clients, your pricing, last year's post-mortem or the deal you walked away from. So when someone asks a question, they do what any bright new starter does when they're out of their depth. They answer confidently from general knowledge and hope for the best.
I watched this play out with a deal advisory team recently. A partner asked their AI assistant whether the firm had ever looked at a business like the one on their desk. It had, twice, back in 2019. The analysis was sitting in an email thread and a shared drive folder the assistant had never been allowed near.
So it answered from the public internet. Fluent, polite and absolutely no use whatsoever.
The obvious fix is to hand the assistant the archive. That's also the one thing IT will never sign off, because twelve years of client deal work does not get uploaded to somebody else's cloud. Fair enough.
What is private AI for business?
Private AI for business is AI that works from your company's own knowledge while that knowledge stays inside infrastructure you control.
It has three moving parts, and most companies sort out the first and stall on the other two.
The model does the reading and writing. It can be a hosted model under an enterprise agreement, or an open model on your own servers.
The knowledge is everything the company knows that the model doesn't: past projects, policies, client history, pricing, decisions, and who worked on what.
The access layer connects the two. It decides what the model can reach, on whose behalf, and where the data travels along the way.
Nearly all the risk lives in that access layer, yet almost all the attention goes on the model.
Why your AI can't find anything
Company knowledge was never organised for AI. It grew wherever it was convenient at the time.
A proposal is in SharePoint. The reasoning behind the proposal is in an email thread between three people, one of whom left last spring. The final numbers ended up in a spreadsheet called something like "v7 FINAL2" (be honest, we all have those). And the context that ties it all together came out on a call, so it now lives in a transcript nobody will reopen, or in one person's head.
Humans get by on this system, because they know who to ask. As one head of knowledge put it to us, everyone knows who to ask and nobody knows where to look. Your AI has neither option.
Two things make this much harder than an ordinary search problem.
Permissions, for a start. A consultant on one client team must never see another client's work and an underwriter shouldn't see claims they aren't cleared for. Any AI that touches company knowledge has to respect the same boundaries your people do.
Then there's geography. Client contracts, regulators and your own security team all have firm views on where data is allowed to go. For a lot of firms, "send it to a third-party cloud to be indexed" is dead on arrival. Long before anyone books a demo.
Three ways companies try to fix it
1. Upload it to a chatbot
The quickest fix is dragging files into ChatGPT, Claude or Copilot projects.
For one person on one task, it's great. Across a company it turns into hundreds of private copies, each going stale the moment it's uploaded, with no permissions and no record of who shared what. It also sends the data exactly where your security team said it couldn't go.
2. Buy enterprise AI search
Enterprise AI search tools connect to your systems and let people ask questions across all of them.
The snag is the architecture. Most work by indexing a copy of your content, and the best-known ones keep that index in their own cloud. For firms with strict data rules, that copy is the whole problem. It also only feeds the search tool, so the rest of your AI is still in the dark.
3. Build your own RAG on a private LLM
Technical teams go further. They run an open model on their own servers, pour documents into a vector database and build retrieval on top.
The data stays in-house, and for one use case it works well. It's also a proper engineering project. Every new tool needs its own pipeline, permissions get rebuilt by hand, and the vector store is one more copy of your company's knowledge to guard and keep fresh.
| Approach | Where your data goes | Permissions | Stays current | Works across AI tools |
|---|---|---|---|---|
| Uploading documents to a chatbot | Copied into the chatbot provider's cloud | None. Anyone with the file sees it all | No. Stale the day it's uploaded | No. One copy per tool |
| Enterprise AI search | Copied into an index, held in the vendor's cloud | Synced from your systems | Depends on how often the index refreshes | No. Serves the search tool only |
| DIY RAG on a private LLM | Copied into your own vector store | Rebuilt by hand for each pipeline | Only if you rebuild the pipeline | No. One pipeline per tool |
| Knowledge gateway | Stays where it already lives, inside your environment | Checked on every request | Yes. Reads from the live source | Yes. One way in for every tool |
A private LLM is only half the answer
This is the misunderstanding I run into most.
A private LLM is a model you run yourself, or one hosted under terms that keep your prompts out of training and inside a region you choose. That's a real problem solved: you control where the model lives and who sees what you send it.
What it doesn't change is what the model knows. A Llama or Mistral deployment humming away in your own data centre knows exactly as much about your business as a public chatbot does. Nothing at all.
You need both halves. A private model with no route to company knowledge is safe and useless. A route to company knowledge with no control over where it goes is useful and terrifying.
Build a GPS, not a warehouse
Every fix above shares one habit. Before the AI can use your knowledge, it copies it somewhere new. Copies go stale, need guarding and multiply with every tool you add, and they're precisely what your data rules were written to stop.
My co-founder Adil puts the alternative neatly: build a GPS, not a warehouse. A GPS doesn't store the roads. It knows where everything is, the quickest way there, and which roads you're allowed down.
A knowledge gateway does the same job for company knowledge.
First, it connects to the places knowledge already lives, like SharePoint, email, drives, the CRM and call transcripts, without bulk-copying any of it.
Every request gets a permissions check, so each person and each AI tool sees only what they're cleared to see.
Answers come back with a link to the original source, so people can check the work and read the full context.
And the whole thing runs inside your environment, so nothing leaves the building.
Because the gateway sits between your knowledge and your AI tools, it doesn't care which tools you use. Claude, Copilot and the agents your engineers build all come through the same front door, under the same rules.
How to judge a private AI setup
Whatever you build or buy, hold it up against these six tests.
- It runs in your environment, whether that's your cloud tenancy, your data centre or a fully air-gapped network.
- It uses the permissions you already have, rather than asking you to rebuild them from scratch.
- Every answer links to its source, so people can check it.
- It works with more than one AI tool, so your knowledge isn't tied to one vendor's roadmap.
- It doesn't need a full copy of your data to function.
- It keeps an audit trail of who asked what, and which sources answered.
If a setup fails the first two, it isn't private, whatever the sales deck says.
Where this matters most
Consultancies sit on years of client work and a nagging feeling that someone, somewhere, has already built the slide they need, and finding it must never mean one client's material leaking into another client's project.
Deal and investment teams carry a decade of analysis in inboxes and shared drives. It's the firm's edge, and it's locked in a cupboard.
Insurers and other regulated firms hold policies, claims history and hard-won underwriting judgement under strict rules about where data can go.
Government and public bodies need everything inside their own walls, frequently with no outside connection at all.
Where Oiya fits
This is the problem we built Oiya to solve.
Oiya is a knowledge gateway. It connects to the sources your company already uses, respects who can see what, and gives every AI tool one governed way in. Answers point back to where they came from, and you choose where it runs, including entirely inside your own environment.
We're working with a small number of design partners in advisory, consulting and regulated industries. If your AI is the smartest new joiner nobody gave a handbook, let's talk.
Get a demoThirty years on, it's about time HP knew what HP knows.
FAQ
What is private AI for business?
Private AI for business is AI that works from your company's own knowledge while that knowledge stays inside infrastructure you control. It combines a model, access to company knowledge, and a layer that controls what the model can reach and where the data goes.
Is a private LLM enough?
No. A private LLM controls where the model runs and what happens to your prompts. It gives the model no access to your company's knowledge, which is where most of the value sits.
Do I need to run AI models on my own servers?
Only if your data rules require it. Plenty of firms use hosted models under enterprise terms and keep the knowledge layer in their own environment. Others, in government or defence for example, need everything air-gapped.
How is a knowledge gateway different from enterprise search?
Enterprise search tools index a copy of your content to power their own search box. A knowledge gateway leaves content where it lives and gives every AI tool you use the same controlled, permission-checked access.
Can private AI work with Claude or Microsoft Copilot?
Yes. A gateway sits between your knowledge and your AI tools, so Claude, Copilot and internally built agents can all use the same company knowledge under the same permissions.
Related reading
- The multi-agent knowledge problem
Why agents in the same company give confidently different answers.
- How to switch AI models without losing company knowledge
Keeping company knowledge separate from the model layer.
- Oiya vs enterprise search
How a knowledge gateway differs from an indexed search tool.