Private AI is not one architecture.
When an organization asks for its own AI, the words can conceal several different requirements. Some teams want company knowledge in an approved assistant. Others need strict retention controls, a private network path or the ability to run inference entirely within infrastructure they manage. The answer should be determined by those constraints, not by a fashionable diagram.
What does private have to mean here?
Turn training, retention, geography, network, access and operational ownership into explicit requirements before choosing a deployment.
Separate the controls that the word private conceals.
- Data use
- May submitted content be retained or used for training?
- Network & region
- Where may processing occur, and through which path?
- Access
- How do source permissions and revocations propagate?
- Operation
- Who owns inference, patching, evaluation and uptime?
More infrastructure ownership is not automatically more useful control.
A private endpoint does not imply a privately owned model. Fine-tuning does not keep documents current. An approved existing tool with a small integration may be the right architecture.
Inspect a revocation, not a privacy slogan.
- A permission and data-flow map
- A removed user and retired document tested end to end
- Retention, logging, responsibility and cost assumptions
The complete private enterprise AI argument
For CIO, CTO, CISO, Head of AI
The situation behind the request
Imagine a manufacturer with years of internal service documents, role-specific procedures and employees across regions. Staff want reliable answers, but IT needs permission-aware retrieval and clarity about where content flows. A product that answers questions but ignores document revocations is not an enterprise solution.
Where the obvious solution falls short
“Self-host everything” can be unnecessary and expensive. “Just connect ChatGPT” may fail data-handling or identity requirements. Even an enterprise API is not the complete application: sources need indexing, citations, updates, authorization and evaluation. The architecture must define what the model can see and whether it can take actions.
How we would approach the work
We begin by writing down the meaning of privacy in this environment: training policy, retention, access, geography, network, ownership, audit and operating responsibility. We examine existing enterprise tools, establish a permission and data-flow map, and compare managed model APIs, secured cloud infrastructure and licensed self-hosted models.
What the engagement may include
- Requirements and data-flow assessment covering identity, data classification and deployment restrictions.
- Evaluation of managed enterprise services against custom integration and self-hosting options.
- A bounded proof using approved/synthetic data and explicit negative access tests.
- Architecture, operating cost, reliability and handover documentation.
The choices worth making explicitly
A private network endpoint does not necessarily mean a privately owned model. A fine-tuned model is not a replacement for current documents. A self-hosted model may provide additional control, but its inference, updates, guardrails and uptime still have to be operated. The right design may be a small custom app around an existing approved platform.
What we would test before calling it ready
Test authorized and unauthorized users, changed group membership, retired documents, malformed uploads, missing citations and model failure. Validate logging and retention against documented policy, not a marketing assurance. Review source-permission synchronization and specific preview/GA features in the chosen stack.
A sensible first step
A private AI architecture decision sprint should produce an option matrix, security questions, a first reference workflow and an honest cost estimate. If the existing enterprise tool solves it, recommend configuring it instead.