How to commission private enterprise AI.

The sponsor says employees must not use public chatbots for company information. The CTO says the application needs company knowledge. The legal team says certain data must stay in a designated geography. Security asks for audit trails and access controls. A procurement officer suggests an on-premises model. Everyone may be describing a valid concern, but those concerns do not automatically require the same architecture.

A private AI project is successful when the resulting system honors an agreed set of constraints. Private is not a binary product attribute.

Translate the word private into requirements

Model training: Must the provider be contractually prohibited from training on inputs? Enterprise API products may provide no-training commitments under specific terms. Verify the actual agreement, retention and subprocessors, not a sales summary.

Data residency: Must input, output, logs, search indexes and backups stay in a named region? Evaluate every component, including telemetry, embeddings, indexing and support paths. A regional inference endpoint does not necessarily govern every other data flow.

Infrastructure control: Is an approved cloud virtual private network sufficient, or must the model run entirely inside a controlled customer environment? Dedicated endpoints, customer-managed keys and private networking are different choices from self-hosted model weights.

Organizational knowledge: Does 'our AI' mean employees can ask questions over internal documents, systems and policies? That often calls for retrieval and application engineering, not a new model trained from scratch.

Choose the smallest viable architecture

Start with a governed business question and authorized users. Identify the systems holding the source data, retention requirements, response quality, expected load, budget, model limitations and the human decisions supported. Then compare a packaged enterprise assistant, a private-cloud application using an approved provider model, a hosted open-weight model and a self-managed deployment.

The right answer can be counterintuitive. A managed enterprise assistant may already satisfy many access and procurement requirements. A custom RAG application can create a second data store and permission synchronization problem. A self-hosted model can reduce exposure to an external inference endpoint but increase operational responsibility and GPU costs.

Questions for the technical architecture review

Where does user identity originate? Which roles and source documents can each person access? Can permissions be revoked promptly? How are citations established? What information is logged and who can inspect it? Who evaluates incorrect or harmful answers? What happens when the approved model version changes? Can the application refuse to answer? What will operators do if inference is unavailable?

For applications that initiate actions, add tool permissions, approval roles, read/write separation, idempotency and audit history. The model is not a substitute for application authorization.

A good first engagement is narrow

Instead of commissioning a generic 'company ChatGPT,' select one knowledge domain and one employee task. Establish a representative source corpus, permission groups, current baseline and evaluation set. Test retrieval quality and forbidden-access cases before polishing the assistant interface. Decide which architecture can meet the requirements with reasonable operating cost.

The outcome of the first phase should be a documented build/buy/extend decision and an implementation scope. It may also correctly recommend that the company continue using its current enterprise platform rather than building something new.

The decision: Specify data use, identity, geography, infrastructure and workflow control separately. Only then decide whether the company needs an enterprise service, a private application or a self-hosted model.

References and limits

Microsoft Learn, Azure AI Search document permissions; AWS Bedrock knowledge bases; NIST Gen AI risk profile. Provider terms, controls and regional availability need revalidation at contracting.