# How to commission private enterprise AI. > 'We need our own AI' can describe four different requirements. The first project decision is to find out which one the business actually has. Canonical URL: https://marmation.com/review/how-to-commission-private-ai/ By Vikram Roy · 2026-11-12 A private AI brief needs to say what information is protected, from whom and under which conditions. Otherwise a buying group can agree on the word “private” while funding a system that solves the wrong restriction. The sponsor says employees must not use public chatbots for company information. The CTO says the application needs company knowledge. The legal team says certain data must stay in a designated geography. Security asks for audit trails and access controls. A procurement officer suggests an on-premises model. Everyone may be describing a valid concern, but those concerns do not automatically require the same architecture. A private AI project is successful when the resulting system honors an agreed set of constraints. Private is not a binary product attribute. ## Translate the word private into requirements **Model training:** Must the provider be contractually prohibited from training on inputs? Enterprise API products may provide no-training commitments under specific terms. Verify the actual agreement, retention and subprocessors, not a sales summary. **Data residency:** Must input, output, logs, search indexes and backups stay in a named region? Evaluate every component, including telemetry, embeddings, indexing and support paths. A regional inference endpoint does not necessarily govern every other data flow. **Infrastructure control:** Is an approved cloud virtual private network sufficient, or must the model run entirely inside a controlled customer environment? Dedicated endpoints, customer-managed keys and private networking are different choices from self-hosted model weights. **Organizational knowledge:** Does 'our AI' mean employees can ask questions over internal documents, systems and policies? That often calls for retrieval and application engineering, not a new model trained from scratch. ## Choose the smallest viable architecture Start with a governed business question and authorized users. Identify the systems holding the source data, retention requirements, response quality, expected load, budget, model limitations and the human decisions supported. Then compare a packaged enterprise assistant, a private-cloud application using an approved provider model, a hosted open-weight model and a self-managed deployment. The right answer can be counterintuitive. A managed enterprise assistant may already satisfy many access and procurement requirements. A custom retrieval-augmented generation (RAG) application, which finds source passages for a model to use in an answer, can create a second data store and permission synchronization problem. A self-hosted model can reduce exposure to an external inference endpoint but increase operational responsibility and GPU costs. ## Questions for the technical architecture review Where does user identity originate? Which roles and source documents can each person access? Can permissions be revoked promptly? How are citations established? What information is logged and who can inspect it? Who evaluates incorrect or harmful answers? What happens when the approved model version changes? Can the application refuse to answer? What will operators do if inference is unavailable? For applications that initiate actions, add limits on connected tools, approval roles and separate permissions to read information and change records. Repeated requests must not repeat the business change (idempotency), and the history must show what happened. The application must enforce these permissions independently of the model. ## A good first engagement is narrow Instead of commissioning a generic 'company ChatGPT,' select one knowledge domain and one employee task. Establish a representative source corpus, permission groups, current baseline and evaluation set. Test retrieval quality and forbidden-access cases before polishing the assistant interface. Decide which architecture can meet the requirements with reasonable operating cost. The outcome of the first phase should be a documented build/buy/extend decision and an implementation scope. It may also correctly recommend that the company continue using its current enterprise platform rather than building something new. **The decision:** Specify data use, identity, geography, infrastructure and workflow control separately. Only then decide whether the company needs an enterprise service, a private application or a self-hosted model. ### References and limits [Microsoft Learn, Azure AI Search document permissions](); [AWS Bedrock knowledge bases](); [NIST Gen AI risk profile](). Provider terms, controls and regional availability need revalidation at contracting. ## Sources - [Azure AI Search document-level access]() — Microsoft Learn. Enterprise search can align retrieval to source document permissions when correctly configured. Azure AI Search can support document access patterns; verify identity/preview limitations Limit: Access protection depends on the original permissions, connectors and configuration. Product features and preview status may change; check the version being used. - [Knowledge bases for Amazon Bedrock]() — AWS Docs. Managed retrieval and knowledge base infrastructure available, making simple RAG less differentiated. Limit: Features, charges and regional availability depend on the configuration and should be checked for an actual deployment. - [Generative AI Profile for AI RMF]() — NIST. Reference governance and risk design principles for gen AI. Risk management reference for generative AI Limit: This is a risk-management reference, not a certification or evidence that Marmation has been audited or complies with it. ## Related reading - [Ideas that stand up to implementation.](https://marmation.com/review/) - [The choices behind technology investments](https://marmation.com/review/decisions/) - [Build, buy, extend or integrate? The AI decision that comes before the technology.](https://marmation.com/review/build-buy-extend-or-integrate/)