# Private AI doesn't mean training your own model. > “We need our own AI” is a perfectly reasonable business request. It is also an incomplete technical requirement. Ownership of a model, privacy of data and control of an application are three different questions. Canonical URL: https://marmation.com/review/private-ai-does-not-mean-owning-a-model/ By Vikram Roy · 2026-11-12 A company can want more control over its information without wanting to operate an AI model. Separate the privacy requirement from the ownership decision: each can lead to a different contract, system design and operating budget. A leadership team decides its employees should have an internal assistant. The company cannot simply send confidential documents to any consumer account. Someone suggests building a private large language model. Another proposes fine-tuning. A third points to a Microsoft enterprise product. Soon the discussion is about the graphics processors (GPUs) used to run models, even though no one has agreed on the original restriction. This is a common failure of translation. The business describes the control it wants using the language of the technology it has heard about. An architect's first job is to separate the underlying needs. ## 01. What exactly must stay private? Privacy might mean that inputs are not used to train a provider's foundation models. It might mean limited retention, processing in a named region, access through a private network connection, source documents remaining inside the client's storage or no processing by an external model service at all. The required architecture changes with each meaning. A commercial enterprise API may satisfy the first requirement under appropriate terms without meeting the last. A model hosted in an approved cloud account can offer additional control, but still depend on licensed infrastructure and managed services. A fully self-hosted model with downloadable parameters (an open-weight model) may address network restrictions and where results are generated. It also creates obligations to install fixes, serve requests, monitor reliability, test quality and provide enough capacity. None of these options automatically provides a correct answer or enforces an employee's document permissions. Those are application responsibilities. ## 02. Data ownership is not model ownership If a company owns its policies, manuals and transactional data, it can connect that information to an approved model through retrieval. The model can use relevant excerpts as context for a response, without being trained on all of the documents. This pattern is called retrieval-augmented generation (RAG). A custom RAG application can be genuinely company-specific even if it uses a commercial foundation model. The custom elements may include ingestion, access control, ranking, citation, user experience, tools, integrations, tests and operating procedures. Those elements can be more strategically valuable than the model itself. The company's data remains subject to its own governance and the chosen service's contractual terms. Nothing in a RAG design should be described as “100% secure” by default. ## 03. Fine-tuning is not a document search strategy Fine-tuning adjusts a model's behavior using additional training examples. It may help with format, classification, tone or specialized tasks where labeled examples and clear evaluation exist. It is not generally the best way to give an assistant rapidly changing manuals, policies or product information. Updating that knowledge through retrieval is usually more direct and easier to govern. Training a new foundation model from scratch is a separate, far larger research and compute undertaking. Most mid-market business problems do not require it. ## 04. The difficult problem is often permission-aware retrieval Suppose an HR document is visible to one employee but not another. A search index that contains it must preserve that distinction. The application should not retrieve the content and then ask the model to keep it secret. Authorization belongs before information reaches the model context, with additional enforcement in the application. Microsoft's Azure AI Search documentation describes document-level access control patterns, including filters and native permissions features. Some capabilities may still be in preview or subject to specific limitations, so the engineer needs to verify the actual version and source synchronization behavior. [\[1\]]() Source content also changes. A revoked permission or withdrawn policy needs to propagate through the retrieval layer. Otherwise, a privately hosted system may still reveal material it should no longer use. ## 05. A practical architecture decision Start by documenting the data classes, allowed users, hosting/network restrictions, geography, retention, vendor agreements, model access, retrieval freshness, required actions and acceptable operating cost. Evaluate existing enterprise platforms, approved managed models and private hosting against those constraints. Then test one important use case with people who should and should not have access. The test should include a good answer with verifiable sources; a no-answer case; an inaccessible document; changed permissions; an obsolete source; and a deliberate adversarial instruction embedded in retrieved material. If the system cannot handle those, 'private' does not make it ready. ## The case for not building An organization's Microsoft, cloud or knowledge platform may already satisfy common search and assistant needs. Configuring the existing service can be faster and easier to operate than a custom stack. A bespoke application is justified when the workflow, access model, UX, integration or deployment requires capability the commercial platform cannot provide economically. **The rule:** Specify the privacy and control requirement, then choose the simplest architecture that satisfies it. Model ownership is not the starting point. ### Sources and limitations [1] Microsoft Learn, *Document-level access control in Azure AI Search*, current page (review preview/GA states and sync caveats). [2] NIST, *AI RMF Generative AI Profile*, July 2024, [https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence]() [3] OpenAI, *Enterprise privacy* and business data terms; specific configuration/contract matters. [https://openai.com/enterprise-privacy/]() [4] Anthropic, *Commercial terms and privacy*; verify current contract. [https://privacy.claude.com/]() ## Sources - [Azure AI Search document-level access]() — Microsoft Learn. Enterprise search can align retrieval to source document permissions when correctly configured. Azure AI Search can support document access patterns; verify identity/preview limitations Limit: Access protection depends on the original permissions, connectors and configuration. Product features and preview status may change; check the version being used. - [Generative AI Profile for AI RMF]() — NIST. Reference governance and risk design principles for gen AI. Risk management reference for generative AI Limit: This is a risk-management reference, not a certification or evidence that Marmation has been audited or complies with it. - [Enterprise data and privacy]() — OpenAI. Business product policies describe model training defaults and controls Limit: Data handling depends on the contract, region, retention settings and product used. - [Commercial data privacy]() — Anthropic. Commercial data arrangements and model training policies Limit: Data handling depends on the applicable commercial contract and current retention terms. ## Related reading - [RAG is not a custom-trained model. That distinction can save a project.](https://marmation.com/review/rag-is-not-a-trained-model/) - [Permission-aware RAG is an identity project](https://marmation.com/review/permission-aware-enterprise-rag/) - [You can own the application without owning the model.](https://marmation.com/review/what-owning-ai-actually-means/)