# RAG is not a custom-trained model. That distinction can save a project. > Retrieval, fine-tuning and model hosting solve different problems. Confusing them can turn a useful enterprise AI initiative into an unnecessarily expensive technical program. Canonical URL: https://marmation.com/review/rag-is-not-a-trained-model/ By Vikram Roy · 2026-11-12 Keeping an assistant informed and changing how a model behaves are different jobs. Buying the wrong kind of customization can consume the budget while leaving employees with outdated answers. The distinction starts with where the information comes from. A company has 40,000 internal documents. Its leadership asks for a “custom-trained AI that understands our business.” The phrase is understandable. People associate knowledge with training. But the organization probably does not need to modify a foundation model to let employees search current manuals and policies. It needs the right data available to a model, with permission boundaries and source evidence. That is a different engineering problem. ## What retrieval actually does Retrieval-augmented generation (RAG) creates an application path between source information and a generative model. It identifies relevant source material, includes selected context in a model request and presents a response to the user. The model does not automatically absorb the documents into its own weights. The application can update its index as source information changes. The difficult part is making the retrieved context trustworthy. Source documents may be scanned, duplicated, poorly structured or outdated. A useful pipeline has to parse information, preserve context, index it for search, filter it by permissions and return passages that answer the actual question. Search can compare meanings using numerical representations of text called embeddings. That is only one technique. Combining exact words with meaning-based search, filtering by document attributes, reordering candidate passages and splitting documents along meaningful sections may work better for certain tasks. These are known as hybrid retrieval, metadata filtering, reranking and structure-aware chunking. ## Why fine-tuning is different Fine-tuning changes model behavior using examples. It can help a model produce information in a required structure (an output schema), follow a recurring style or assign the correct category. It does not automatically establish a live connection to changing company knowledge. A policy updated yesterday does not appear in a fine-tuned model unless the relevant system is designed to bring that information forward. Fine-tuning also requires evaluation. Without a representative dataset and measurable improvement, it can be a costly way to solve the wrong problem. ## Where self-hosting fits Self-hosting addresses deployment control and some licensing or network constraints. It can be useful when an organization has a strict environment, predictable operating volume or particular model requirements. But it introduces infrastructure and operational responsibility. It does not make retrieval quality or access controls automatic. A privately served model can still answer from the wrong document or reveal restricted information if the application feeds it material it should not have received. ## The failure no model benchmark catches Suppose a knowledge assistant is configured to answer maintenance questions. The index contains two versions of a service procedure. One is current; the other was withdrawn. Both are semantically relevant. A model can produce a fluent answer using the obsolete procedure. The failure is source governance and retrieval, not a lack of language capability. An enterprise system therefore needs versioning, source authority, citations, freshness and a state for insufficient or contradictory evidence. It may need human escalation for consequential use cases. ## A simple diagnostic Ask these questions in order: Is the information changing? Does the user need references? Are permissions source-specific? Does the task require actions in another system? Is the problem output behavior, knowledge freshness, latency or deployment policy? Which condition can be evaluated with a realistic dataset? If the information changes and evidence matters, retrieval is likely relevant. If the output format is unusually strict and repeated labeled examples exist, selective fine-tuning may be worth testing. If data location is the constraint, solve deployment. If the workflow requires actions, design the application and tool boundaries. ## The case against custom RAG Commercial enterprise search products and managed knowledge bases can already solve many common needs. Their capabilities, licensing and permission models vary. It is often more prudent to configure or extend an existing platform than build an entire retrieval system. Custom engineering becomes justified when source structure, permissions, UX or workflow integration goes beyond the platform's capabilities. **The practical rule:** Start with the job and the information boundary. Choose retrieval, fine-tuning and hosting separately. They are architectural options, not interchangeable promises. ### Sources and limitations [1] Microsoft Learn, *Document-level access control in Azure AI Search*: [https://learn.microsoft.com/en-us/azure/search/search-document-level-access-overview]() (features marked preview require specific review). [2] AWS Bedrock Knowledge Bases documentation: [https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html](). [3] Model/provider documentation is version-specific; validate chosen stack before quoting behavior. ## Sources - [Azure AI Search document-level access]() — Microsoft Learn. Enterprise search can align retrieval to source document permissions when correctly configured. Azure AI Search can support document access patterns; verify identity/preview limitations Limit: Access protection depends on the original permissions, connectors and configuration. Product features and preview status may change; check the version being used. - [Knowledge bases for Amazon Bedrock]() — AWS Docs. Managed retrieval and knowledge base infrastructure available, making simple RAG less differentiated. Limit: Features, charges and regional availability depend on the configuration and should be checked for an actual deployment. ## Related reading - [Private AI doesn't mean training your own model.](https://marmation.com/review/private-ai-does-not-mean-owning-a-model/) - [Permission-aware RAG is an identity project](https://marmation.com/review/permission-aware-enterprise-rag/) - [Evaluate the application, not just the model.](https://marmation.com/expertise/model-evaluation/)