# The case for, and against, model-provider abstraction.

> Keeping the option to change AI suppliers can protect a product. Decide which dependencies matter without adding layers that make the software harder to understand.

Canonical URL: https://marmation.com/review/model-provider-abstraction/

By Vikram Roy · 2026-11-12

The ability to change AI suppliers can protect the business from price, availability or contractual changes. Building that flexibility also costs money and may limit useful features. The architecture should protect a specific business dependency rather than promise effortless switching.

A team wants to make its AI product model-agnostic from the first week. It designs a wrapper that exposes the same interface for every model, tool format and output capability. The abstraction is elegant on paper but eliminates useful features and complicates testing.

## Model services are not identical parts

Different models can have different structured-output behavior, context limits, tool calling, cost, regional availability and latency. Treating them as interchangeable at every level can either weaken the application or create a costly lowest-common-denominator platform.

## Separate the pieces that actually need to move

Separate the parts of inference, the process of asking a model to produce a result, that the product truly needs to switch: request/response contracts, credentials, error handling, logging and evaluation. Isolate provider-specific capabilities explicitly rather than pretending they do not exist. Use a representative benchmark to determine which portability matters.

## Test the cost of switching on your own task

A knowledge assistant might be portable across two approved model services with little UI change. A sophisticated streaming agent with provider-specific tool behavior may require more adaptation. Document the switching cost rather than promising one-click migration.

## When a direct connection is enough

In a single-use application with stable requirements, direct integration may be simpler and safer. For products with sovereignty, pricing or resilience constraints, a defined provider boundary can be valuable.

## Name the dependency worth protecting

Before building an abstraction layer, identify the real risk being insured: cost, continuity, region, legal terms or model quality. Keep architecture proportional to that risk.

### References and context

[McKinsey 2026 AI survey](<https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai>); [NIST AI RMF](<https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence>).

## Sources

- [The state of AI in 2026](<https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai>) — McKinsey. AI adoption/agentic coding and build-vs-buy survey; 32% of respondents had foregone a product/feature purchase due to ability to build with coding agents. Limit: The findings apply to the survey’s questions and population. They do not establish that software-as-a-service products are obsolete.
- [Generative AI Profile for AI RMF](<https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence>) — NIST. Reference governance and risk design principles for gen AI. Risk management reference for generative AI Limit: This is a risk-management reference, not a certification or evidence that Marmation has been audited or complies with it.

## Related reading

- [From working prototype to dependable system](https://marmation.com/review/engineering/)
- [Design the action boundary before designing the agent.](https://marmation.com/review/designing-an-agent-with-bounded-actions/)
- [Engineering autonomy is a control design problem.](https://marmation.com/review/engineering-autonomy-is-a-control-design/)