What production-ready AI actually means.

A model extracts most of the fields from a purchase order. The demo runs in front of a delighted executive team. The engineer opens a cloud bill and sees an acceptable number. The product manager puts a launch date on the roadmap. Then the system encounters a document in an unfamiliar language, a revoked employee permission, a failed API call after a successful write and a user who needs to correct a misidentified account. The demonstration answered none of those questions.

We use the term production-ready too easily. It sounds like the final step of development, as though a prototype becomes a product when it is hosted behind a login. In reality, the decision to put AI into production begins before the first integration. It is an agreement about what the application can do, what it must not do, and how the organization will know whether those boundaries are being maintained.

The product must have a job, not just a prompt

Begin with a user task. 'Answer internal questions' is a category, not a user task. 'Help a service coordinator identify the latest approved procedure for repairing this equipment, with the source and effective date' is closer to an implementable job. It names the user, material, expected output and an important attribute of correctness.

That specificity changes architecture. The latest procedure may live in a repository with versioned documents. Some procedures are restricted by region or role. The answer may need an explicit statement that the source is incomplete. A successful prototype should test these details, not only whether the language sounds plausible.

For a commercial AI product, the task also belongs to a product experience. The user expects a history, a way to correct the result, a stable state across devices and a support path. A system that responds fluently but loses the user's review decisions has a product problem, even if its model is excellent.

Boundaries decide reliability

A useful production diagram should show at least five boundaries: the authenticated user, the application, the relevant source data, the model and the systems that can take consequential actions. These are not interchangeable.

The application should authorize a user to request a particular operation. Source retrieval should respect identity and document permissions before restricted content is sent to inference. Model output should be treated as untrusted until checked against domain-specific rules. A write to a CRM or ERP should happen through a typed, permissioned operation whose behavior does not depend on persuasive model text.

The system must be able to say 'I cannot answer from authorized evidence' or 'This request requires review.' That is not a failure of AI. It is a deliberate state in a reliable application.

A test suite should know how the application fails

A percentage score on a handful of friendly examples is not a release criterion. Build a dataset from representative cases and the situations that make the application struggle: ambiguous entities, conflicting documents, multilingual input, stale source material, adversarial instructions, revoked access and integration outages.

Evaluate distinct qualities separately. Retrieval coverage is not the same as source permission correctness. Extraction precision is not the same as transaction correctness. Human reviewer acceptance is not the same as an independent quality label. You cannot combine everything into one confidence score without hiding useful information.

Test the application with the model unavailable. For some jobs, a manual path is an acceptable fallback. For others, the right choice is to pause the workflow. Document which behavior is expected and practice it.

Operations begins at deployment

A functioning AI application needs logs, cost accounting, rate and timeout controls, incident ownership, versioned prompts and model identifiers, data retention decisions, monitoring, deployment rollback and an evaluation process for model or source changes. None of these are exotic features. They are the ordinary responsibilities of software operations adapted to a probabilistic component.

Someone must own the incident when the system suggests a wrong value. Someone else may approve a model upgrade, but a clear process should link the change to regression tests. If a system performs actions, its audit history needs to connect the user, proposed action, authorization, final API response and recovery state.

The commercial consequences are immediate. A cheaper model that requires more manual correction may be more expensive to operate. A faster extractor that creates duplicate ERP transactions is not more efficient. The useful measurement is the number of correct tasks completed, including review, error handling and operational cost.

The final readiness review

Before declaring a system ready for real work, require evidence of the following: defined users and tasks, authorized source boundaries, representative evaluation cases, negative tests, system-of-record behavior, failure recovery, accessibility, monitoring, incident ownership, support and a documented handover. The exact depth varies by the risk of the application. Not every internal draft assistant needs the same controls as a payment or healthcare workflow.

The decision: Put AI into production when the entire application has clear and tested responsibilities. A strong model is one prerequisite. It is not the definition of readiness.

Sources and limitations

NIST AI RMF Generative AI Profile; OWASP LLM Top 10; Thoughtworks Technology Radar. This article describes an engineering decision framework, not a Marmation client's deployment results.