A reliable system is designed for more than the ideal case.
A system should be evaluated against the work it must perform, the information it may access and the conditions under which it could fail. Reliability is the combination of product clarity and engineering discipline.
Which failure would change the decision to release?
Make that failure an acceptance case before building. A convincing answer is only one observation; permissions, source validity, retries and recovery are part of the same system.
Open the acceptance dossier.
Illustrative cases, not reported test results. These turn a broad assurance claim into behavior a buyer can ask to inspect in the actual system.
The answer reveals too much
A user without permission asks about a restricted document. Even its title may be confidential.
- Required behavior
- The response must not expose restricted content or metadata. Access checks must apply to retrieval and the resulting answer.
- Evidence to request
- Ask for the same question under different identities, the relevant permission rules and a record showing what was retrieved and disclosed.
The integration repeats the action
A request times out after an order is accepted; a retry sends the same business action again.
- Required behavior
- Retry behavior must match the agreed transaction rules. An ambiguous response must not silently become a duplicate order.
- Evidence to request
- Ask for timeout and retry cases, the duplicate-prevention mechanism and an audit trail that distinguishes request, acceptance and recovery.
The citation is no longer valid
A policy is withdrawn, but the assistant can still find an older copy and cite it as current.
- Required behavior
- The system must follow the agreed source-validity policy: exclude, qualify or abstain when the supporting material is no longer authoritative.
- Evidence to request
- Ask for source withdrawal and refresh cases, the expected answer after the change and the rule for unresolved or conflicting evidence.
Assurance should match the consequence of failure.
A bounded pilot and a regulated production system need different evidence. The operating boundary must be named in both. Internal test results do not constitute independent certification.
Ask to see the acceptance criteria, the negative cases, the unresolved limitations and the person authorized to accept them.
Three delivery choices you can inspect.
A handover should be readable, a release deliberate and a running service inspectable. These small examples show how those principles are implemented on Marmation’s own website.
- 01 / CONTENT CONTRACT
A portable argument.
Your team and its tools should be able to read the same argument. This page has both HTML and Markdown representations from the same reviewed content, with the alternate format identified in the page and HTTP headers.
Open the Markdown edition - 02 / RELEASE CONTROL
A deliberate release.
A draft should not become public by accident. This sitemap lists released pages; held material stays outside discovery and its production route returns an unavailable response. Source content and permission to publish are separate.
Inspect the live sitemap - 03 / OBSERVABILITY
An inspectable runtime.
An operating owner needs a concrete signal to check. This read-only endpoint reports the website service status and released-route count. It shows that this runtime responds; workload resilience and service-level commitments require different evidence.
Check the runtime response
Scope: Marmation’s own website. The Lab separately identifies its concepts and local synthetic demonstrations. For a proposed engagement, ask for evidence relevant to your workload, data and operating scale.
The full argument for engineering quality
For CTO, CIO, CPO, COO
The situation behind the request
An AI assistant works for a privileged tester but leaks a document title to a lower-privileged user. An order integration duplicates a transaction after a retry. A generated answer cites a withdrawn policy. These are system defects even when the model sounds convincing.
Where the obvious solution falls short
Generic claims of “enterprise-grade security” tell buyers little. Security and quality are specific to architecture, operating environment, data, threat model and acceptable business impact.
How we would approach the work
We establish meaningful acceptance criteria before building, create deterministic and model evaluation cases, test access boundaries, simulate failures, maintain observability and define support/rollback behavior.
What the engagement may include
- Scenario-based acceptance and error taxonomy.
- Permission, injection, integration and failover tests.
- Documentation, code review and controlled release practice.
- Monitoring, incident and handover responsibilities.
The choices worth making explicitly
The level of control is proportional to project risk. We will not suggest that a pilot carries the same assurance as a regulated production system, or that passing an internal test equals independent certification.
What we would test before calling it ready
Published security statements must describe controls actually implemented; no SOC 2, ISO, HIPAA or other compliance badge without independent verification. Technical information should remain truthful and current.
A sensible first step
In any project, ask which failure would be most expensive or dangerous and ensure the plan explicitly tests it.