Useful retrieval is a system, not a vector database.
A strong retrieval pipeline must find the relevant evidence, preserve its meaning and deliver it to a model within time and cost constraints. Embeddings are one component, not the entire solution.
Can the system retrieve the qualification as well as the rule?
Judge the evidence path before the answer. A fluent response cannot repair a missing exception, an obsolete manual or a table split in the wrong place.
Five places where meaning can be lost.
Work backwards from a representative support question. At every boundary, retain enough structure to explain why this passage became evidence.
Parse
Keep the table, its heading and its qualification together.
Inspect this boundary
A source-to-extracted-document comparison, including tables, diagrams and damaged input.
Partition
Preserve meaning across chunk boundaries.
Inspect this boundary
The rule and exception in adjacent chunks, with document, section and version identifiers.
Retrieve
Apply source, version and access constraints before ranking evidence.
Inspect this boundary
Candidate passages from sparse and dense retrieval, including the relevant passage that was missed.
Rerank
Prefer the passage that answers this task, not simply similar wording.
Inspect this boundary
Before-and-after candidate order on exact values, exceptions and cross-document questions.
Ground
Every material answer claim needs accessible, applicable evidence.
Inspect this boundary
A passage-level citation and an explicit check that the cited version applies to the question.
No answer is sometimes the correct output.
- No applicable source was found.
- Authoritative sources conflict or the version is ambiguous.
- The only supporting source is inaccessible to this user.
- Corrupted ingestion makes the qualification unreliable.
Illustrative quality chain for technical manuals; a design argument, not a measured deployment.
Ask to inspect the missed passage.
- A versioned query set with expected evidence and explicit missing-answer cases.
- Separate records for retrieval misses, grounding failures, freshness and response time.
- Manually reviewed failures, with the source passage and ranking trace attached.
Custom retrieval must earn its complexity.
Managed retrieval may be sufficient for straightforward sources. Build specialized parsing, filters or ranking when representative failures demonstrate that the simpler path cannot preserve the task's meaning.
The retrieval engineering argument
For CTO, CIO, CPO, COO
The situation behind the request
A technical support organization has manuals with tables, versions, diagrams and exceptions. A naive text chunker separates the qualification from the rule and causes a confident wrong answer.
Where the obvious solution falls short
Weak parsing, chunk boundaries, missed metadata and ambiguous source authority can hurt quality before the model ever reads the material. Longer contexts are not a substitute for precise evidence.
How we would approach the work
Evaluate source preprocessing, structural chunking, sparse and dense retrieval, reranking, citations and versioning on representative queries. Use task-specific metrics and manually inspected misses.
What the engagement may include
- Parsing and provenance pipeline for target document types.
- Hybrid retrieval/reranking design with filters and source metadata.
- Answer grounding and clear citation interface.
- Quality, freshness, permissions and latency evaluation.
The choices worth making explicitly
Built-in managed RAG may be enough for simple sources; bespoke retrieval is more useful for specialized document structures, fine-grained filtering or domain-specific ranking.
What we would test before calling it ready
Test exact-value lookup, cross-document synthesis, missing answers, version drift, inaccessible documents and corrupted ingestion. Record source passage quality separately from answer fluency.
A sensible first step
Build an evaluation set before optimizing search. Without known queries and expected evidence, better retrieval cannot be distinguished from a more persuasive answer.