Diagnose a failing RAG workflow
Use observable evidence to separate source, ingestion, retrieval, permission and generation failures.
Start from a failed request
Keep a sanitized question, user role, source versions, retrieval configuration and expected outcome. Capture retrieved document IDs and passages separately from the final answer. Avoid putting credentials or confidential documents in a shared debugging ticket.
| Symptom | First evidence to inspect | Likely repair |
|---|---|---|
| Old rule is returned | Source version and last successful ingestion | Correct the source or update pipeline |
| Identifier or table value is missing | Parsed text, chunk boundaries and metadata | Repair parsing or retrieval strategy |
| A forbidden document appears | Caller identity and server-side filter | Repair authorization before more testing |
| Correct passage, unsupported answer | Prompt, generation settings and cited span | Constrain or change answer generation |
| No approved source exists | Source inventory and reviewer assessment | Return no answer or add approved material |
Change the right layer
If the document never entered the index, changing the model does not solve the problem. If the right passage is retrieved but misinterpreted, compare prompts and models with retrieval fixed. Preserve the failing sample as a regression case.
For updates and deletions, measure the time from source change to searchable state. Test that removed content disappears from retrieval and from application caches according to the agreed process. An ingestion job reporting success is insufficient evidence by itself.
Close the repair with a counterexample
Re-test the original failure and a nearby request that previously worked. Include a user without permission and a request for information the corpus does not contain. Deliver the cause, change, before/after evidence and remaining limitation. Treat model output as a result to verify, not a diagnostic explanation of what happened inside retrieval.
Official references
Documentation checked: 17 October 2026. The instructions are reference guidance; no cloud API was called for this article.