Build and evaluate a Bedrock knowledge workflow

Prepare sources and the access model

Inventory each source's owner, version, update frequency and allowed users. Resolve conflicting policies before indexing them. Preserve document identifiers and locations so reviewers can trace an answer back to the approved version.

Choose the Knowledge Base type deliberately. AWS currently distinguishes Managed Knowledge Bases from customer-managed pipelines. Connector, ACL and retrieval capabilities differ; do not assume a vector store automatically inherits the source system's permissions. Test the actual connector and authorization design with two users who should see different documents.

Establish a retrieval baseline

Create questions with expected documents, including exact identifiers, recent updates, missing information and unauthorized material. Test retrieval before asking the model to answer. Inspect returned chunks, metadata and ranking: a fluent answer cannot repair an absent or incorrect source.

Change one retrieval setting at a time. Larger chunks preserve context but can dilute a precise match; smaller chunks can separate a table value from its heading. Compare changes on the same sample set and keep document versions fixed during the comparison.

Evaluate generated answers separately

Where supported, RetrieveAndGenerate combines retrieval and generation with citations. Retrieve supports applications that assemble context themselves. Check the API and supported generation model for the chosen Knowledge Base; not every Bedrock model is available as its native generation target.

Verify whether each important statement is supported by its cited passage. Require an appropriate no-answer response when approved sources are insufficient. A citation indicates a source relationship; it does not prove the conclusion is correct.

Before release, repeat permission, stale-document, deletion and missing-source tests. Deliver the source register, ingestion/update procedure, retrieval results and answer evaluation. Name the owner who will repair failed ingestion or conflicting documents.

Official references

Documentation checked: 17 October 2026. The instructions are reference guidance; no cloud API was called for this article.