Choose a Bedrock model with a task evaluation
Compare quality, latency, permissions and cost on the same workload.
Build the comparison before choosing a model
Prepare a versioned dataset of normal requests, long documents, ambiguous questions and prohibited requests. Include expected outcomes and a scoring rubric. Keep prompts, source documents and output limits consistent across candidates; otherwise a cheaper request may simply have received less context or a shorter answer.
Record the exact model ID or inference profile, Region, API and date. Separate mandatory constraints, such as permitted processing geography or image input, from preferences such as response style. Eliminate candidates that fail a mandatory constraint before comparing quality.
Measure the complete task
| Measure | Record | Decision it informs |
|---|---|---|
| Quality | Correctness, supported citations, appropriate refusal | Whether the task is usable |
| Latency | First token and completed answer, including retrieval | Whether users can finish the workflow |
| Cost | Input, output and cache categories; tool and retrieval costs | Cost per accepted task |
| Reliability | Throttling, timeouts, incomplete results | Capacity and fallback design |
A large model can be unnecessary for a narrow task; a small model can create expensive retries or manual correction. Compare cost per accepted result, not only token price.
Introduce routing only after a baseline works
Start with a measurable default. Add a route to a lighter model for tasks that model can complete successfully, and an escalation route for defined exceptions. A fallback must satisfy the same data boundary and output contract. Do not silently replace a failed request with a different provider if that changes the approved processing location.
The evaluation output is a model decision record: dataset version, results, exclusions, chosen configuration and change triggers. Re-run relevant samples after model, prompt or retrieval changes; retain a rollback configuration.
Official references
Documentation checked: 17 October 2026. The instructions are reference guidance; no cloud API was called for this article.