Amazon Bedrock's RetrieveAndGenerate API is usually described as the query front-end for Knowledge Bases: it searches an indexed knowledge base, then "generates responses based on the retrieved results and using the specified foundation model or inference profile" — and Claude is an explicitly supported generation model, with AWS's own request examples passing Claude model ARNs and inference profiles. Less well known is that the same API accepts an externalSourcesConfiguration: instead of pointing at a knowledge base, you point at the documents themselves — S3 objects or raw byte content supplied in the request — and Bedrock grounds the generation directly in them. No ingestion job, no vector store, no index.
Two grounding modes, one API
| Knowledge base mode | External sources mode | |
|---|---|---|
| Setup before first query | Create KB, connect data source, ingest and index | None — documents ride along with the request |
| Document source | Indexed corpus (S3, SharePoint, Confluence, Google Drive, OneDrive, Web Crawler) | S3 objects or raw bytes named in externalSourcesConfiguration |
| Retrieval step | Vector/semantic search over the index | Skipped — the supplied documents are the context |
| Best at | Large corpora, many queries over the same data | Small document sets, ad hoc or one-shot analysis |
Both modes return the same response structure, which is a large part of the appeal: generated text plus a citations[] array mapping spans of the generated response (generatedResponsePart.textResponsePart.span) to retrievedReferences carrying the supporting content and its source location. Your application renders attribution the same way regardless of which mode produced the answer. The API also supports session continuity — a sessionId is auto-generated on the first call and reused for follow-up questions in the same conversation — along with guardrails and prompt templates.
Where on-demand beats a managed index
Documents that live for one conversation. A user uploads a contract and asks questions about it. Building an index for a document you'll query for ten minutes is overhead with no payoff; external sources mode answers immediately, and there is nothing to clean up afterwards.
Volatile content where the index would always be stale. If the authoritative version of a document changes faster than your ingestion cadence, an index answers from the past. Passing the current S3 object grounds the answer in what is true right now.
Compliance-sensitive material you'd rather not persist in another store. A knowledge base is a second copy of your data — embeddings and chunks in a vector store, with its own lifecycle to govern. On-demand grounding processes the document in the request path without creating a persistent derived corpus. (Retention specifics inherit your cloud provider's posture — confirm with your provider.)
Low query volume over many distinct documents. Indexing pays off when many queries amortize one ingestion. One or two questions each against thousands of different documents inverts the economics; on-demand wins.
The boundary in the other direction is equally clear: external sources mode does no retrieval, so the documents you supply must fit the request. Once the corpus is bigger than what you can reasonably attach — or the same corpus serves many users and queries — you want real search over an index, which is what Knowledge Bases exists for. One API-shape caveat from AWS's reference: RetrieveAndGenerate works with customer-managed knowledge bases but cannot be used with the newer managed knowledge bases, which use AgenticRetrieveStream or Retrieve instead — the external-sources pattern belongs to the RetrieveAndGenerate surface.
Why this matters specifically on Bedrock
On the Anthropic API you might solve the same problem with document blocks plus citations, or the web fetch tool. On Bedrock, Anthropic's Files API and server-side web tools are unavailable, and Knowledge Bases runs on Bedrock's own agent-runtime APIs (the legacy InvokeModel/Converse-style surface, with anthropic.-prefixed and ARN model IDs — not the Anthropic Messages API surface). externalSourcesConfiguration is therefore the Bedrock-native equivalent of "attach these documents and answer with citations": a first-class, AWS-operated path that needs no standing infrastructure.
Where to go next
For the indexed path, start with RetrieveAndGenerate against a knowledge base and Knowledge Base data sources. For the Anthropic-native alternative on other platforms, see citations.