Bedrock RetrieveAndGenerate API: Using Claude as the Generation Model

Amazon Bedrock Knowledge Bases is AWS's managed retrieval-augmented generation (RAG) service: it ingests your documents, indexes them, and at query time "searches your data to find relevant information to answer the query," with citations back to the original sources. RetrieveAndGenerate is the one-shot API on top of it — retrieval plus answer generation in a single call, where the docs say it "generates responses based on the retrieved results and using the specified foundation model or inference profile."

That last clause is the important one for this site. Unlike Google's Vertex RAG Engine, which does not list Claude among its supported generators (details here), AWS's own API reference passes Claude models in its request examples — both as a direct modelArn (the example uses "modelArn": "anthropic.claude-v2:1", an older model, but the field is the same for current ones) and as a cross-region inference profile ARN of the form arn:aws:bedrock:us-west-2:...:inference-profile/us.anthropic.claude-3-5-sonnet-20240620-v1:0. Which current Claude models are eligible is governed by Bedrock's general "Models at a glance" support table rather than a standalone list, so check it for the model you plan to use.

The request, in essence

The core request names your knowledge base and the generation model:

{
  "input": {"text": "What does our travel policy say about upgrades?"},
  "retrieveAndGenerateConfiguration": {
    "type": "KNOWLEDGE_BASE",
    "knowledgeBaseConfiguration": {
      "knowledgeBaseId": "KB12345",
      "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/..."
    }
  }
}

Around that core, the API documents a rich set of optional configuration: guardrails, custom prompt templates, query transformation, metadata filters, and reranking configuration. There is also externalSourcesConfiguration, which lets you generate over supplied S3 objects or byte content directly, without a knowledge base at all. Two caveats worth knowing before you design around it: cross-region inference profiles can share data across AWS Regions (the docs warn about this explicitly — relevant for data-residency reviews), and RetrieveAndGenerate works with customer-managed knowledge bases, not the newer managed knowledge base type, which uses AgenticRetrieveStream or Retrieve instead.

Reading the citation spans

The response's citations[] array is what makes this API more than a convenience wrapper. Each citation maps a span of the generated answer to the retrieved chunks that support it: generatedResponsePart.textResponsePart.span gives the character range in the generated text, and retrievedReferences carries the supporting content plus source locations — S3, SharePoint, Confluence, Salesforce, Google Drive, OneDrive, Kendra, SQL, web, or custom. Your UI can walk the array and underline exactly which sentence came from which document. Note this is Bedrock's own citation format, not Anthropic's citations content blocks — if you later move to a bring-your-own-retrieval design with search_result blocks, the response shape changes completely.

Session continuity for multi-turn conversations

Knowledge-base Q&A is rarely one-shot — users ask follow-ups. RetrieveAndGenerate handles this with a sessionId: the first call auto-generates one and returns it; you pass it back on subsequent calls and the service maintains the conversation context for you. No manual replay of message history, which is the main ergonomic difference from stitching together raw Messages API calls yourself.

When to choose it over BYO retrieval

Choose RetrieveAndGenerate when you want AWS to own ingestion, chunking, embedding, and retrieval, and you are content with Bedrock's citation format and its Titan/Cohere embedding menu. Choose bring-your-own-retrieval — your index, Claude via the Messages API, search_result blocks for attribution — when you need custom embeddings (e.g., Voyage), portability across clouds, or Anthropic-native citations.

Where to go next

Continue with the Knowledge Bases mechanics: data sources and permission filtering, chunking modes, and embedding model choice.

Sources

• RetrieveAndGenerate API reference — AWS Docs
• Amazon Bedrock Knowledge Bases — AWS Docs
• Supported models and regions for Knowledge Bases — AWS Docs
• Web search tool — Claude Docs