Retrieval & Document Workflows

Why Claude Is Not a Vertex RAG Engine Generator — and What to Do Instead

On AWS, Claude plugs into the cloud's managed RAG service. On Google Cloud, the equivalent service doesn't list Claude at all. Here is the documented state of play — and the pattern that works instead.

Claude 3P 101 · Updated July 2026 · Unofficial guide

Teams arriving from AWS often assume the pattern transfers: Bedrock Knowledge Bases explicitly supports Claude as a generation model, so surely Vertex AI RAG Engine does too? It doesn't — at least not anywhere in Google's documentation. Understanding exactly what is and isn't documented saves you from designing an architecture that has no supported implementation.

What the supported-models list actually says

Vertex AI RAG Engine (now documented under Google's "Gemini Enterprise Agent Platform") is "a data framework for developing context-augmented large language model (LLM) applications" — Google's managed retrieval-augmented generation service, the counterpart to Bedrock Knowledge Bases. Its supported generative models page lists: Gemini models (3.5 Flash, 3.1 Pro and Flash-Lite, 3 Flash, and the 2.5 line — fine-tuned Gemini unsupported), self-deployed Model Garden endpoints for open models, and two partner-model families, Mistral and Llama 3.1/3.2.

Claude is not on the list. No mention of Anthropic anywhere on the page — and, importantly, no explicit exclusion either. The accurate phrasing for your architecture review is "Claude is not a documented RAG Engine generation model," not "Google blocks Claude." The gap extends to Google's grounding stack generally: the grounding overview documents many methods — Grounding with Google Search, Google Maps, Agent Search (Vertex AI Search), your own search API, RAG, Elasticsearch, and web grounding options — but its examples are Gemini models, and no Claude-as-grounded-generator path is documented, including inside Vertex AI Search grounded generation.

Why the server-side grounding tools don't attach

The structural reason is a one-liner in Google's Claude documentation: Claude models on the platform "support Client tools, but Server tools are not supported." Server tools are functions the platform executes for the model — Google Search grounding, Vertex AI Search retrieval — and those attach to Gemini, not to partner models like Claude. Client tools (ones your code executes) work fine with Claude on Vertex. The one Anthropic-side exception: Anthropic's own basic web search tool is available for Claude on Vertex — the basic version, without dynamic filtering. The broader implications are covered in why Claude on Vertex has no server tools.

The documented pattern: bring your own retrieval

What is documented — and GA for Claude on Vertex — is Anthropic's citations feature and search_result content blocks. So the working architecture inverts the managed-RAG shape: instead of the platform retrieving for the model, your code retrieves, and Claude grounds its answer in what you pass.

  1. Retrieve with any backend you like — Vertex AI Search used directly as a search API, Vertex embeddings with a vector database, Elasticsearch, or an existing internal search service.
  2. Package the top results as search_result blocks with citations: {"enabled": true}.
  3. Generate with Claude via the AnthropicVertex client; citations come back as search_result_location spans tied to your sources.
from anthropic import AnthropicVertex

client = AnthropicVertex(project_id="my-project", region="global")
results = [{"type": "search_result", "source": r.uri, "title": r.title,
            "content": [{"type": "text", "text": r.text}],
            "citations": {"enabled": True}} for r in my_retriever(query)]
msg = client.messages.create(
    model="claude-sonnet-5", max_tokens=1024,
    messages=[{"role": "user",
               "content": results + [{"type": "text", "text": query}]}])

Mechanics of the blocks — indexing, granularity, the all-or-nothing citations rule — are in the search_result deep dive; the end-to-end design is the citation-grounding pattern.

What you give up, and what you keep

You give up the managed conveniences: RAG Engine's ingestion and indexing pipeline, and single-call retrieve-and-generate. You own retrieval quality, chunking, and index freshness yourself. You keep — arguably gain — portability: the BYO pattern is identical on all four platforms, whereas Bedrock KB or RAG Engine designs are cloud-specific. Two Vertex-specific notes for the cost model: batch inference on Vertex (BigQuery/GCS-based, 50% pricing) can pair with pre-fetched retrieval for offline workloads, and requests ≥200K tokens are billed at Google's long-context rates — a reason to keep retrieval selective rather than stuffing context (see long context vs RAG economics).

Where to go next

Set up the client with the Vertex SDK guide, check web search grounding on Vertex for the live-data exception, and compare the fully managed alternative in Bedrock RetrieveAndGenerate.

Sources

← All articles