Retrieval-augmented generation (RAG) exists partly because context windows used to be small. Now that Claude Opus 4.8, Sonnet 5, Sonnet 4.6, and Claude Fable 5 all offer 1M-token windows — GA by default, no beta header — a competing architecture is viable: skip the vector database and load the whole corpus into the prompt. Call it context stuffing. Sometimes it genuinely wins. But the economics differ by platform in a way that catches teams off guard, and there's a quality ceiling no pricing table shows.
The case for stuffing
Loading full documents wins when the working set is bounded and interrelated: a contract set where clauses cross-reference each other, a codebase slice, a due-diligence bundle. Retrieval fragments such material; a chunk-based pipeline may never surface the clause that modifies the one you asked about. Long context keeps every cross-reference in view, and you skip an entire subsystem — no embeddings, no index, no sync jobs, no retrieval-quality evaluations. Combined with prompt caching (cache the document prefix; pay roughly a tenth of base input price on hits), repeated questions over a static bundle can be surprisingly economical. And Anthropic's prompting guidance for long inputs is well documented: longform data at the top, query at the end — worth up to ~30% response-quality improvement — with documents wrapped in <document>-style tags.
The platform pricing split
Here is the trap. Anthropic's pricing page states that 1M-window models "include the full 1M token context window at standard pricing. (A 900k-token request is billed at the same per-token rate as a 9k-token request.)" Cloud marketplace list prices match 1P list prices, so on the Claude API, Claude Platform on AWS, and Amazon Bedrock, a stuffed 900K-token prompt costs 100× a 9K prompt — linear, no penalty band.
Vertex is different. Google's pricing page states: "If a query input context is longer than or equal to 200K tokens, all tokens (input and output) are charged at long context rates." Cross the 200K threshold on Vertex and every token in the request — not just the excess, and including output — moves to the long-context rate. Check Google's page for the current rates; the structural point is the cliff itself.
| Platform | Billing for a request ≥200K tokens |
|---|---|
| Claude API (1P) | Standard per-token rates across the full 1M window |
| Claude Platform on AWS / Amazon Bedrock | Marketplace list prices match 1P — standard rates apply |
| Google Vertex AI | All input and output tokens billed at long-context rates |
Remember what counts toward the window: system prompt, all messages (tool results, images, documents), tool definitions, and the generated output including thinking. Cached prefixes still occupy the window — caching changes cost, not capacity.
Context rot: the non-economic limit
Even where stuffing is billed at standard rates, Anthropic's own docs warn about context rot: "As token count grows, accuracy and recall degrade," making curating what's in context "just as important as how much space is available." A million tokens of capacity is not a million tokens of reliable recall. Past some corpus size, a good retrieval pipeline that puts 20K relevant tokens in front of Claude beats 800K indiscriminate ones on answer quality — before any cost argument. This is why the mature end state is usually hybrid: retrieval for selection, long context for generous headroom around what's selected.
A decision sketch
Stuff the context when the corpus is under a few hundred thousand tokens, interrelated, and queried repeatedly (cache it) — and, on Vertex specifically, when you've priced the ≥200K band or can stay under it. Build RAG when the corpus is large or growing, freshness matters, per-request cost must stay flat, or you're on Vertex near the threshold. Route by need when workloads mix. For the retrieval designs, see RAG basics and citation grounding; for stuffing-side techniques, long-context strategies and context placement.
Where to go next
Context cost strategy covers budgeting either architecture; context windows explained covers the mechanics. Vertex billing details live in Vertex billing mechanics.