Buried in the example grader code of Anthropic's evaluation documentation is a one-line comment that deserves top billing: it is "generally best practice to use a different model to evaluate than the model used to generate the evaluated output." The advice sounds like bureaucratic caution. It isn't. It addresses a real structural weakness in LLM-as-judge evaluation.
The problem with self-grading
An LLM judge is a model applying a rubric to text. If the judge is the very model (and often the very prompt style) that produced the text, the evaluation loses independence in two ways. First, shared blind spots: whatever gap in knowledge or reasoning caused the generator to produce a subtly wrong answer is likely present in the judge too, so the error reads as correct. Second, stylistic familiarity: a model grading output shaped by its own preferences is grading, in part, how much the answer resembles what it would have written — which is not the same as how good the answer is. A different model breaks that correlation. It brings different training, different failure modes, and no stake in the answer under review. Errors that one model reliably makes, another model can often reliably catch.
The cost-versus-independence trade-off
Independence isn't free. Every judged test case is a second inference call, so the judge's price and speed multiply across your eval set. The pragmatic pattern most teams land on: generate with the model you'll actually ship, and judge with a different, usually cheaper model — testing the judge's reliability against a hand-graded sample first, as the rubric-writing guide describes. At July 2026 list prices, Claude Haiku 4.5 ($1/$5 per million input/output tokens) can grade outputs from Claude Opus 4.8 ($5/$25) or Claude Fable 5 ($10/$50) for a small fraction of generation cost. Judging with a bigger different model buys extra rigor on nuanced rubrics, at higher price. Either direction preserves the property that matters: the grader is not the generated model grading itself. When even a cheap judge is too expensive at your volume, the answer usually isn't self-grading — it's moving more of the rubric into code-based graders, which cost nothing per case.
How the principle plays out per platform
The four Claude platforms give you very different amounts of control over judge selection, so applying the best practice looks different on each.
| Platform | Judge situation (July 2026 docs) |
|---|---|
| Your own harness (any platform) | Full control — pick any generator/judge pair you can call |
| Amazon Bedrock evaluations | Claude judges supported, but only up to Opus 4.5 / Sonnet 4.5 / Haiku 4.5; newest generations not documented as judges |
| Vertex AI Gen AI evaluation | Claude documented as an evaluated partner model; Claude as judge is not documented |
| Microsoft Foundry evaluators | Judges are Azure OpenAI / OpenAI models only; Claude not a judge option |
Two details are worth spelling out. On Bedrock, the documented evaluator lists for LLM-as-a-judge jobs top out at Claude Opus 4.5, Sonnet 4.5, and Haiku 4.5 (legacy date-versioned IDs); Claude Fable 5, Opus 4.8, and Sonnet 5 are not documented judge options. That constraint has a silver lining: if you're evaluating a current-generation Claude model, the platform effectively guarantees your judge differs from your generator. Bedrock also accepts bring-your-own inference responses, so outputs generated anywhere — including the first-party Claude API — can be judged inside a Bedrock evaluation job. On Foundry, judge independence is enforced in the other direction: since LLM-judge evaluators only accept Azure OpenAI or OpenAI judges, any Claude output you evaluate there is automatically cross-family graded — though Microsoft's docs don't state whether Claude deployments can be evaluation targets at all, so check current documentation before building on it. On Vertex AI, Google documents partner-model evaluation (including Anthropic models) through Model Garden, but whether Claude can serve as the judge is undocumented — assume the default judge arrangement and verify in the evaluation overview.
Where to go next
For the managed-job mechanics, see Bedrock LLM-as-a-judge setup, Vertex partner-model evaluation, and Foundry's judge-model limitation. For building the rubric itself, start with writing reliable judge rubrics.