Every organization pays a quiet tax on findability: employees interrupt colleagues, file IT tickets, and email HR for answers that are written down somewhere. An internal knowledge assistant — Claude, grounded in your own documents and running through your existing cloud platform — turns "search, skim, give up, ask someone" into a direct answer with a link to the source. It is also, usefully, one of the lowest-risk first projects: the audience is internal, mistakes are recoverable, and usage tells you exactly what employees need.
The problem is retrieval, not intelligence
An ungrounded model answering questions about your expense policy is guessing, however fluent the guess. The fix is retrieval-augmented generation (RAG): when an employee asks a question, deterministic code searches your content sources for the most relevant passages, and Claude answers using only those passages, citing them. The model supplies reading comprehension and clear writing; your documents supply the facts. If retrieval finds nothing relevant, the right answer is "I couldn't find this — here's where to ask," and the assistant should be explicitly instructed to say so rather than improvise. The mechanics are covered in RAG Basics.
A reference design
Three components, with clear ownership. The content pipeline (deterministic code) indexes chosen sources — wiki spaces, policy folders, handbook pages — and refreshes on a schedule, keeping track of each document's owner and last-reviewed date. The retrieval layer (also deterministic code) takes the user's question, finds candidate passages, and — critically — filters them by the asking user's permissions before anything reaches the model. The answering layer is Claude: given the question and the permitted passages, produce a concise answer with citations, or decline. Sonnet 5 is the usual fit; a long system prompt describing tone, citation format, and refusal behavior is a natural candidate for prompt caching, which every platform supports and which meaningfully cuts the cost of repeated calls.
What humans and code still own
The assistant does not become the authority on anything; the documents remain the authority, and their owners remain responsible for them. That has two practical consequences. First, every answer carries citations so employees can verify — and will trust — what they read. Second, the assistant surfaces content problems rather than papering over them: log the questions that retrieval failed to answer and the answers users flagged as wrong, and route those to content owners as a to-do list. Many teams find this feedback loop is worth as much as the assistant itself, because it shows precisely which documentation is missing or stale.
Rollout advice
Start narrow and high-traffic: IT how-tos plus HR policy is the classic opening scope, because the questions are frequent, the answers are documented, and the audience is everyone. Clean the source content for that scope before launch — an assistant grounded in three contradictory versions of the travel policy will faithfully relay the contradiction. Put the assistant where employees already are (chat tool, intranet search box) rather than asking them to visit a new destination. Then publish what it is and is not: an assistant that answers from documents, not an oracle; and for consequential personal questions — leave entitlements, visa matters, anything disciplinary — it should point to the human owner, not adjudicate.
Pitfalls
The recurring failures are predictable. Permission leaks from indexing everything and restricting nothing. Stale content, because nobody owned refreshing the index or reviewing the sources. Scope creep into every department's documents before retrieval quality is proven on one. Answers without citations, which employees rightly distrust. And measuring success by conversation volume instead of resolved questions — a better yardstick is whether repeat questions to IT and HR actually decline. All four platforms — Bedrock, Vertex AI, Foundry, and Claude Platform on AWS — support everything this design needs, so choose based on where your content and identity systems already live.
Where to go next
Read RAG Basics: Grounding Claude in Your Documents for the retrieval design in detail, and From Shadow AI to Sanctioned AI for why a sanctioned assistant beats the unsanctioned alternatives employees are already using. The quickstart gets you to a first call on your cloud.