Use Case: Knowledge Search for Analysts

The scenario

A business analyst at a bank, law firm, or large enterprise needs to answer a question like:

• "What are our contractual obligations to client X regarding SLA response times?"
• "Summarize the key risk factors mentioned in our last 5 annual reports."
• "Which internal policies apply to cross-border data transfers to Brazil?"

Manually searching through SharePoint, email threads, and PDF archives takes hours. With a RAG-powered assistant over the right corpus, it takes seconds — with source citations.

Why this use case is different from consumer Q&A

Consumer chat products use public knowledge. Enterprise knowledge search is harder because:

1. Documents are private and heterogeneous. PDFs, Word docs, Excel sheets, PowerPoints, emails, ticketing systems — all in the same corpus, with different formatting, language, and quality.

2. Accuracy requirements are much higher. A wrong answer in a legal or compliance context isn't just embarrassing — it can be costly or illegal. Hallucination tolerance is near zero.

3. Access control is mandatory. An analyst in the trading desk shouldn't see HR documents. Document-level permissions must be enforced, not just document-level indexing.

4. Audit trail is required. In regulated industries, every answer must be traceable: which document, which version, retrieved on which date, by which user.

Architecture

Same RAG foundation as consumer Q&A, with additional enterprise layers:

[Document sources]  →  ingestion pipeline  →  vector index (with ACL metadata)
                                                       ↓
User query  →  [Auth check]  →  [Filtered retrieval]  →  [LLM generation]
                                    (only docs user can see)         ↓
                                                         Answer + citations + source metadata

Key difference: retrieval must be ACL-aware. When storing chunks, metadata includes the document's access permissions. At query time, the search filters to only chunks the requesting user is authorized to see.

Ingestion challenges

Getting enterprise documents into a searchable index is often the hardest part:

Format diversity. PDFs with scanned content need OCR. Excel tables need special handling (row-by-row vs. summary). PowerPoint decks need slide-by-slide chunking. Email threads need deduplication.

Metadata extraction. For good citations and filtering, you need document metadata: title, author, date, department, version, classification. Extracting this automatically is often imperfect.

Language heterogeneity. Multinational companies have documents in multiple languages. The embedding model and the LLM must handle all of them well.

Change management. Documents are revised regularly. Version control of the index is necessary — knowing that a chunk comes from v2.1 of a contract, not v2.0.

Making answers trustworthy

In an enterprise context, the answer alone is not enough. The analyst needs to be able to defend the answer to a manager, a regulator, or a client.

Verbatim citations. Don't just say "according to document X" — quote the exact sentences from the source document and show the page/section reference.

Confidence signals. If retrieval similarity is low (the question is about something not well-covered in the corpus), say so explicitly.

Date-awareness. Include the document version and date in citations. "According to the Privacy Policy updated 2025-11-01..." prevents confusion when policies change.

Human-in-the-loop escalation. For high-stakes questions, the system should suggest: "This answer involves regulatory risk — you may want to verify with Legal."

Typical corpus sizes and latency expectations

Corpus size	Number of chunks	Retrieval latency	Comment
Small (< 500 docs)	< 100K chunks	< 200ms	pgvector is enough
Medium (500–5K docs)	100K–1M chunks	200–500ms	Dedicated vector DB needed
Large (> 5K docs)	> 1M chunks	500ms–2s	Sharding, approximate search

For most enterprise knowledge search deployments, medium scale is the norm.

Failure modes specific to enterprise

Conflicting documents. Two policy documents say different things (the 2022 version and the 2024 version). The LLM synthesizes a blended answer that's wrong. Fix: surface both sources, let the user see the conflict explicitly.

Table and chart blindness. Key data is in a table or graph that the embedding model doesn't represent well as text. Fix: table-aware chunking, description of charts added by preprocessing.

Jargon and abbreviation mismatch. The user asks about "the MRO protocol" but the document calls it "Material Requirements Ordering Process." Fix: synonym expansion, ontology-based query enrichment.

Overconfident answers on edge cases. The model answers confidently on rare topics with sparse coverage. Fix: retrieval confidence threshold + explicit fallback message.

The governance checklist

Before deploying a knowledge search tool in an enterprise:

• [ ] Who owns the document corpus and approves indexing?
• [ ] How is document-level access control enforced in the index?
• [ ] What is the data retention policy for query logs?
• [ ] Can the system see PII, confidential, or classified documents? Should it?
• [ ] How are document updates reflected in the index (frequency, lag)?
• [ ] What is the escalation path when the AI answer is wrong?
• [ ] Is there a feedback loop so analysts can flag bad answers?

Typical stack

Layer	Examples
Ingestion	Unstructured.io, Azure Document Intelligence, Apache Tika
Embeddings	Cohere Embed v3, OpenAI text-embedding-3, private models
Vector DB	Weaviate (ACL-aware), Qdrant, Elasticsearch with kNN
LLM	GPT-4o, Claude 3.5/4, Azure OpenAI (data residency)
Orchestration	LangChain, LlamaIndex, custom
Access control	Integration with enterprise IAM (Azure AD, Okta)