Checklist

RAG Deployment & Citation Checklist

An interactive, practical checklist to design, deploy, verify, and monitor Retrieval-Augmented Generation systems with a focus on chunking, embeddings, index maintenance, citation and provenance, freshness, latency/recall tradeoffs, and hallucination mitigation. Save responses for audits, iterative improvement, and team handoffs.

Interactive Tool

RAG Deployment & Citation Checklist

Use this checklist to plan and verify key RAG design decisions, deployment readiness, provenance/citation behavior, and operational monitoring. Complete fields, save the results, and return later to iterate. The checklist emphasizes explainability, updateability, and hallucination resistance.

Name and role of the person accountable for this RAG deployment.
Date of this checklist completion (YYYY-MM-DD).
Short name used internally for this RAG system.
Select the kinds of content you will index.
Has legal/security/business approved the listed sources for indexing?
How will you split content into retrievable segments?
Typical token length per chunk. Tune based on embedding model and retrieval performance.
If using overlapping chunks, how many tokens overlap between adjacent chunks?
Model name and provider used to create embeddings.
Dimension size of vectors (useful for index and storage planning).
Choose the index or vector DB technology you will use.
Describe how new or changed documents are indexed, deleted, and re-embedded. Include triggers, frequency, and rollback steps.
How often the index is refreshed/updated to maintain freshness.
How will retrieved content be cited in answers? Select all that apply.
Can each retrieved item be traced back to a canonical document and location (e.g., URL + byte offsets)?
Provide one or two example citations your system will present to users (copy-paste ready).
Maximum tokens sent to the LLM after retrieval and prompt assembly.
Maximum acceptable end-to-end retrieval latency for user interactions.
Slider: 1 = prioritize latency, 5 = prioritize recall/coverage
1.0 10.0
Describe how retrieved candidates are scored or re-ranked (ML reranker, cross-encoder, hybrid signals).
Select mitigation techniques you plan to implement.
Provide a sample question used in your QA test suite to check for hallucinations.
Define measurable pass/fail criteria for correctness, citation presence, and response style.
Does ingestion pipeline detect and flag PII or sensitive content?
Describe how flagged sensitive content is handled (redaction, access controls, not indexed, etc.).
Has the content been reviewed for regulatory or contractual restrictions?
How are retrieval results filtered by role, team, or clearance? Describe provenance policy enforcement.
How provenance will be shown to users.
Can users flag incorrect answers or rate helpfulness to improve retrieval?
Select key operational metrics to monitor in production.
Define actionable thresholds and alerting rules for the selected metrics.
Describe steps to take when a deployed change causes unacceptable hallucination, breaches, or outages.
Does the team consider the system ready for production based on the above items?
Record unresolved items, tradeoffs, or follow-ups.
1 = Not ready, 5 = Production-ready
1.0 10.0
Typed name serves as a lightweight sign-off.
You can explore this tool now. Sign in or create an account to save your responses and return to them later.
Sign In to Save Create an Account
Make this tool part of your work

Save a personal copy, bring it to your team, or tailor the questions and workflow to fit what you are hungry to improve.

Member customization and team collaboration are coming soon.
Discuss Ask the Librarian

Discussion

Comments and conversation will live here.