RAG Citation Debugger

Check a RAG answer's citations against the retrieved chunks: uncited claims, wrong sources, numbers.

Try an example: Answer with a wrong number, a misattributed citation and an uncited claim

How it works

Paste a generated answer with citation markers ([1], [doc2], [^3], ^1, 【4】) and the retrieved chunks. Each sentence is mapped to the chunks it cites and checked for lexical support: uncited claims, citations to chunks that do not support the sentence, numbers that are not in the cited chunk, better-matching chunks, markers pointing to non-existent chunks, and retrieved chunks that were never used.

Markers recognised: [1], [1, 3], [doc2], [^1], ^1, 【4】 and (source: x). Markers placed after the full stop are attached to the preceding sentence. Labels resolve to chunk ids exactly, then by trailing number (doc2 ↔ 2), then a bare number n to the n-th chunk.
Support = share of the sentence's content words (stop words removed, light stemming) found in the cited chunk; the best cited chunk counts. ≥ threshold → supported, ≥ half the threshold → weak support, below → unsupported citation.
Numbers in the sentence must appear in at least one cited chunk, otherwise number-mismatch. Sentences with ≥ 3 content words or a number but no marker are uncited claims.
If another chunk covers the sentence clearly better (≥ threshold and +0.2), it is suggested as the correct citation. Chunks never cited are unused, or relevant-but-uncited if they would support a sentence.
This is lexical overlap, not entailment: paraphrases can look unsupported and negations can look supported. Treat results as pointers for review.

For scripts and AI agents

JSON API: POST https://ragcitationdebugger.usefulweb.site/api/v1/calculate (same calculation as this page)
Interface description: OpenAPI 3.1 · AI guide: llms.txt
MCP server (Streamable HTTP): https://ragcitationdebugger.usefulweb.site/mcp · tool name rag_citation_debugger

Example request

curl -s -X POST https://ragcitationdebugger.usefulweb.site/api/v1/calculate \
  -H 'content-type: application/json' \
  -d '{…see OpenAPI examples…}'

Related tools

AI Answer Verifier – Split an AI answer into claims and see which ones your sources actually support.
Model Behavior Regression Explorer – Compare eval results of two model versions: accuracy, regressions, McNemar p-value, flaky items.
Structured Output Validator – Validate AI JSON output against a JSON Schema: parse errors with line/column, plain-language fixes.