Your AI cited a company document. Does it support the answer?
Contents
Does the source support it?
Check the claim. Then the citation.
An answer can cite the right document and still promise something it does not say. “See the export guide” looks reassuring until the answer adds attachments, every user and a 24-hour turnaround. The useful check is small: put the exact answer beside the exact passage and ask which words the source actually earns.
A real-looking citation, four unsupported promises
Here is a complete fictional example. A support agent asks an assistant: “Can every user export our project data, including attachments, and how long do larger exports take?” The assistant receives only the following source passage.
Source S1 — Project export guide, revision 3, effective 1 October 2026:
“Project owners can download a CSV containing up to 10,000 completed tasks from one project. Attachments are not included in the CSV. Requests for larger exports require a support review. This guide does not promise a turnaround time for that review.”
Unsupported answer: “Yes. Every workspace user can instantly export all project data, including attachments. Exports over 10,000 tasks are approved within 24 hours. Source: S1, Project export guide.”
The citation names the document that was supplied. That establishes where to look; it does not establish the answer. Read the matching claims separately:
| Answer claims | S1 says | Finding |
|---|---|---|
| Every workspace user | Project owners can download the CSV. | Unsupported expansion of who can export. The passage establishes owners, not every user. |
| All project data, including attachments | Up to 10,000 completed tasks from one project; attachments excluded. | Contradicted on attachments and broader than the stated scope. |
| Instantly | No download duration is given. | Unresolved. The passage neither promises nor disproves instant availability. |
| Larger exports approved within 24 hours | Support review required; no review turnaround promised. | Unsupported approval and deadline. A review is not an approval. |
Write the answer the source can support
A corrected answer would be:
“S1 describes a CSV export available to project owners for up to 10,000 completed tasks from one project. It excludes attachments. Larger exports need support review; the guide gives no review turnaround time and does not say that a request will be approved. It does not establish export access for every user or a download duration.”
This answer is useful even though it leaves two things open. The support agent can explain the documented export and ask the product owner about other user roles or timing if the customer needs those details. There is no need to pretend that missing information makes the whole document unusable.
Notice the distinction between “unsupported” and “false”. The attachment promise conflicts with an explicit exclusion. The download-speed promise lacks evidence. You should remove both from this answer, but the second finding does not prove the system is slow.
Check the small words that change the promise
When reviewing your own answer, first copy the sentence someone might act on. Split a sentence that combines eligibility, scope and timing. Then open the source and read enough surrounding text to include exceptions and conditions, rather than stopping at a matching heading.
Look especially for expansions: “owners” becoming “everyone”, “some” becoming “all”, “can request” becoming “will receive”, and a stated limit disappearing. A correct number can sit inside an incorrect promise. In this example, retaining 10,000 did not rescue the claim about approval within 24 hours.
Check the source’s version and applicable date when they matter to your question. A passage may accurately describe another product plan or an earlier release. If the citation opens a document but not the supporting passage, locate the passage yourself before treating the answer as ready.
NIST’s Generative AI Profile, action MS-2.5-003, recommends checking sources and citations in generated outputs during evaluation and monitoring. This claim-by-claim comparison is a practical editorial way to do that; it is not a NIST certification or a guarantee that all errors will be found.
Correct the claim, then investigate the failure
If the original answer was already copied into a customer reply or an internal recommendation, correct that work as well as the saved AI answer. State the documented limit plainly. For S1, “attachments are excluded and larger requests need review” is more helpful than “the AI hallucinated”.
For diagnosis, keep the question, source passage, wrong answer and corrected answer together as a small example. Confirm that the source actually reached the assistant. If it did not, investigate selection or retrieval. If it did, the example lets you examine how the answer stretched its support. Use fictional or appropriately permitted material when sharing the case.
Do not call the issue fixed because one retry sounds better. Check the same case after a targeted change and include a nearby case where the source really does grant the broader permission or promise. That helps catch a repair that merely teaches the assistant to refuse everything.
If several runs disagree, the inconsistent-answer comparison helps separate changed inputs from answer variation. Here, the immediate job is simpler: keep the useful facts and remove the promises the cited passage cannot support.
Prepared by HeyBrain Editorial Team with AI assistance. The linked NIST publication was checked on 12 October 2026. The export guide, question and answers are fictional teaching material; they describe no HeyBrain feature, customer incident or tested model result.
