AnswersAgents & RAG

How Do You Audit an AI Answer?

When compliance asks what an AI answer was based on, what makes the answer auditable, and why must citations come from retrieval rather than from the model?

3 min readJuly 2026

The short answer

An AI answer is auditable when every claim carries a reference to a specific document and page that a human can open and check. That requires citations produced by the retrieval layer, which records exactly which passages the answer was built from. Citations written by the model are text like any other text: fluent, plausible, and easy to fabricate.

The measurable form of this property is citation precision: of the references an answer gives, how many actually support the claim they are attached to.

Why can't you trust citations the model writes?

A language model generates plausible text, and a reference is just more text. Asked to cite, a model produces something citation-shaped whether or not a source exists, and the fabrications look identical to the real thing. The failure is documented at scale: Dahl et al. (2024) document pervasive fabricated case citations in LLM legal outputs, and a fabricated reference in a filing or a client memo is a compliance incident, not a formatting bug.

Your compliance officer asks where a number in the answer came from. "The model said so" is not something you can file.

Two ways an answer gets its citations.

Model-written citations

Model writes the answer

Model also writes the references

Nothing links to a real page

Plausible and unverifiable

Retrieval-produced citations

Engine retrieves passages

document and page recorded

Model composes from those passages

Every fact points at its reference

A human can open and check it

What does a retrieval-produced citation look like?

The retrieval layer knows something the model never can: which passages were actually fetched and placed in context. An auditable pipeline keeps that record and attaches it to the output, so each fact in the answer points at a retrieved passage with its document ID, file name, and page numbers. The citation is a pointer into evidence the system verifiably used, not a claim the model composed.

In TopK's ask API, that structure is the response format. Answers come back as facts, each carrying reference IDs that map to the matched passages with document name and page-level locations, plus a 0–100 confidence score:

for message in client.ask("What was the total net income of Bank of America in 2024?", ["my-docs"]):
print(message)

A reviewer takes the reference, opens the named file at the named page, and checks the claim. That round trip is what "auditable" means in practice.

How do you measure auditability?

Citation precision: the fraction of an answer's references that genuinely support the claims they back. On BrowseComp-Plus (June 2026), an agent running on TopK's retrieval stack reached 88.54% citation precision alongside 80.48% task accuracy, averaging about 14 search calls per task. High citation precision indicates the retriever surfaces the right documents early enough for the model to ground every claim instead of improvising.

What should a finance team demand?

Ask any vendor three questions. Does every claim carry a reference a human can open, at page granularity? Are the references produced by retrieval rather than generated by the model? Is citation precision measured and published, with a date? A vendor that cannot answer all three is asking to be trusted, and audits do not run on trust. Apply the same scrutiny to the recall numbers on the box.

Retrieval quality is what makes the citations good in the first place: an answer grounded in the wrong table row cites its wrong source faithfully. Auditability and retrieval accuracy are the same investment, which is why TopK publishes both numbers together.