Catching sentences with no source — a sentence-level citation verification pipeline

A plausible wrong answer is the dangerous one

In engineering practice, the biggest risk with AI is not what it does not know. It is what it gets wrong convincingly. Fabricating a clause number, citing a precedent that does not exist, stating a calculation result with confidence and getting it wrong — in an industry that works with drawings and specifications, this habit of general-purpose LLM chatbots is grounds for rejection.

doAZ solves this with structure, not prompts.

Sentence-level citation verification

The doAZ RAG pipeline decomposes an LLM answer into individual sentences, then compares each sentence semantically against the retrieved source chunks. A sentence with a missing or weak citation does not pass through silently; it is flagged in a source verification report — a white-box structure in which the user sees "this sentence has no source" at a glance.

Three principles are central.

1. Every answer must carry a document and page citation — a structure in which an answer without a source cannot exist
2. Abstain when the evidence is weak — when retrieval scores are low, the AI declares that it is withholding judgment and hands the case to a person
3. Numbers belong to the deterministic engine — the LLM does not take part in calculation; it suggests where to look and explains why

Measured results

This structure was proven in production on Lotte E&C specification QA, with a source attribution rate of 97.8%. We do not publish a figure without its denominator — every accuracy number is measured against a frozen gold set.

AI that can be verified is AI that gets adopted.