Letting AI Say I Don't Know

An abstaining model withholds a definitive answer rather than guessing. This can be useful when the prompt identifies what evidence is allowed and what to do when that evidence is missing. For example, a document-grounded assistant can answer from retrieved passages and state “not found in these sources” when the passages do not support a response. The instruction changes the behavior requested in context; it does not create a dependable internal gauge of knowledge. The model can still fail to notice that a question is unanswerable, or abstain even when the answer is available. Measure both sides of the tradeoff. A useful evaluation set includes answerable and unanswerable cases, and records correct answers, unsupported answers, correct abstentions, and unnecessary abstentions. An abstention benchmark study introduced Abstain-QA across different question types and domains, while a later benchmark tested a broader set of unknown, underspecified, false-premise, subjective, and outdated questions. These research efforts show that abstention is an evaluation problem as well as a prompting choice; good behavior on one collection does not guarantee performance on another. Improve grounding by supplying authoritative source material and asking the system to identify whether the needed support is present. A retrieval check or deterministic rule can require a matching passage before the application emits an answer; the details depend on the product design and source quality. Cite the passage or field used, and route consequential gaps to a qualified person. Do not treat a refusal phrase as proof of safety, or a low-confidence percentage as proof that an answer is unreliable. Test the full workflow, including relevant-but-insufficient sources and ambiguous questions, then monitor both fabricated responses and over-refusal.