自然语言推理和蕴涵

Natural language inference asks a question people answer constantly without noticing: if this sentence is true, what follows? A system receives two pieces of text. The premise is taken as true, for example "A man is playing a guitar on stage." The hypothesis is a claim to evaluate, such as "A man is performing music." The system outputs one of three labels. Entailment means a typical reader would conclude the hypothesis is true given the premise. Contradiction means the hypothesis must be false, as with "The man is asleep in bed." Neutral means the premise does not settle it, as with "The man is a famous musician." The judgment is about what a reasonable person would infer, not strict formal logic.

Modern NLI systems are text classifiers built on pretrained Transformer language models. The premise and hypothesis are joined into one input with a separator token between them, so every word of the hypothesis can attend to every word of the premise. A small classification layer on top of the model's output produces three scores, which a softmax function turns into probabilities for entailment, neutral and contradiction. The model is fine-tuned, meaning further trained, on hundreds of thousands of labeled pairs until its predictions match the human labels.

The task is older than neural networks. Early Recognizing Textual Entailment (RTE) challenges used small test sets, which limited learning-based approaches. That changed in August 2015, when Samuel Bowman and colleagues released the Stanford Natural Language Inference corpus (SNLI) with 570,000 human-labeled sentence pairs, about two orders of magnitude larger than earlier resources of its kind. SNLI premises are image captions, and crowd workers wrote a hypothesis for each label. In 2017 Adina Williams, Nikita Nangia and Bowman released MultiNLI with 433,000 examples from ten genres of written and spoken English, published at NAACL 2018. MultiNLI became a standard training set for general-purpose NLI models.

High scores soon turned out to hide shortcuts. In 2018 Suchin Gururangan and colleagues showed that a classifier that never sees the premise, only the hypothesis, labels about 67% of SNLI and 53% of MultiNLI correctly, against a chance rate of about 33%. Crowd workers left annotation artifacts: negation words, for example, correlate strongly with contradiction. In 2019 Tom McCoy, Ellie Pavlick and Tal Linzen built HANS, a test set where three tempting heuristics fail: assuming entailment when the hypothesis reuses the premise's words, when it is a contiguous piece of the premise, or when it is a complete subtree of the premise's parse. Models trained on MultiNLI, including BERT, did very poorly on it. Adversarial NLI (ANLI), released by Yixin Nie and colleagues in 2019, answered by having people write examples specifically to fool strong models in a human-and-model-in-the-loop process, producing a harder test set.

NLI is now used more as a tool than a leaderboard. Wenpeng Yin, Jamaal Hay and Dan Roth showed in 2019 that a trained NLI model can act as a zero-shot classifier: treat the text as the premise, write each candidate label as a hypothesis, and pick the label with the highest entailment score. The bart-large-mnli model on Hugging Face packages exactly this method. A second use is checking generated text. A summary sentence that is entailed by the source is supported; one that is contradicted or neutral may be a hallucination, meaning a statement the source does not back.

Compared with alternatives, NLI-based checks are cheap and transparent. A fine-tuned NLI model is far smaller than a chatbot-scale language model, runs quickly on modest hardware and returns probabilities that can be thresholded and audited. Question-answering-based metrics, which generate questions from a summary and check whether the source answers them the same way, are slower but catch different errors; the TRUE study found the two families complementary. Asking a large language model to judge consistency is flexible but costlier, and its verdicts can vary with the prompt.

The limitations are real. NLI datasets are built at sentence level, while documents spread facts across many sentences; Laban and colleagues identified this granularity mismatch as the reason early NLI-based summary checkers performed poorly, and their SummaC method, which aggregates sentence-pair scores, reached 74.4% balanced accuracy on its six-dataset benchmark. Labels also depend on background assumptions about what a reasonable reader infers, so annotators sometimes disagree. Shortcut learning can reappear in any new domain, and a model trained on captions and fiction may misjudge legal or medical text. A neutral label means only that the premise does not settle the claim; it does not mean the claim is false.

In practice, building with NLI requires a pretrained model already fine-tuned on MultiNLI or a combination of SNLI, MultiNLI and ANLI, which are freely available, plus a validation set from the target domain to set decision thresholds. Fine-tuning a new model takes a single modern GPU and the public datasets. The main ongoing cost is evaluation: checking examples where the model's labels and human judgment differ, and watching for the overlap and negation shortcuts the research literature has documented.