Quiz dotyczący metod wykrywania halucynacji

Odpowiedzi i wyjaśnienia pojawiają się w miarę rozwiązywania quizu.

1. Jakie obserwacje leżą u podstaw metod zapewniających samospójność, takich jak SelfCheckGPT?
   • When a model knows a fact, repeated samples tend to agree, while fabricated details tend to vary
   • Models always give identical answers to repeated questions
   • Longer answers are always more accurate
   • Setting temperature to zero removes all hallucinations
2. Jakie jest kluczowe ograniczenie sprawdzania spójności wewnętrznej?
   • It requires a large labeled dataset before use
   • It works only on image outputs
   • It misses errors the model repeats consistently, and it costs extra model calls
   • It cannot run without retrieved source documents
3. Co tak naprawdę mierzą kontrole konsekwencji oparte na źródle?
   • Ostateczna data szkolenia modela
   • Faithfulness to the provided sources, which is not the same as truth if the sources are wrong
   • How quickly the model responds
   • Grammar and spelling quality
4. Czym entropia semantyczna różni się od zwykłej niepewności na poziomie tokena?
   • It counts the number of words in each answer
   • It relies only on human raters
   • It avoids sampling multiple answers
   • It groups sampled answers by meaning before measuring uncertainty
5. Dlaczego przewodnik zaleca wymaganie od weryfikatora podania rozpiętości podpór?
   • To sprawia, że ​​wyjście jest krótsze
   • It makes verification auditable, and a simple string match confirms the quote appears in the source
   • It makes the verifier more creative
   • It reduces verification cost to zero
6. Which checks should run first in a detection pipeline?
   • Cheap deterministic checks, such as confirming that cited URLs, IDs or case numbers exist
   • Full human review of every answer
   • Fine-tuning the model on the flagged answers
   • Semantic entropy on every token
7. What caveat applies to using an LLM as a judge?
   • Judges cannot read source documents
   • Judges work only in English
   • Judges are always correct
   • Judges have their own biases and error rates, so they should be checked against human labels
8. Why is simply asking a model 'are you sure?' an unreliable detection method?
   • Models cannot read follow-up questions
   • It is prohibited by API terms
   • Models may abandon correct answers or defend wrong ones
   • Podwaja rozmiar okna kontekstowego