Perplexity и тест по языковым показателям

Ответы и пояснения появляются по мере прохождения теста.

1. Under the same autoregressive text-scoring setup, what does lower perplexity indicate?
   • Lower average negative log-likelihood for the reference tokens
   • A guarantee of more factually correct generated answers
   • A guarantee of a larger parameter count
   • A requirement that every top-choice prediction be correct
2. Using natural-log negative log-likelihood, how is autoregressive perplexity computed from its average?
   • By multiplying the parameter count by the context length
   • By subtracting the average from the number of attention heads
   • By taking the average learning rate
   • By exponentiating the average negative log-likelihood
3. Модель присваивает равномерную вероятность словарю из 10 000 слов без обучения. В чем его недоумение?
   • 1
   • 10 000
   • 0
   • 100
4. Why do tokenizer and corpus settings matter when directly comparing per-token perplexity values?
   • Perplexity ignores every scored reference token
   • All corpora necessarily produce the same likelihood
   • They change the units or reference data over which probabilities are averaged
   • Only model parameter count determines the score
5. Which translation-reference metric uses modified n-gram precision and a brevity penalty?
   • Perplexity
   • Среднеквадратическая ошибка пикселя
   • Количество параметров модели
   • БЛЮ
6. Какое ключевое ограничение заставляет исследователей преодолевать затруднения при оценке реального качества?
   • Perplexity измеряет предсказание следующего слова, а не то, является ли результат полезным, правдивым или полезным.
   • Perplexity слишком дорого вычислить
   • Perplexity невозможно вычислить на текстовых данных
   • Perplexity работает только во время вывода, а не обучения
7. When scoring a long text with a fixed-context model, what can a sliding-window procedure provide that disjoint chunks lose near boundaries?
   • A new vocabulary learned automatically at every boundary
   • A guarantee that generated answers are factually correct
   • More preceding context for the scored target tokens
   • A requirement to count each overlapping target repeatedly
8. Why is conventional left-to-right perplexity not directly the same score for a BERT-style masked language model?
   • BERT has no learned parameters
   • The masked model does not directly define the same autoregressive sequence factorization
   • Every masked-token probability is necessarily zero
   • Perplexity is defined only for image pixels