Idanwo ti o sopọ mọ itọsọna · Lile Ipele

Apeere odi ati Idiyewo Idiyele Ariwo

Checks how NCE and negative sampling use noise examples, reduce full-softmax cost, and differ in their probabilistic goals.

Jẹmọ itọsọna awọn ọnaIṣapẹẹrẹ odi Ati Ifoju Idiyele Ariwo
Ibeere 1 ti 8

Iṣoro wo ni iṣapẹẹrẹ odi ati NCE yanju ni akọkọ?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Iṣoro wo ni iṣapẹẹrẹ odi ati NCE yanju ni akọkọ?
    • Awọn idiyele giga ti iširo softmax kikun lori aaye iṣelọpọ ti o tobi pupọ
    • Overfitting lori kekere datasets
    • Apanirun gradients ni loorekoore awọn nẹtiwọki
    • Aini data aami fun ẹkọ abojuto
  2. Bawo ni Ifoju Itansan Ariwo ṣe atunṣe iṣẹ-ṣiṣe ikẹkọ naa?
    • Bi ipadasẹhin lori awọn ibi-afẹde lemọlemọfún
    • Bi ipin alakomeji laarin awọn ayẹwo data otitọ ati awọn ayẹwo ariwo
    • Bi iṣupọ ti awọn aaye ti ko ni aami
    • Bi ẹkọ imuduro pẹlu awọn ere
  3. Awoṣe wo ni o gbajumọ iṣapẹẹrẹ odi fun kikọ awọn ifibọ ọrọ?
    • BERT
    • ResNet
    • word2vec ká foo-giramu awoṣe
    • GPT-2
  4. Bawo ni iṣapẹẹrẹ odi ṣe yatọ si NCE ni kikun?
    • O ṣe afikun awọn ofin isọdi deede fun deede
    • O nilo gbogbo fokabulari ni gbogbo igbese
    • O nlo ipadanu ipadasẹhin dipo sigmoid kan
    • O ju awọn ofin isọdọtun NCE silẹ, ṣiṣe ni iyara ṣugbọn kii ṣe iṣiro iwuwo deede
  5. Kini idi ti awọn odi nigbagbogbo ṣe apẹẹrẹ lati pinpin unigram dide si agbara 3/4?
    • Lati dọgbadọgba ipa ti loorekoore ati awọn ọrọ toje kuku ju iṣapẹẹrẹ nikan nipasẹ igbohunsafẹfẹ
    • Lati foju awọn ọrọ ṣọwọn patapata
    • Lati jẹ ki gbogbo awọn ọrọ se seese
    • Lati yọ awọn ọrọ idaduro kuro ni ikẹkọ
  6. For a large vocabulary, how can sampled negative pairs reduce work compared with evaluating every word in a full softmax?
    • It computes an exact probability for every vocabulary item
    • The update scores a positive pair and a limited set of sampled negatives instead of normalizing over the whole vocabulary
    • It eliminates the need to score the positive pair
    • It sorts all words by frequency on each step
  7. In skip-gram negative sampling, what is treated as the positive training pair?
    • Two words sampled independently from the noise distribution
    • A word paired with an empty context
    • A word and a context word observed together in the corpus
    • A validation label paired with a model checkpoint
  8. Why is negative sampling not generally interpreted as estimating a normalized language-model probability?
    • It always sums over the full vocabulary
    • It requires exact maximum-likelihood normalization
    • It never uses negative examples
    • Its simplified binary objective drops NCE normalization terms and targets useful embeddings instead