Guide-yakabatana mibvunzo · Zvakaoma Nhanho

RMSNorm uye Pre-Layer Normalization Quiz

Inotarisa RMSNorm manhamba, Pre-LN kuiswa, nzira dzakasara uye query-kiyi yekumisikidza pasina kuvhiringa idzi dzakazvimirira dhizaini sarudzo.

Nzira dzekutungamira dzinoenderanaRmsnorm uye Pre Layer Normalization
Mubvunzo 1 ye 8

Chii chinosiya RMSNorm ichienzaniswa neyakajairwa LayerNorm?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Chii chinosiya RMSNorm ichienzaniswa neyakajairwa LayerNorm?
    • Kurevesa kubvisa (re-centing)
    • Iyo yakadzidza chiyero parameter
    • Kubatana kwasara
    • Iko kushanda kweSoftmax
  2. Nehuwandu hupi RMSNorm inogovanisa yega yega activation vector?
    • Kusiyana kwayo
    • Mudzi wayo unoreva mativi mana
    • Huwandu hwayo hwakakwana hwakakwana
    • Yayo L1 yakajairika
  3. Ndeapi maitiro egradient anotsigira Pre-LN pakutanga mune zvakadzika Transformers?
    • Inovimbisa kudzidziswa kwakagadzikana kune yega yega modhi pasina tuning
    • Inobvisa rukova rwakasara kubva mubhuroko
    • Inogona kuita kuti ma gradients aite zvirinani pakutanga uye kuderedza mamwe matambudziko ekugadzikana
    • Inoita kuti magradients ese aenzane chaizvo pamatanho
  4. Mune Pre-LN block, ndeipi nzira iyo iyo normalization layer inosiya nemaune isina kubatwa?
    • Iwo mafeed-forward weights
    • Kuiswa kwechiratidzo
    • Iyo yakabuda softmax
    • Iyo yakasara (identity) yekubatanidza
  5. Ndedzipi dzemazuva ano dzakavhurika-huremu modhi mhuri dzinoshandisa RMSNorm nekukasira?
    • LLaMA, Mistral, uye Qwen
    • AlexNet uye VGG
    • ResNet uye Kutanga
    • BERT-base uye yepakutanga Transformer
  6. Chii chinonzi 'QK-norm,' ichangoburwa application yeRMSNorm?
    • Normalizing ekupedzisira gohogoti
    • Kushandisa RMSNorm kumibvunzo yekutarisisa uye makiyi ekudzora kukura kwelogiti
    • Kuenzanisa kiyi-kukosha cache
    • Nzira yekudarika normalization zvachose
  7. A feature vector ine nonzero zvinoreva pamberi peRMMSNorm. Chii chinonzi normalization chinoita kune izvo zvinorevei?
    • Inogara ichiisa chirevo chaicho kusvika zero
    • Inotsiva chinhu chimwe nechimwe nechinzvimbo chayo
    • Iyo haibvisi zvakajeka zvinoreva, kunyange kudzoreredza kunogona kushandura kukosha kwayo
    • Inobvisa iyo yakanyanya activation kubva kune ese maficha
  8. Ko RMSNorm uye Pre-LN ine hukama sei sesarudzo dzekugadzira?
    • Ndiwo mazita maviri ekubvisa zvinoreva
    • RMSNorm inogara ichireva normalization mushure mekusara
    • Pre-LN inotsanangura iyo optimizer kwete iyo block
    • RMSNorm inotsanangura iyo normalization kuverenga; Pre-LN inotsanangura panogara normalization mune yakasara block