quiz bu lëkkale ak tegtal · Dëgër Niveau

RMSNorm ak laaj ak tontu ngir yamale bu njëkk

Saytu RMSNorm lim, Pre-LN plasement, yoon yu des ak laaj-caabi normalisasioŋ te baña jaawatle yii tànneef jëmmal moom seen bopp.

Yooni njiit yu jëm ciRmsnorm ak normalisasioŋ bu njëkk
Laaj 1 de 8

Lan la RMSNorm bàyyi buñu ko méngale ak LayerNorm buñ miin?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lan la RMSNorm bàyyi buñu ko méngale ak LayerNorm buñ miin?
    • Dindi moyenne (centrewaat)
    • parametru eskaal biñ jàng
    • Lëkkaloo gi des
    • Liggéeyu softmax bi
  2. Ak ban limu RMSNorm di xaaj vecteur bu nekk ci aktivasioŋ?
    • variance bi
    • Rasin bi mooy kaare moyenne
    • Valër absolu bi gëna mag
    • L1 norm
  3. What gradient behavior does Pre-LN support at initialization in deep Transformers?
    • It guarantees stable training for every model without tuning
    • It removes the residual stream from the block
    • It can make gradients better behaved at initialization and reduce some stability problems
    • It makes all gradients exactly equal across layers
  4. Ci benn bloku Pre-LN, ban yoon la couche normalisation bi di bàyyi te kenn laalu ko?
    • poids feed-forward yi
    • token yiñ dugal
    • softmax biy génn
    • Lëkkaloo gi des (dàntite)
  5. Ban famiy xeetu poid ouvert bu bees bi mooy jëfandikoo RMSNorm ci default?
    • Llama, Mistral, ak Qwen
    • AlexNet ak VGG
    • ResNet ak ndoorte
    • BERT-base ak Transformatër bu njëkk bi
  6. Luy 'QK-norm,' aplikaasioŋ bu bees bu RMSNorm?
    • Normaliser logits yiy mujjee
    • Jëfandikoo RMSNorm ngir bàyyi xel ci laaj ak caabi ngir saytu màggug logit
    • Kantite cache valeur clé
    • Benn anam ngir teggi normalisasioŋ
  7. A feature vector has a nonzero mean before RMSNorm. What does the normalization do to that mean?
    • It always sets the mean exactly to zero
    • It replaces each feature with its rank
    • It does not explicitly subtract the mean, though rescaling can change its value
    • It subtracts the maximum activation from every feature
  8. How are RMSNorm and Pre-LN related as design choices?
    • They are two names for subtracting the mean
    • RMSNorm always means normalization after the residual
    • Pre-LN specifies the optimizer rather than the block
    • RMSNorm specifies the normalization calculation; Pre-LN specifies where normalization sits in the residual block