RMSNorm ati Ibere-Layer Normaization Quiz

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

1. Kini RMSNorm yọkuro ni akawe si LayerNorm boṣewa?
   • Itumọ iyokuro (tun-ti dojukọ)
   • Paramita asekale eko
   • Awọn iyokù asopọ
   • Awọn iṣẹ-ṣiṣe softmax
2. Nipa iye wo ni RMSNorm ṣe pin awọn fekito imuṣiṣẹ kọọkan?
   • Iyatọ rẹ
   • Gbongbo rẹ tumọ si onigun mẹrin
   • O pọju idi iye
   • Ilana L1 rẹ
3. Ihuwasi gradient wo ni atilẹyin Pre-LN ni ibẹrẹ ni Awọn Ayirapada jinlẹ?
   • It guarantees stable training for every model without tuning
   • It removes the residual stream from the block
   • It can make gradients better behaved at initialization and reduce some stability problems
   • It makes all gradients exactly equal across layers
4. Ni a Pre-LN Àkọsílẹ, ọna wo ni awọn normalization Layer koto fi untouched?
   • Awọn iwọn ifunni-siwaju
   • Awọn ifibọ àmi
   • O wu softmax
   • Awọn iyokù (idanimọ) asopọ
5. Awọn idile awoṣe-ìmọ iwuwo ode oni wo lo RMSNorm nipasẹ aiyipada?
   • LLAMA, Mistral, ati Qwen
   • AlexNet ati VGG
   • ResNet ati ibẹrẹ
   • BERT-mimọ ati awọn atilẹba Amunawa
6. Kini 'QK-norm,' ohun elo aipẹ ti RMSNorm?
   • Normalizing ik o wu logits
   • Lilo RMSNorm si awọn ibeere akiyesi ati awọn bọtini lati ṣakoso idagbasoke logit
   • Didiwọn kaṣe iye bọtini
   • Ọna kan lati fo isọdọtun patapata
7. A feature vector has a nonzero mean before RMSNorm. What does the normalization do to that mean?
   • It always sets the mean exactly to zero
   • It replaces each feature with its rank
   • It does not explicitly subtract the mean, though rescaling can change its value
   • It subtracts the maximum activation from every feature
8. How are RMSNorm and Pre-LN related as design choices?
   • They are two names for subtracting the mean
   • RMSNorm always means normalization after the residual
   • Pre-LN specifies the optimizer rather than the block
   • RMSNorm specifies the normalization calculation; Pre-LN specifies where normalization sits in the residual block