SwiGLU ak quiz biy tàmbali

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

1. Ban jëf core mooy màndargaal ab Unité Lineaire Gated?
   • Yokkateg benn projection ci ñaareelu projection buñ aktive
   • Yokk ñaari projection ligneer
   • Boppu yiy boole
   • Séddale liggéey yi ci seeni norm
2. Ban fonction activation la SwiGLU di jëfandikoo ci branche gate bi?
   • ReLU
   • Swish (Silu)
   • Tanh
   • Softmax
3. Ñaata matris yu diis la benn bloku feed-forward bu SwiGLU di jëfandikoo?
   • Benn
   • Ñaar
   • Ñatt
   • ñeent
4. Lan moo waral dimension bu nëbbu bi di gëna bari lu tollu ci ñaar ci ñett ci SwiGLU layers?
   • Ngir gëna dëgëral lim yi
   • Ngir gëna xóotal model bi
   • Ngir dindi soxla normalisasioŋ
   • Ngir tëye parametre yi ak FLOP yi ñu méngoo ak MLP buñ miin doonte matrix bi dafa yokk
5. Benn ci màndarga yi wuutale Swish ak ReLU mooy:
   • Nooy te du monotone, may negatif yu ndaw
   • Yokk bu baax ba noppi tënk ko ci 1
   • Dafay nuru fonction sigmoïde
   • Dañu ko tënk ci ay dugal yu baax kese
6. Kan moo siiwal SwiGLU ngir transformatër feed-forward ci 2020?
   • Jeofrey Hinton
   • Noam Shasier
   • Yann Lekun
   • Andréj Karpati
7. In the SwiGLU product Swish(xW) multiplied by xV, what happens when one gate value is near zero?
   • The entire Transformer layer is skipped
   • The output projection is replaced by a sigmoid
   • The matching value-branch dimension contributes little to the output
   • The value branch is permanently frozen
8. Why can keeping a standard MLP’s hidden width unchanged increase SwiGLU cost?
   • The SiLU gate removes the output projection
   • SwiGLU computes a full Hessian
   • The value branch has no learned weights
   • SwiGLU uses an extra projection matrix, so the comparable-width design would have more parameters and FLOPs