Ikibazo gihuza ibibazo · Biragoye Urwego

Ikibazo Cyitonderwa Ikibazo

Ikizamini cyibibazo-urufunguzo-agaciro kwitondera, gupima softmax, imitwe ibangikanye, igiciro cyinshi cyo kwitaho, kubara-uburemere-kubara, hamwe no guhisha impamvu kubisekuruza.

Inzira zijyanye no kuyoboraUburyo bwo Kwitondera
Ikibazo 1 Bya 8

What does token attention combine when forming an output representation?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. What does token attention combine when forming an output representation?
    • Permanent copies of the entire conversation on disk
    • Value information from allowed positions using computed weights
    • Only manually chosen keywords with no learned scores
    • A translation dictionary that removes every other token
  2. Ni izihe mpapuro zerekana 2017 zatangije ubwubatsi bwa transformateur?
    • 'Kwiga Byimbitse Bisigaye Kumenyekanisha Ishusho'
    • 'Gukina Atari hamwe no Kwiga Byimbitse'
    • 'Kwitonda nibyo ukeneye byose'
    • 'Imiyoboro rusange yo guhangana'
  3. Nibihe bice bitatu byabazwe kuri buri kimenyetso mubitekerezo?
    • Ikibazo, urufunguzo, nagaciro
    • Iyinjiza, ibisohoka, hamwe no kubogama
    • Hagati, hagati, nuburyo
    • Encoder, decoder, nigihombo
  4. Muri formula softmax (QK ^ T / √d_k) V, kuki kugabana na √d_k?
    • Guhindura ibisubizo mubice
    • Kuraho imibare itari myiza
    • Gukora icyitegererezo gikora kuri GPU
    • Kugirango utume amanota yibicuruzwa ataba manini cyane mbere ya softmax
  5. Why does dense self-attention involve growing pairwise work for very long token sequences?
    • Kuberako igomba gukuramo amakuru mashya kuri enterineti
    • Kuberako inyandiko ndende ihora irimo amagambo adasanzwe
    • Kuberako igiciro gikura hamwe na kare yuburebure bwikurikiranya (O (n²))
    • Kuberako ihinduranya imvugo itinda
  6. What capacity does multi-head attention provide through separate learned projections?
    • A guarantee that every head has one unique named meaning
    • Several attention representations that can capture different relationships in parallel
    • Removal of every token embedding
    • A reduction of vocabulary size by definition
  7. In a scalar attention example, values two and six receive weights three-quarters and one-quarter. What is their weighted output?
    • Umunani
    • Bane
    • Babiri
    • Bitatu
  8. What does a causal attention mask prevent in an autoregressive text decoder?
    • Using later token positions when forming the current position's representation
    • Using any previous token
    • Computing any query or key vectors
    • Learning all model parameters