Kedis ku xidhan hagaha · Adag Heerka

YaRN iyo Kediska Fidinta Dhererka Macnaha

Waxay hubisaa sida YaRN u habayso soo noqnoqoshada xadhkaha, u ilaaliso tafaasiisha gudaha, cabirta heerkulka dareenka waxayna ku fidisaa macnaha guud iyadoo si fiican loo habeeyey.

Wadooyinka haga ee la xidhiidhaDhererka dunta iyo macnaha guud
Su'aal 1 ee 8

Habka codaynta booska ayay YaRN wax ka beddeshaa si ay u kordhiso macnaha guud?

Horudhac su'aal kasta8 su'aalo

Jawaabaha iyo sharraxaadaha waxay soo baxayaan markaad qaadanayso kediska.

  1. Habka codaynta booska ayay YaRN wax ka beddeshaa si ay u kordhiso macnaha guud?
    • Ku-xidhka Booska Wareega (Rotary Position Embeddings)
    • Ku-xidhnaanta sinusoidal oo dhammaystiran
    • Miisaska goobaynta ee la bartay
    • Eexda qaraabo kaliya
  2. Maxaa khaldama marka nooca RoPE uu arko taxane ka dheer dhererkiisa tababarka?
    • Tokenizerku wuu burburaa
    • Xaglaha wareeggu waxay galaan kala duwanaansho aan la arkin, waxqabadkuna wuu burburaa
    • Qaabku wuxuu soo saaraa eber kaliya
    • Gradients ayaa noqda mid sax ah
  3. Sidee ayuu YaRN u daaweeyaa cabbirrada soo noqnoqda ee xad-dhaafka ah ee kala duwan?
    • Waxa ay isku xidhaa dhammaan cabbirrada si siman
    • Waxay tirtirtaa cabbirrada soo noqnoqda
    • Waxay isku xirtaa cabbirrada soo noqnoqda hooseeya iyadoo inta badan la ilaalinayo kuwa soo noqnoqonaya
    • Waxay kala soocaysaa cabbir kasta
  4. Marka laga reebo dib-u-cusboonaynta soo noqnoqoshada, waa maxay hagaajinta dheeraadka ah ee YaRN ay khusayso?
    • Tokenizer cusub
    • Laban laabaya tirada lakabyada
    • Ka saaritaanka caadiga ah ee lakabka
    • Habaynta heerkulka feejignaanta (softmax scaling).
  5. Waa maxay faa'iidada dhabta ah ee muhiimka ah ee YaRN marka loo eego isdhexgalka naive?
    • Waxay gaartaa macnaha guud oo leh xog hagaajin aad u yar iyo tillaabooyin
    • Wax model ah uma baahna
    • Waxa kaliya oo ka hooseeya 512 calaamadood
    • Waxay meesha ka saaraysaa baahida GPU-yada
  6. Waa maxay sababta is-dhexgalka tooska ah ee dhammaan soo noqnoqoshada RoPE ay waxyeello u leedahay?
    • Kaliya waxay saamaysaa ereyada eexda
    • Waxay cadaadisaa cabbirada soo noqnoqda sare ee qeexaya faahfaahinta maxalliga ah
    • Waxay ka dhigaysaa qaabka mid weyn
    • Waxay beddeshaa cabbirka erayada
  7. In RoPE, which vectors receive position-dependent rotations?
    • Only the value vectors
    • Only the model’s token IDs
    • The query and key vectors
    • The optimizer’s gradient and momentum
  8. What does YaRN’s attention-temperature adjustment address when context length is extended?
    • The number of GPU layers
    • The dimensions of the token vocabulary
    • The weight decay coefficient
    • Changes in attention entropy or softmax sharpness caused by the longer context