Guide-yakabatana mibvunzo · Zvakaoma Nhanho

YaRN uye Context Length Extension Quiz

Inotarisa kuti YaRN inogadzirisa sei maFrequency eRoPE, inochengetedza ruzivo rwemunharaunda, inoyera tembiricha yekutarisisa uye inotambanudza mamiriro nekunyatso-tuning.

Nzira dzekutungamira dzinoenderanaYarn uye Context Length Kuwedzerwa
Mubvunzo 1 ye 8

Ndeipi chinzvimbo-encoding nzira inogadziriswa neYaRN kuti iwedzere mamiriro?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Ndeipi chinzvimbo-encoding nzira inogadziriswa neYaRN kuti iwedzere mamiriro?
    • Rotary Position Embeddings (RoPE)
    • Absolute sinusoidal embeddings
    • Akadzidza matafura ekutarisa chinzvimbo
    • Relative bias matrices chete
  2. Chii chinotadza kana modhi yeRoPE ikaona kutevedzana kwakareba kupfuura kureba kwayo kwekudzidziswa?
    • Iyo tokenizer inoparara
    • Makona ekutenderera anopinda muzvikamu zvisingaonekwe uye maitiro anodonha
    • Iyo modhi inoburitsa zero chete
    • Gradients inova imwe chaiyo
  3. YaRN inobata sei akasiyana RoPE frequency zviyero?
    • Inopindirana mativi ese zvakaenzana
    • Inobvisa hukuru-frequency dimensions
    • Iyo inopindirana yakaderera-frequency zviyero uku ichinyanya kuchengetedza yakakwirira-frequency
    • Inoita zvimiro zvese
  4. Kunze kwekudzoreredza ma frequency, ndeipi imwe gadziriso inoshandiswa neYaRN?
    • A new tokenizer
    • Kupeta kaviri nhamba yezvikamu
    • Kubvisa layerization
    • Kutarisa tembiricha (softmax scaling) kugadzirisa
  5. Ndechipi chakakosha chinoshanda mukana weYaRN pamusoro pekusaziva kududzira?
    • Iyo inosvika kureba kureba iine kushomeka-tuning data nematanho
    • Hazvidi modhi zvachose
    • Inongoshanda pasi pe512 tokens
    • Inobvisa kudiwa kweGPUs
  6. Sei kusaziva mutsara kududzira kwese maFrequency eRoPE kuchikuvadza?
    • Zvinongokanganisa mazwi ekurerekera
    • Iyo inomanikidza yakakwira-frequency zviyero izvo encode zvemunharaunda
    • Inoita kuti modhi ive yakakura
    • Inoshandura hukuru hwemashoko
  7. In RoPE, which vectors receive position-dependent rotations?
    • Only the value vectors
    • Only the model’s token IDs
    • The query and key vectors
    • The optimizer’s gradient and momentum
  8. What does YaRN’s attention-temperature adjustment address when context length is extended?
    • The number of GPU layers
    • The dimensions of the token vocabulary
    • The weight decay coefficient
    • Changes in attention entropy or softmax sharpness caused by the longer context