Ikibazo gihuza ibibazo · Biragoye Urwego

Multi-Umutwe Utinze Kwitondera Ikibazo

Shakisha abadepite basangiye cache yihishe, kwiyubaka kurwego rwo hasi, kwiyubaka kwa projection, inzira-kodegisi, hamwe no kuzigama-kwibuka.

Inzira zijyanye no kuyoboraMulti Umutwe Utinze Kwitonda
Ikibazo 1 Bya 8

Ni ikihe kibazo cyibanze Multi-Head Latent Attention yagenewe kugabanya?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ni ikihe kibazo cyibanze Multi-Head Latent Attention yagenewe kugabanya?
    • Igiciro cyo kwibuka cyurufunguzo-agaciro cache mugihe cyo gufata umwanzuro
    • Amahugurwa make
    • Umubare w'icyitegererezo
    • Ingano yamagambo
  2. Nigute Umudepite agabanya cache ya KV?
    • Isiba ibimenyetso bishaje muri cache
    • Irabika ntoya ntoya yo murwego rwihishwa aho kuba urufunguzo rwumutwe nindangagaciro
    • Ikoresha uburemere bwa 1-bito kuri buri kintu
    • Ihagarika ibitekerezo rwose kubimenyetso bimwe
  3. Niyihe moderi yabanje kwerekana Multi-Head Yatinze Kwitonda?
    • GPT-2
    • BERT
    • Kumurongo-V2
    • LLaMA 1
  4. Ni ukubera iki Umudepite akeneye inzira 'decoupled' itandukanye yo guhinduranya imyanya?
    • Kuberako imyanya ntaho ihuriye no kwitabwaho
    • Kuberako RoPE ikubye kabiri amagambo
    • Kuberako byihutisha optimizer
    • Kuberako kuzunguruka bidashobora guhindurwamo ibyinjijwe hejuru-projection nkibisigaye urufunguzo rushobora
  5. Hafi ya KV-cache yo kugabanya ububiko bwa DeepSeek-V2 bwatanzwe na Depite hamwe nibisanzwe imitwe myinshi?
    • Kurenga 90%
    • Abagera kuri 5%
    • Nukuri 50%
    • Nta kugabanuka, gusa byunguka umuvuduko
  6. Ni izihe nyungu zifatika zikurikira biturutse ku nteko ishinga amategeko ntoya kuri buri kashe?
    • Kurandura ibikenewe kuri GPU iyariyo yose
    • Ingano nini yerekana ibyiciro binini hamwe nibisobanuro birebire kubikoresho bimwe
    • Guhindura byikora mu ndimi zose
    • Gukuraho imikorere ya softmax
  7. What is the purpose of absorbing MLA up-projection weights into other projections?
    • Remove positional information from every token
    • Increase the size of the cached latent
    • Avoid materializing full per-head keys and values during inference
    • Replace attention with a recurrent network
  8. What positional information does MLA handle on a separate decoupled key path?
    • Class labels from the training data
    • Image segmentation maps
    • The output vocabulary probabilities
    • Rotary position embeddings that cannot be absorbed like the other key projections