Ikibazo gihuza ibibazo · Biragoye Urwego
Multi-Umutwe Utinze Kwitondera Ikibazo
Shakisha abadepite basangiye cache yihishe, kwiyubaka kurwego rwo hasi, kwiyubaka kwa projection, inzira-kodegisi, hamwe no kuzigama-kwibuka.
Ikibazo 1 Bya 8
Ni ikihe kibazo cyibanze Multi-Head Latent Attention yagenewe kugabanya?
Reba buri kibazo8 ibibazo
Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.
- Ni ikihe kibazo cyibanze Multi-Head Latent Attention yagenewe kugabanya?
- Igiciro cyo kwibuka cyurufunguzo-agaciro cache mugihe cyo gufata umwanzuro
- Amahugurwa make
- Umubare w'icyitegererezo
- Ingano yamagambo
- Nigute Umudepite agabanya cache ya KV?
- Isiba ibimenyetso bishaje muri cache
- Irabika ntoya ntoya yo murwego rwihishwa aho kuba urufunguzo rwumutwe nindangagaciro
- Ikoresha uburemere bwa 1-bito kuri buri kintu
- Ihagarika ibitekerezo rwose kubimenyetso bimwe
- Niyihe moderi yabanje kwerekana Multi-Head Yatinze Kwitonda?
- GPT-2
- BERT
- Kumurongo-V2
- LLaMA 1
- Ni ukubera iki Umudepite akeneye inzira 'decoupled' itandukanye yo guhinduranya imyanya?
- Kuberako imyanya ntaho ihuriye no kwitabwaho
- Kuberako RoPE ikubye kabiri amagambo
- Kuberako byihutisha optimizer
- Kuberako kuzunguruka bidashobora guhindurwamo ibyinjijwe hejuru-projection nkibisigaye urufunguzo rushobora
- Hafi ya KV-cache yo kugabanya ububiko bwa DeepSeek-V2 bwatanzwe na Depite hamwe nibisanzwe imitwe myinshi?
- Kurenga 90%
- Abagera kuri 5%
- Nukuri 50%
- Nta kugabanuka, gusa byunguka umuvuduko
- Ni izihe nyungu zifatika zikurikira biturutse ku nteko ishinga amategeko ntoya kuri buri kashe?
- Kurandura ibikenewe kuri GPU iyariyo yose
- Ingano nini yerekana ibyiciro binini hamwe nibisobanuro birebire kubikoresho bimwe
- Guhindura byikora mu ndimi zose
- Gukuraho imikorere ya softmax
- What is the purpose of absorbing MLA up-projection weights into other projections?
- Remove positional information from every token
- Increase the size of the cached latent
- Avoid materializing full per-head keys and values during inference
- Replace attention with a recurrent network
- What positional information does MLA handle on a separate decoupled key path?
- Class labels from the training data
- Image segmentation maps
- The output vocabulary probabilities
- Rotary position embeddings that cannot be absorbed like the other key projections