Ikibazo gihuza ibibazo · Hagati Urwego

Ikibazo cya KV

Kugenzura imikurire ya KV-cache, kugabanwa kurupapuro, kugabana ibibazo kubitsinda, kubanziriza kongera gukoresha ibihe, kubara, no gupima kwibuka hamwe nibisobanuro hamwe.

Inzira zijyanye no kuyoboraKv Cache Optimisation
Ikibazo 1 Bya 8

Ububiko bwa KV bubika iki kandi kuki?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ububiko bwa KV bubika iki kandi kuki?
    • Mbere yabazwe urufunguzo nindangagaciro kuburyo bitabazwe buri ntambwe
    • Icyitegererezo cyamahugurwa yo kongera imyitozo
    • Ijambo ryibanga ryumukoresha kugirango wemeze
    • Inyandiko yanyuma isohoka gusa
  2. Kuki Kache ishobora guhinduka ikibazo cyo kwibuka?
    • Yonona GPU burundu
    • Irakura hamwe nuburebure bwikurikiranya, ingano yicyiciro, hamwe nibice, rimwe na rimwe bikarenga uburemere bwikitegererezo
    • Cyakora gusa kumyandiko ngufi kuruta amagambo 10
    • Yigana sisitemu y'imikorere
  3. Ni ubuhe buryo bukoreshwa-sisitemu ikora PagedAttention iguza?
    • Ibikoresho bya disikuru
    • Gusikana Antivirus
    • Virtual-memory paji hamwe nubunini buhamye
    • Igipimo cyerekana imiterere
  4. Nigute Itsinda-Ikibazo na Multi-Ikibazo Kwitonda bigabanya ubunini bwa KV?
    • Mugusiba uburyo bwo kwitondera
    • Kubika cache nkibishusho
    • Muguhugura kumibare mike
    • Mureke imitwe myinshi yibibazo isangire bike urufunguzo / agaciro imitwe
  5. Ni izihe nyungu imwe yo gusangira prefix muri cache cache?
    • Gusaba hamwe nikintu kimwe cyo gutangira gishobora kongera gukoresha kashe imwe
    • Ihishe icyitegererezo
    • Itinda icyifuzo cyose kimwe
    • Bikuraho gukenera kwitabwaho
  6. Compared with storing the same KV entries in 16-bit, what does 8-bit storage do to raw cache-value bytes?
    • It doubles the raw bytes
    • It roughly halves the raw bytes, before metadata and other runtime memory
    • It removes the key and value tensors entirely
    • It guarantees twice the inference speed
  7. With full-context attention, no prefix sharing, and a fixed model and cache dtype, what happens if sequence length and batch size both double?
    • It stays constant because model weights are fixed
    • It roughly doubles
    • Raw KV storage grows to roughly four times its previous size
    • Ihinduka kimwe cya kane kinini
  8. What must generally match for two requests to reuse the same cached prefix blocks?
    • Only the user account name
    • The requests must have different model weights
    • The prompts must have the same character count but different tokens
    • Their tokenized prefix and relevant model/cache configuration must be compatible