Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Kuboresha Akiba ya KV

Hukagua ukuaji wa akiba ya KV, mgao wa kurasa, ushiriki wa hoja uliopangwa kwa vikundi, masharti ya kutumia kiambishi awali, ukadiriaji, na kuongeza kumbukumbu kwa muktadha na upatanifu.

Njia za mwongozo zinazohusianaUboreshaji wa Akiba ya Kv
Swali 1 ya 8

Kashe ya KV huhifadhi nini na kwa nini?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Kashe ya KV huhifadhi nini na kwa nini?
    • Vifunguo na thamani zilizokokotwa hapo awali ili zisihesabiwe upya kwa kila hatua
    • Data ya mafunzo ya mfano kwa ajili ya mafunzo upya
    • Nywila za mtumiaji kwa uthibitishaji
    • Toleo la mwisho la maandishi pekee
  2. Kwa nini kashe ya KV inaweza kuwa shida ya kumbukumbu?
    • Inaharibu GPU kabisa
    • Inakua na urefu wa mlolongo, saizi ya bechi na tabaka, wakati mwingine huzidi uzani wa mfano
    • Inafanya kazi kwenye maandishi mafupi kuliko maneno 10 pekee
    • Inarudia mfumo wa uendeshaji
  3. Je, ni dhana gani ya mfumo wa uendeshaji ambayo PagedAttention inakopa?
    • Huduma za kugawanyika kwa diski
    • Uchanganuzi wa antivirus
    • Uwekaji ukurasa wa kumbukumbu-pepe na vizuizi vya saizi isiyobadilika
    • Kuongeza ubora wa skrini
  4. Je, Maswali ya Kikundi na Umakini wa Maswali mengi hupunguza vipi saizi ya akiba ya KV?
    • Kwa kufuta utaratibu wa kuzingatia
    • Kwa kuhifadhi kache kama picha
    • Kwa mafunzo juu ya data kidogo
    • Kwa kuruhusu vichwa vingi vya hoja kushiriki vichwa vichache vya vitufe/thamani
  5. Je, ni faida gani moja ya kushiriki kiambishi awali kwenye kashe ya KV?
    • Maombi yenye kidokezo sawa cha kuanzia yanaweza kutumia tena vizuizi vile vile vilivyoakibishwa
    • Inasimba kielelezo kwa njia fiche
    • Inapunguza kasi ya kila ombi kwa usawa
    • Huondoa hitaji la umakini
  6. Compared with storing the same KV entries in 16-bit, what does 8-bit storage do to raw cache-value bytes?
    • It doubles the raw bytes
    • It roughly halves the raw bytes, before metadata and other runtime memory
    • It removes the key and value tensors entirely
    • It guarantees twice the inference speed
  7. With full-context attention, no prefix sharing, and a fixed model and cache dtype, what happens if sequence length and batch size both double?
    • It stays constant because model weights are fixed
    • It roughly doubles
    • Raw KV storage grows to roughly four times its previous size
    • It becomes one quarter as large
  8. What must generally match for two requests to reuse the same cached prefix blocks?
    • Only the user account name
    • The requests must have different model weights
    • The prompts must have the same character count but different tokens
    • Their tokenized prefix and relevant model/cache configuration must be compatible