PagedAttention uye vLLM
PagedAttention inzira yekurangarira-yekutarisira iyo inochengeta kutarisisa kwemuenzaniso wemutauro mumabhururu madiki anogona kushandiswa zvakare pachinzvimbo chechunk hombe.
Pfupiso
It powers vLLM, an open-source serving engine that dramatically boosts how many requests a single GPU can handle.
Kudzika Kwakadzika
Kana modhi yemutauro yaburitsa mavara, inochengeta 'KV cache' (kiyi uye kukosha mavheta) pachiratidzo chega chega chayakaona kuitira kuti chiratidzo chinotevera chigone kuenderana nemamiriro akazara. Sechivanhu chikumbiro chega chega chaichengeta imwe hombe inobatika yeGPU ndangariro yakaenzana nekureba kwayo, kutambisa huwandu hukuru apo kutevedzana kwaive kupfupi kana kusiana pakureba. PagedAttention, yakaunzwa mu2023 vLLM bepa kubva kuUC Berkeley, inokwereta iyo pfungwa yekurangarira ndangariro paging kubva kune anoshanda masisitimu: inotsemura iyo KV cache kuita yakagadziriswa-saizi mabhuroko anogona kugara chero kupi mundangariro uye kugovaniswa painodiwa. Tafura yekutarisa inoburitsa logic token positions kumabhuroko emuviri. Izvi zvinenge zvapedza kupatsanurwa kwendangariro uye zvinoita kuti zvivharo zvigoverwe, semuenzaniso pane zvakawanda zvinobuda kubva pakukurumidza kumwechete.
Technical Insight
Iyo KV cache yakakamurwa kuita mapeji akasarudzika-saizi, imwe neimwe inobata makiyi uye kukosha kune yakatarwa nhamba yezviratidzo. Yese-inotevedzana block tafura inomepu zvinzvimbo kune echokwadi mapeji nzvimbo, saka kutevedzana cache haifanire kuve yakabatana. Nekuti zvakafanana prefixes (yakagovaniswa sisitimu yekumhanyisa, kana matavi ekutsvagisa danda) anogona kunongedza kune mamwe mapeji emuviri kuburikidza nekopi-pa-kunyora, ndangariro inoshandiswa patsva pachinzvimbo chekudzokororwa, kupwanya marara kubva pamusoro pe60% kusvika mashoma muzana.
Strategic Impact
Mutengo uye bhajeti
Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
Sarudzo dzakajeka
Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
Kudzora kwemhando yepamusoro
Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Ramangwana rePagedAttention uye vLLM
vLLM yave yakasarudzika yakavhurika-sosi inference musana, uye pfungwa dzePagedAttention zvino dzava kuoneka pane akawanda anoshumira stacks. Tarisira zvakadzika prefix caching (kushandisazve cached system zvinokurudzira vashandisi vese), prefill yakapatsanurwa uye decode pamichina yakaparadzana, mitemo yekudzinga yakangwara, uye kubatana kwakasimba nehuwandu uye kufungidzira decoding. Sezvo mahwindo emukati achikura kuita mamirioni ezviratidzo, inoshanda peji yeKV manejimendi inova yakanyanya pakati pekuchengetedza kushumira kuchikwanisika.
Real-World Implementation
Kubata yakavhurika-sosi LLM API uko vLLM inoshandira vazhinji vanobatana vashandisi vekutaura kubva kune imwe GPU pakakwirira kuburikidza.
Kugovera hurongwa hurefu hwekukurumidza muzviuru zvezvikumbiro kuburikidza ne prefix caching saka inogadziriswa kamwe chete, kwete kudzokororwa.
Kumhanya kutsvaga danda kana akawanda masampuli ekupedzisira anogovera KV zvidhinha zveyakajairwa kukurumidza kuburikidza nekukopa-pa-kunyora.
Kucheka marara echiyeuchidzo cheGPU kubva mukutsemuka kuitira kuti mupi agone kurongedza mamwe masesheni panguva imwe chete pane imwechete hardware.
Njodzi & Guardrails
Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Implementation Roadmap
Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
Benchmark pasi pechokwadi mutoro uye data mamiriro.
Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the PagedAttention and vLLM quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Kusiyanisa Kuvanzika
Mibvunzo inowanzo bvunzwa
What is PagedAttention and vLLM?
PagedAttention inzira yekurangarira-yekutarisira iyo inochengeta kutarisisa kwemuenzaniso wemutauro mumabhururu madiki anogona kushandiswa zvakare pachinzvimbo chechunk hombe. Inopa vLLM simba, yakavhurika-sosi inoshumira injini iyo inowedzera zvinoshamisa kuti zvingani zvikumbiro zvinogona kubata GPU imwe chete.
Chii chinochengeterwa 'KV cache' panguva yekugadzira zvinyorwa?
Iyo KV cache inobata kiyi uye inokoshesa mavheji kune yega yega chiratidzo, ichirega modhi ichipinda kune yakazara mamiriro pasina kuidzokorora nhanho imwe neimwe.
Ndeipi inoshanda-system pfungwa yakafuridzira PagedAttention?
PagedAttention inokwereta chaiyo yekurangarira paging: iyo KV cache yakakamurwa kuita yakagadziriswa-saizi mapeji anogona kugara chero kupi, akamepurwa nebhuroka tafura.
Ndeipi dambudziko neyakagoverwa yeKV cache inogadziriswa PagedAttention?
Kuchengeta imwe hombe contiguous slab pachikumbiro, saizi yehurefu hwemax, yakapambadza huwandu hukuru hweGPU ndangariro. Paging inogovera zvidhinha zvidiki pane zvinodiwa, kucheka marara.
Sei PagedAttention inorega akawanda anobuda achigovera ndangariro kune yakajairwa kukurumidza?
Zvakafanana prefixes (kukurudzira kwakagovaniswa kana matavi ekutsvagisa danda) anonongedza iwo mapeji eKV chaiwo, achikopa chete kana bazi rasiyana.
VLLM nePagedAttention zvakagadziridzwa kupi?
vLLM uye iyo PagedAttention algorithm yakabuda muUC Berkeley mubepa ra2023 uye nekukasira ikava injini inoshandiswa zvakanyanya yakavhurika sosi.