PagedAttention kanye ne-vLLM
I-PagedAttention iyindlela yokuphatha inkumbulo egcina inqolobane yokunaka yemodeli yolimi kumabhulokhi amancane asebenziseka kabusha esikhundleni sesiqephu esisodwa esikhulu esihlangene.
Uhlolojikelele
It powers vLLM, an open-source serving engine that dramatically boosts how many requests a single GPU can handle.
I-Deep Dive
Uma imodeli yolimi ikhiqiza umbhalo, igcina 'inqolobane ye-KV' (amavekhtha angukhiye kanye nenani) kuwo wonke amathokheni ayibonile ukuze ithokheni elandelayo ihambisane nengqikithi egcwele. Ngokwesiko isicelo ngasinye besigodla i-slab eyodwa enkulu yememori ye-GPU enosayizi wobude bayo obukhulu obungaba khona, simosha amanani amakhulu lapho ukulandelana kukufushane noma kuhluka ngobude. I-PagedAttention, eyethulwe ephepheni le-vLLM lango-2023 elivela e-UC Berkeley, iboleka umqondo wokuphethwa kwememori ebonakalayo ezinhlelweni zokusebenza: ihlukanisa inqolobane ye-KV ibe amabhulokhi anosayizi ongashintshi ongaphila noma kuphi enkumbulo futhi anikezwe ngokufunwa. Ithebula lokubheka libonisa ukuma kwamathokheni okunengqondo kumabhulokhi abonakalayo. Lokhu kucishe kuqede ukuhlukana kwenkumbulo futhi kuvumela amabhulokhi ukuthi abelwane, isibonelo kuyo yonke imiphumela eminingi evela ekwazisweni okufanayo.
I-Technical Insight
Inqolobane ye-KV ihlukaniswa ibe amakhasi anosayizi ongashintshi, ngalinye liphethe okhiye namanani enombolo emisiwe yamathokheni. Ithebula lebhulokhi elilandelanayo libonisa izindawo ezinengqondo ezindaweni zekhasi, ngakho-ke inqolobane yokulandelana akufanele ihlangane. Ngenxa yokuthi iziqalo ezifanayo (ukwaziswa kwesistimu okwabelwana ngayo, noma amagatsha okusesha i-beam) zingakhomba amakhasi abonakalayo afanayo ngokukopisha-phezu kokubhala, inkumbulo iphinda isetshenziswe esikhundleni sokuphindaphinda, ukusika imfucuza isuka ngaphezu kuka-60% iye kumaphesenti ambalwa.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa le-PagedAttention kanye ne-vLLM
I-vLLM isiphenduke inkomba ezenzakalelayo yomthombo ovulekile, futhi imibono ye-PagedAttention manje isivela kuzo zonke izitaki eziningi. Lindela ukugcinwa kwesikhashana kwesiqalo esijulile (ukusebenzisa kabusha ukwaziswa kwesistimu efakwe kunqolobane kubasebenzisi bonke), ukugcwalisa kwangaphambili okuhlukanisiwe kanye nokukhipha ikhodi emishinini ehlukene, izinqubomgomo zokuxoshwa ezihlakaniphile, nokuhlanganiswa okuqinile nobuningi nokukhishwa kwamakhodi okucabangelayo. Njengoba amafasitela womongo ekhula abe izigidi zamathokheni, ukuphathwa kwe-KV okunekhasi okusebenzayo kuba okuphakathi kakhulu ekugcineni ukukhonza kuthengeka.
Ukuqaliswa Komhlaba Wangempela
Ukusingatha i-LLM API yomthombo ovulekile lapho i-vLLM isiza abasebenzisi abaningi bengxoxo ngesikhathi esisodwa kusukela ku-GPU eyodwa ekuphumeni okuphezulu
Ukwabelana ngokwaziswa kwesistimu ende ezinkulungwaneni zezicelo ngokusebenzisa ukulondoloza okwesikhashana kwesiqalo ukuze kucutshungulwe kanye, hhayi ngokuphindaphindiwe.
Ukuqalisa ukusesha kwe-beam noma amasampula amaningi aqediwe abelana ngamabhulokhi e-KV ngokwaziswa okuvamile ngokukopisha-ngokubhala
Ukusika imfucuza yememori ye-GPU kusukela ekuhlukaneni ukuze umhlinzeki akwazi ukupakisha izikhathi eziningi ngasikhathi sinye kuhadiwe efanayo
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the PagedAttention and vLLM quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ubumfihlo Obuhlukile
Imibuzo evame ukubuzwa
What is PagedAttention and vLLM?
I-PagedAttention iyindlela yokuphatha inkumbulo egcina inqolobane yokunaka yemodeli yolimi kumabhulokhi amancane asebenziseka kabusha esikhundleni sesiqephu esisodwa esikhulu esihlangene. Inika amandla i-vLLM, injini enikeza umthombo ovulekile ekhuphula ngokuphawulekayo ukuthi zingaki izicelo i-GPU eyodwa engakwazi ukuzisingatha.
Igcina ini 'inqolobane ye-KV' ngesikhathi sokwenziwa kombhalo?
Inqolobane ye-KV iphethe ama-vectors ayisihluthulelo kanye nenani layo yonke ithokheni yangaphambili, ivumela imodeli ukuthi inakekele umongo ogcwele ngaphandle kokubala kabusha isinyathelo ngasinye.
Imuphi umqondo wesistimu yokusebenza ophefumulelwe i-PargedAttention?
I-PagedAttention iboleka i-virtual memory pageging: inqolobane ye-KV ihlukaniswa ibe amakhasi anosayizi ongashintshi ongahlala noma yikuphi, afakwe imephu nge-block table.
Iyiphi inkinga ngokwabiwa kwenqolobane ye-KV yendabuko i-PagedAttention eyixazululayo?
Ukugcina i-slab eyodwa enkulu ehlangene ngesicelo ngasinye, esilinganiselwe ngobude obukhulu, kumoshe inani elikhulu lememori ye-GPU. Ukupakisha kwabela amabhulokhi amancane ngokufunwa, ukusika imfucuza.
I-PagedAttention ivumela kanjani okuphumayo okuningi ukwabelana ngenkumbulo ngokwaziswa okujwayelekile?
Iziqalo ezifanayo (izaziso ezabiwe noma amagatsha okusesha i-beam) zibhekisela kumakhasi afanayo e-KV aphathekayo, akopisha kuphela uma igatsha lihlukana.
Yakhelwa kuphi i-vLLM kanye ne-PagedAttention ekuqaleni?
I-vLLM kanye ne-algorithm ye-PagedAttention kuphume e-UC Berkeley ephepheni lango-2023 futhi ngokushesha yaba injini esetshenziswa kabanzi yomthombo ovulekile.