Paged Ntị na vLLM
Paged Ntị bụ usoro njikwa ebe nchekwa nke na-echekwa oghere nlebara anya nke ụdị asụsụ n'obere ngọngọ enwere ike iji ya mee ihe kama ịbụ otu nnukwu mkpọ.
Nchịkọta
It powers vLLM, an open-source serving engine that dramatically boosts how many requests a single GPU can handle.
Ime miri emi
Mgbe ụdị asụsụ na-ewepụta ederede, ọ na-edobe 'KV cache' (igodo na uru vectors) maka akara ọ bụla ọ hụrụ ka akara na-esote wee banye n'uju. Na omenala, arịrịọ ọ bụla debere otu nnukwu ebe nchekwa GPU na-aga n'ihu maka ogologo ogologo ya, na-efunahụ nnukwu ego mgbe usoro dị mkpụmkpụ ma ọ bụ dịgasị iche n'ogologo. PagedNttention, ewebatara na 2023 vLLM akwụkwọ sitere na UC Berkeley, na-enweta echiche nke paging ebe nchekwa mebere site na sistemụ arụmọrụ: ọ na-ekewa oghere KV ka ọ bụrụ ihe mgbochi nwere ike ibi ebe ọ bụla na ebe nchekwa wee kenye ya na ihe achọrọ. Tebụl nchọta na-esetịpụ ọnọdụ akara ezi uche dị na ngọngọ anụ ahụ. Nke a fọrọ nke nta ka ọ kwụsịlata nkewa ebe nchekwa ma na-ahapụ ka ekesa ihe mgbochi, dịka ọmụmaatụ n'ofe ọtụtụ nsonaazụ sitere na otu ngwa ngwa.
Nghọta nka nka
A na-ekewa cache KV n'ime ibe ndị nwere oke, nke ọ bụla na-ejide igodo na ụkpụrụ maka ọnụọgụ akara. Maapụ tebụl nke usoro-usoro, na-esetịpụ ọnọdụ ezi uche dị na ya na ebe ihu akwụkwọ anụ ahụ, yabụ cache nke usoro ekwesịghị ịdị na-aga n'ihu. N'ihi na prefixes yiri ya (otu usoro ngwa ngwa, ma ọ bụ alaka-nchọgharị ọkụ) nwere ike na-atụ aka n'otu ibe anụ ahụ site na idetu-na-dere, a na-ejigharị ebe nchekwa kama ịmegharị ya, na-ebelata ihe mkpofu site na 60% ruo pasent ole na ole.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke Paged Ntị na vLLM
vLLM abụrụla ọkpụkpụ ntụnye isi mmalite mepere emepe, na echiche PagedAttention na-apụta ugbu a n'ofe ọtụtụ nchịkọta ozi. Na-atụ anya caching prefix miri emi (iji sistemụ cache eme ihe na-akpali ndị ọrụ), prefill ekewapụrụ na decode na igwe dị iche iche, atumatu nchụpụ ka mma, yana njikọta siri ike na ọnụọgụ na ngbanwe ntule. Ka windo ndị gbara ya gburugburu na-etolite n'ime ọtụtụ nde token, njikwa KV nwere ihu ọma na-aghọwanye ihe dị mkpa maka idobe ozi ọnụ.
Mmejuputa n'ezie n'ụwa
Bochum API LLM mepere emepe ebe vLLM na-ejere ọtụtụ ndị na-akparịta ụka n'otu oge site na otu GPU na ntinye dị elu.
Ịkekọrịta ogologo usoro ọsọ ọsọ gafere puku kwuru puku arịrịọ site na caching prefix ka e wee hazie ya otu ugboro, ọ bụghị ugboro ugboro.
Ọchụchọ beam na-agba ọsọ ma ọ bụ ọtụtụ mmecha egosipụtara na-ekekọrịta KV blocks maka ngwa ngwa a na-ahụkarị site na idetu-na-dere.
Ibelata ihe mkpofu ebe nchekwa GPU site na nkewa ka onye na-eweta ya nwee ike ibukọ oge n'otu oge karịa n'otu ngwaike
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the PagedAttention and vLLM quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nzuzo dị iche
Ajụjụ a na-ajụkarị
What is PagedAttention and vLLM?
Paged Ntị bụ usoro njikwa ebe nchekwa nke na-echekwa oghere nlebara anya nke ụdị asụsụ n'obere ngọngọ enwere ike iji ya mee ihe kama ịbụ otu nnukwu mkpọ. Ọ na-akwado vLLM, injin na-enye ọrụ mepere emepe nke na-akwalite arịrịọ ole otu GPU nwere ike ijikwa.
Kedu ihe 'KV cache' na-echekwa n'oge ọgbọ ederede?
Ihe nchekwa KV na-ejide igodo na uru vectors maka akara ngosi ọ bụla gara aga, na-ahapụ ka ihe nlereanya ahụ gaa n'ọnọdụ zuru ezu na-agụgharịghị ya nzọụkwụ ọ bụla.
Kedu echiche sistemụ arụmọrụ kpaliri PagedNttention?
PagedNttention na-anata paging ebe nchekwa mebere: A na-ekewa cache KV ka ọ bụrụ ibe edobere nke nwere ike ibi ebe ọ bụla, nke okpokoro ngọngọ mapụtara.
Kedu nsogbu dị na oke nchekwa nchekwa KV ọdịnala ka PagedAttention na-edozi?
Idokwa otu nnukwu slab na-aga n'ihu n'otu arịrịọ, nha maka ogologo ogologo, mefuru nnukwu ebe nchekwa GPU. Ibe akwụkwọ na-ekenye obere ngọngọ na-achọ, na-egbutu ihe mkpofu.
Kedu ka PagedAttention si ekwe ka ọtụtụ mmepụta kesaa ebe nchekwa maka otu ngwa ngwa?
Ihe nrịbama ndị yiri ya (akọrọ mkpali ma ọ bụ alaka-nchọgharị ọkụ) na-ezo aka otu ibe KV anụ ahụ, na-eṅomi naanị mgbe alaka ụlọ ọrụ na-agbanwe.
Ebee ka ewepụtara vLLM na PagedNttention?
vLLM na PagedNttention algọridim si na UC Berkeley pụta n'ime akwụkwọ 2023 wee bụrụ injin na-enye ọrụ mepere emepe ngwa ngwa.