Ubuyobozi bwa tekiniki
Urupapuro Icyitonderwa na vLLM
PagedAttention manages transformer key/value caches using fixed-size blocks mapped from logical token positions.
Umva iki gitaboAI-byavuzwe · ~2 min
Kuriyi page2 min soma
Incamake
This reduces cache-allocation waste and supports sharing, while vLLM’s actual throughput depends on the workload and system.
Kwibira cyane
Iyo imvugo yindimi itanga inyandiko, igumana 'KV cache' (urufunguzo nagaciro kerekana agaciro) kuri buri kimenyetso yabonye kugirango ikimenyetso gikurikira gishobora kwitabira imiterere yuzuye. Ubusanzwe buri cyifuzo cyabitswe icyapa kinini gihuza ububiko bwa GPU gifite ubunini buringaniye bushoboka, butakaza amafaranga menshi mugihe ibyakurikiranye byari bigufi cyangwa bitandukanye muburebure. Urupapuro rwitonderwa, rwatangijwe mu mpapuro za 2023 vLLM zivuye muri UC Berkeley, rutira igitekerezo cyo kwibuka ububiko bwa paji kuva muri sisitemu y'imikorere: igabanya cache cache mubice binini byagenwe bishobora gutura ahantu hose mububiko kandi bigatangwa kubisabwa. Imbonerahamwe yerekana ikarita yerekana ibimenyetso byerekana ibimenyetso bifatika. Ibi hafi gukuraho ibice byo kwibuka hanyuma bikareka guhagarika bigasangirwa, kurugero rwibisubizo byinshi bivuye kumurongo umwe.
Gushyira mu bikorwa Isi
Kwakira isoko-ifunguye LLM API aho vLLM ikorera abakoresha ibiganiro byinshi bahurira kuri GPU imwe murwego rwo hejuru
Kugabana sisitemu ndende ihita ibihumbi n'ibisabwa ukoresheje prefix cishing kuburyo itunganywa rimwe, ntabwo inshuro nyinshi
Gukoresha ibiti bishakisha cyangwa ibyitegererezo byinshi byuzuzanya bisangiye KV kubisanzwe mubisanzwe ukoresheje kopi-yandika
Gukata imyanda ya GPU ivuye mu gucamo ibice kugirango uyitanga ashobora gupakira icyarimwe icyarimwe kubikoresho bimwe
Kazoza Kurupapuro Icyitonderwa na vLLM
vLLM yahindutse umwirondoro ufunguye-inkomoko yumugongo, kandi ibitekerezo bya PagedIbyitonderwa ubu bigaragara mubice byinshi bikora. Witegereze byimbitse byimbitse (kongera gukoresha sisitemu ya cache yerekana kubakoresha), prefill itandukanijwe hamwe na decode kumashini zitandukanye, politiki yo kwirukana ubwenge, hamwe no guhuza hamwe no kubara no gushushanya. Mugihe imiterere ya Windows ikura miriyoni yikimenyetso, imiyoborere ya paje ya KV ikora neza cyane kugirango ikomeze serivisi ihendutse.
Kubishyira mubikorwa
Impamvu ari ngombwa
- Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
- Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
- Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Witondere
- Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
- Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
- Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Intambwe ikurikira
- Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
- Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
- Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
- Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Ibaruwa y'amakuru y'ubuntu
Ibuka amakuru ya buri munsi ya AI
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Take the PagedAttention and vLLM quiz
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Ibibazo bikunze kubazwa
Niki Urupapuro rwitonderwa na vLLM?
PagedAttention manages transformer key/value caches using fixed-size blocks mapped from logical token positions. This reduces cache-allocation waste and supports sharing, while vLLM’s actual throughput depends on the workload and system.
Niki 'KV cache' ibika mugihe cyo kwandika inyandiko?
Ubwihisho bwa KV bufite urufunguzo nagaciro byagaciro kuri buri kimenyetso kibanziriza iki, kureka icyitegererezo kikitabira imiterere yuzuye utabanje kubara buri ntambwe.
Ni ubuhe buryo bwa sisitemu y'imikorere yahumekeye Paged?
Urupapuro rwitondewe ruguza ububiko bwibikoresho bya paji: KV cache igabanijwemo impapuro zingana-zishobora gutura ahantu hose, zashushanijwe nimbonerahamwe.
Ni ikihe kibazo kijyanye no kugabura kwa KV gakondo PagedIbyifuzo bikemura?
Kubika icyapa kinini gihuza icyifuzo, gifite ubunini burebure, cyatakaje umubare munini wibikoresho bya GPU. Paji itanga uduce duto kubisabwa, gutema imyanda.
Nigute PagedIbyitonderwa bireka ibisubizo byinshi bigabana kwibuka kubintu bisanzwe?
Imbanzirizamushinga imwe (ibisobanuro bisangiwe cyangwa amashami ashakisha amashami) yerekana page imwe ya KV yumubiri, ikopera gusa iyo ishami ritandukanye.
VLLM na PagedIbyifuzo byatejwe imbere he?
vLLM na PagedAttention algorithm yasohotse muri UC Berkeley mumpapuro 2023 ihita ihinduka moteri ikoreshwa cyane-itanga moteri ikora.
Komeza wige
Kuyobora
Abandi bayobozi batoranijwe kuriyi ngingo