Urupapuro Icyitonderwa na vLLM
Urupapuro rwitonderwa nubuhanga bwo kwibuka-bubika ububiko bwururimi rwibanze rwitondewe mubice bito byongera gukoreshwa aho kuba igice kinini gihuza.
Incamake
It powers vLLM, an open-source serving engine that dramatically boosts how many requests a single GPU can handle.
Kwibira cyane
Iyo imvugo yindimi itanga inyandiko, igumana 'KV cache' (urufunguzo nagaciro kerekana agaciro) kuri buri kimenyetso yabonye kugirango ikimenyetso gikurikira gishobora kwitabira imiterere yuzuye. Ubusanzwe buri cyifuzo cyabitswe icyapa kinini gihuza ububiko bwa GPU gifite ubunini buringaniye bushoboka, butakaza amafaranga menshi mugihe ibyakurikiranye byari bigufi cyangwa bitandukanye muburebure. Urupapuro rwitonderwa, rwatangijwe mu mpapuro za 2023 vLLM zivuye muri UC Berkeley, rutira igitekerezo cyo kwibuka ububiko bwa paji kuva muri sisitemu y'imikorere: igabanya cache cache mubice binini byagenwe bishobora gutura ahantu hose mububiko kandi bigatangwa kubisabwa. Imbonerahamwe yerekana ikarita yerekana ibimenyetso byerekana ibimenyetso bifatika. Ibi hafi gukuraho ibice byo kwibuka hanyuma bikareka guhagarika bigasangirwa, kurugero rwibisubizo byinshi bivuye kumurongo umwe.
Ubushishozi
Ubwihisho bwa KV bugabanijwemo impapuro zingana, buriwese afite urufunguzo nagaciro kumubare wibimenyetso. Imbonerahamwe ikurikiranye yerekana ikarita yerekana imyanya yumwanya wurupapuro rwumubiri, bityo cache ikurikiranye ntigomba guhuzwa. Kuberako imbanzirizamushinga imwe (sisitemu isangiwe, cyangwa amashami ashakisha amashami) irashobora kwerekana urupapuro rumwe rwumubiri ukoresheje kopi-yandika, kwibuka bikoreshwa aho kubigana, kumenagura imyanda kuva hejuru ya 60% kugeza kuri bike ku ijana.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza Kurupapuro Icyitonderwa na vLLM
vLLM yahindutse umwirondoro ufunguye-inkomoko yumugongo, kandi ibitekerezo bya PagedIbyitonderwa ubu bigaragara mubice byinshi bikora. Witegereze byimbitse byimbitse (kongera gukoresha sisitemu ya cache yerekana kubakoresha), prefill itandukanijwe hamwe na decode kumashini zitandukanye, politiki yo kwirukana ubwenge, hamwe no guhuza hamwe no kubara no gushushanya. Mugihe imiterere ya Windows ikura miriyoni yikimenyetso, imiyoborere ya paje ya KV ikora neza cyane kugirango ikomeze serivisi ihendutse.
Gushyira mu bikorwa Isi
Kwakira isoko-ifunguye LLM API aho vLLM ikorera abakoresha ibiganiro byinshi bahurira kuri GPU imwe murwego rwo hejuru
Kugabana sisitemu ndende ihita ibihumbi n'ibisabwa ukoresheje prefix cishing kuburyo itunganywa rimwe, ntabwo inshuro nyinshi
Gukoresha ibiti bishakisha cyangwa ibyitegererezo byinshi byuzuzanya bisangiye KV kubisanzwe mubisanzwe ukoresheje kopi-yandika
Gukata imyanda ya GPU ivuye mu gucamo ibice kugirango uyitanga ashobora gupakira icyarimwe icyarimwe kubikoresho bimwe
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the PagedAttention and vLLM quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Ibanga ritandukanye
Ibibazo bikunze kubazwa
What is PagedAttention and vLLM?
Urupapuro rwitonderwa nubuhanga bwo kwibuka-bubika ububiko bwururimi rwibanze rwitondewe mubice bito byongera gukoreshwa aho kuba igice kinini gihuza. Iha imbaraga vLLM, ifungura-isoko itanga moteri yongerera imbaraga kuburyo bwinshi ibyifuzo GPU imwe ishobora gukora.
Niki 'KV cache' ibika mugihe cyo kwandika inyandiko?
Ubwihisho bwa KV bufite urufunguzo nagaciro byagaciro kuri buri kimenyetso kibanziriza iki, kureka icyitegererezo kikitabira imiterere yuzuye utabanje kubara buri ntambwe.
Ni ubuhe buryo bwa sisitemu y'imikorere yahumekeye Paged?
Urupapuro rwitondewe ruguza ububiko bwibikoresho bya paji: KV cache igabanijwemo impapuro zingana-zishobora gutura ahantu hose, zashushanijwe nimbonerahamwe.
Ni ikihe kibazo kijyanye no kugabura kwa KV gakondo PagedIbyifuzo bikemura?
Kubika icyapa kinini gihuza icyifuzo, gifite ubunini burebure, cyatakaje umubare munini wibikoresho bya GPU. Paji itanga uduce duto kubisabwa, gutema imyanda.
Nigute PagedIbyitonderwa bireka ibisubizo byinshi bigabana kwibuka kubintu bisanzwe?
Imbanzirizamushinga imwe (ibisobanuro bisangiwe cyangwa amashami ashakisha amashami) yerekana page imwe ya KV yumubiri, ikopera gusa iyo ishami ritandukanye.
VLLM na PagedIbyifuzo byatejwe imbere he?
vLLM na PagedAttention algorithm yasohotse muri UC Berkeley mumpapuro 2023 ihita ihinduka moteri ikoreshwa cyane-itanga moteri ikora.