Ubuyobozi bwa tekiniki

Urupapuro Icyitonderwa na vLLM

PagedAttention manages transformer key/value caches using fixed-size blocks mapped from logical token positions.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kazoza Kurupapuro Icyitonderwa na vLLM
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

This reduces cache-allocation waste and supports sharing, while vLLM’s actual throughput depends on the workload and system.

Kwibira cyane

Iyo imvugo yindimi itanga inyandiko, igumana 'KV cache' (urufunguzo nagaciro kerekana agaciro) kuri buri kimenyetso yabonye kugirango ikimenyetso gikurikira gishobora kwitabira imiterere yuzuye. Ubusanzwe buri cyifuzo cyabitswe icyapa kinini gihuza ububiko bwa GPU gifite ubunini buringaniye bushoboka, butakaza amafaranga menshi mugihe ibyakurikiranye byari bigufi cyangwa bitandukanye muburebure. Urupapuro rwitonderwa, rwatangijwe mu mpapuro za 2023 vLLM zivuye muri UC Berkeley, rutira igitekerezo cyo kwibuka ububiko bwa paji kuva muri sisitemu y'imikorere: igabanya cache cache mubice binini byagenwe bishobora gutura ahantu hose mububiko kandi bigatangwa kubisabwa. Imbonerahamwe yerekana ikarita yerekana ibimenyetso byerekana ibimenyetso bifatika. Ibi hafi gukuraho ibice byo kwibuka hanyuma bikareka guhagarika bigasangirwa, kurugero rwibisubizo byinshi bivuye kumurongo umwe.

Gushyira mu bikorwa Isi

Kwakira isoko-ifunguye LLM API aho vLLM ikorera abakoresha ibiganiro byinshi bahurira kuri GPU imwe murwego rwo hejuru

Kugabana sisitemu ndende ihita ibihumbi n'ibisabwa ukoresheje prefix cishing kuburyo itunganywa rimwe, ntabwo inshuro nyinshi

Gukoresha ibiti bishakisha cyangwa ibyitegererezo byinshi byuzuzanya bisangiye KV kubisanzwe mubisanzwe ukoresheje kopi-yandika

Gukata imyanda ya GPU ivuye mu gucamo ibice kugirango uyitanga ashobora gupakira icyarimwe icyarimwe kubikoresho bimwe

Kazoza Kurupapuro Icyitonderwa na vLLM

vLLM yahindutse umwirondoro ufunguye-inkomoko yumugongo, kandi ibitekerezo bya PagedIbyitonderwa ubu bigaragara mubice byinshi bikora. Witegereze byimbitse byimbitse (kongera gukoresha sisitemu ya cache yerekana kubakoresha), prefill itandukanijwe hamwe na decode kumashini zitandukanye, politiki yo kwirukana ubwenge, hamwe no guhuza hamwe no kubara no gushushanya. Mugihe imiterere ya Windows ikura miriyoni yikimenyetso, imiyoborere ya paje ya KV ikora neza cyane kugirango ikomeze serivisi ihendutse.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the PagedAttention and vLLM quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Niki Urupapuro rwitonderwa na vLLM?

PagedAttention manages transformer key/value caches using fixed-size blocks mapped from logical token positions. This reduces cache-allocation waste and supports sharing, while vLLM’s actual throughput depends on the workload and system.

Niki 'KV cache' ibika mugihe cyo kwandika inyandiko?

Ubwihisho bwa KV bufite urufunguzo nagaciro byagaciro kuri buri kimenyetso kibanziriza iki, kureka icyitegererezo kikitabira imiterere yuzuye utabanje kubara buri ntambwe.

Ni ubuhe buryo bwa sisitemu y'imikorere yahumekeye Paged?

Urupapuro rwitondewe ruguza ububiko bwibikoresho bya paji: KV cache igabanijwemo impapuro zingana-zishobora gutura ahantu hose, zashushanijwe nimbonerahamwe.

Ni ikihe kibazo kijyanye no kugabura kwa KV gakondo PagedIbyifuzo bikemura?

Kubika icyapa kinini gihuza icyifuzo, gifite ubunini burebure, cyatakaje umubare munini wibikoresho bya GPU. Paji itanga uduce duto kubisabwa, gutema imyanda.

Nigute PagedIbyitonderwa bireka ibisubizo byinshi bigabana kwibuka kubintu bisanzwe?

Imbanzirizamushinga imwe (ibisobanuro bisangiwe cyangwa amashami ashakisha amashami) yerekana page imwe ya KV yumubiri, ikopera gusa iyo ishami ritandukanye.

VLLM na PagedIbyifuzo byatejwe imbere he?

vLLM na PagedAttention algorithm yasohotse muri UC Berkeley mumpapuro 2023 ihita ihinduka moteri ikoreshwa cyane-itanga moteri ikora.