PagedAttention da vLLM
PagedAttention dabara ce ta sarrafa ƙwaƙwalwar ajiya wacce ke adana ma'aunin hankalin samfurin harshe a cikin ƙananan tubalan da za'a iya sake amfani da su a maimakon guda ɗaya babba mai jujjuyawa.
Dubawa
It powers vLLM, an open-source serving engine that dramatically boosts how many requests a single GPU can handle.
Zurfafa nutsewa
Lokacin da samfurin harshe ya samar da rubutu, yana adana 'KV cache' (maɓalli da ƙima) ga kowane alamar da ya gani don haka alamar ta gaba zata iya zuwa ga cikakken mahallin. A al'ada kowace buƙata tana tanadin babban shinge guda ɗaya na ƙwaƙwalwar GPU mai girma don iyakar yuwuwar tsayinsa, ɓata adadi mai yawa lokacin da jeri ya kasance gajarta ko bambanta tsayi. PagedAttention, wanda aka gabatar a cikin 2023 vLLM takarda daga UC Berkeley, yana ɗaukar ra'ayin rumbun ƙwaƙwalwar ajiya daga tsarin aiki: yana raba cache na KV zuwa ƙayyadaddun ƙayyadaddun tubalan waɗanda za su iya rayuwa a ko'ina cikin ƙwaƙwalwar ajiya kuma a keɓe su akan buƙata. Taswirorin bincike na taswirori na ma'ana a matsayin alama zuwa tubalan jiki. Wannan yana kusan kawar da ɓarnawar ƙwaƙwalwar ajiya kuma yana ba da damar raba tubalan, misali a cikin abubuwan da aka samu da yawa daga sa'a ɗaya.
Fahimtar Fasaha
An raba cache na KV zuwa ƙayyadaddun shafuka masu girma, kowanne yana riƙe da maɓalli da ƙididdiga don saita adadin alamun. Toshe taswirori na kowane-jeri na taswirori na ma'ana zuwa wuraren shafi na zahiri, don haka ma'ajin jerin ba dole ba ne su kasance masu ci gaba. Saboda maƙasudi iri ɗaya (tsarin tsarin da aka raba, ko rassan bincike na katako) na iya nuni zuwa shafuka na zahiri ta hanyar kwafi-kan-rubutu, ana sake amfani da ƙwaƙwalwar ajiya maimakon kwafi, yanke sharar gida daga sama da 60% zuwa ƴan kashi.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar PagedAttention da vLLM
vLLM ya zama tsohuwar ƙashin bayan fayyace tushen tushen tushe, kuma ra'ayoyin PagedAttention yanzu suna bayyana a cikin mafi yawan tarin sabis. Yi tsammanin caching prefix mai zurfi (sake yin amfani da tsarin da aka adana yana haifar da faɗakarwa a cikin masu amfani), rarrabuwar prefill da yanke ƙididdiga akan injuna daban, ingantattun manufofin korar, da haɗin kai tare da ƙididdigewa da ƙididdige ƙididdiga. Yayin da tagogin mahallin ke girma zuwa miliyoyin alamu, ingantaccen tsarin sarrafa KV ya zama maɗaukaki don kiyaye hidima mai araha.
Aiwatar da Gaskiyar Duniya
Bayar da buɗaɗɗen tushen LLM API inda vLLM ke ba da sabis na masu amfani da hira lokaci guda daga GPU ɗaya a babban kayan aiki.
Raba dogon tsari yana faɗakarwa cikin dubban buƙatun ta hanyar caching prefix don sarrafa shi sau ɗaya, ba akai-akai ba.
Binciken bim mai gudana ko kammala samfura da yawa waɗanda ke raba shingen KV don saurin gama gari ta hanyar kwafi-kan-rubuta
Yanke sharar žwažwalwar ajiya na GPU daga rarrabuwa ta yadda mai bada zai iya tattara ƙarin zaman lokaci guda akan hardware iri ɗaya
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the PagedAttention and vLLM quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Keɓaɓɓen Sirri
Tambayoyin da ake yawan yi
What is PagedAttention and vLLM?
PagedAttention dabara ce ta sarrafa ƙwaƙwalwar ajiya wacce ke adana ma'aunin hankalin samfurin harshe a cikin ƙananan tubalan da za'a iya sake amfani da su a maimakon guda ɗaya babba mai jujjuyawa. Yana ba da iko vLLM, injin buɗe tushen sabis wanda ke haɓaka buƙatun nawa GPU ɗaya zai iya ɗauka.
Menene 'KV cache' ke adanawa yayin tsara rubutu?
Cache na KV yana riƙe da maɓalli da ƙima ga kowane alamar da ta gabata, barin ƙirar ta halarci cikakken mahallin ba tare da sake lissafinsa kowane mataki ba.
Wanne ra'ayi-tsarin aiki ya ƙarfafa PagedAttention?
PagedAttention yana aro rumbun ajiyar ƙwaƙwalwar ajiya: KV cache ya rabu zuwa ƙayyadaddun shafuka masu girman gaske waɗanda zasu iya rayuwa a ko'ina, an tsara ta taswirar tebur.
Wace matsala tare da rarraba cache na KV na gargajiya PagedAttention ya magance?
Ajiye babban katako guda ɗaya a kowane buƙatu, wanda aka ƙima don tsayin tsayi, ɓata ɗimbin adadin ƙwaƙwalwar GPU. Paging yana keɓance ƙananan tubalan akan buƙata, yanke sharar gida.
Ta yaya PagedAttention ke barin abubuwan da yawa su raba ƙwaƙwalwar ajiya don faɗakarwa gama gari?
Matsakaicin madaidaicin ƙayyadaddun ƙayyadaddun ƙayyadaddun tsokaci ko rassan binciken katako) suna nuni da shafukan KV na zahiri iri ɗaya, ana kwafi kawai lokacin da reshe ya bambanta.
A ina aka fara haɓaka vLLM da PagedAttention?
vLLM da PagedAttention algorithm sun fito daga UC Berkeley a cikin takarda na 2023 kuma cikin sauri ya zama injin hidimar buɗe tushen tushen amfani da sauri.