PagedAttention iyo vLLM
PagedAttention waa farsamada maaraynta xusuusta oo ku kaydisa kaydka dareenka moodka luqadda baloogyo yar yar oo dib loo isticmaali karo halkii ay ka ahaan lahayd qayb weyn oo iskuxiran.
Dulmar
It powers vLLM, an open-source serving engine that dramatically boosts how many requests a single GPU can handle.
quusid qoto dheer
Marka qaabka luqaddu soo saaro qoraal, wuxuu hayaa 'KV cache' (muhiimka muhiimka ah iyo qiimaha vectors) calaamad kasta oo uu arkay si calaamadda xigta ay ula socoto macnaha guud. Dhaqan ahaan codsi kastaa wuxuu hayaa hal sariirood oo weyn oo iskuxiran oo xusuusta GPU ah oo lagu cabiray dhererkeeda ugu badnaan ee suurtogalka ah, luminaya xaddi aad u badan marka taxanaha ay gaaban yihiin ama kala duwan yihiin dhererka. PagedAttention, oo lagu soo bandhigay xaashida vLLM 2023 ee UC Berkeley, ayaa ka soo amaahday fikradda xusuusta dalwaddii ee nidaamyada hawlgalka: waxay u kala qaybisaa kaydinta KV qaybo cabbir go'an oo ku noolaan kara meel kasta oo xusuusta ah oo loo qoondeeyo baahida. Shaxda la-fiirintu waxay khariidad ka dhigtaa boosaska calaamada macquulka ah ee blocks jirka. Tani waxay ku dhowdahay inay tirtirto kala qaybsanaanta xusuusta waxayna u oggolaanaysaa baloogyada in la wadaago, tusaale ahaan wax soo saar badan oo isla markiiba ah.
Aragtida Farsamada
Kaydka KV wuxuu u kala qaybsan yahay bogag cabbir go'an, mid walbana wuxuu hayaa furayaasha iyo qiimayaasha tiro go'an oo calaamado ah. Miisaska isku xigxiga ee isku xigxiga ah ayaa khariidado mansabyo macquul ah u dhigaya goobaha bogga, markaa kaydinta taxanaha uma baahna inay noqoto mid isku xigta. Sababtoo ah horgalayaasha isku midka ah (waa nidaamka la wadaago, ama laamo-raadinta isha) waxay tilmaami karaan isla bogag muuqaal ah iyada oo loo marayo nuqul-qoraal ah, xusuusta ayaa dib loo isticmaalaa halkii la nuqul ka dhigi lahaa, qashinka ka jaraya in ka badan 60% ilaa dhowr boqolkiiba.
Saamaynta Istiraatijiyadeed
Qiimaha iyo miisaaniyada
Go'aamada qaab-dhismeedku waxay horseedaan waxqabadka iyo kharashka hawlgalka sannadaha.
Go'aamo cad
Waxbarashada farsamada waxay ka caawisaa kooxaha inay doortaan xidhmo sax ah, ma aha oo kaliya kan ugu cusub.
Xakamaynta tayada
Doorashooyinka injineernimada ee wanaagsan waxay yareeyaan shilalka la isku halleyn karo ee wax soo saarka.
Mustaqbalka Paged Feejignaanta iyo vLLM
vLLM waxa ay noqotay laf-dhabarta ka-soo-jeedinta il-furan, iyo fikradaha PagedAttention hadda waxay ka soo muuqdaan inta badan xidhmooyinka adeega. Filo caching qoto dheer oo horgale ah (dib-u-isticmaalka nidaamka kaydsan ee isticmaalayaasha oo dhan), hor-u-buuxinta kala-saaray oo kala-saar mashiinno kala duwan, siyaasado ka saarid oo caqli-gal ah, iyo is-dhexgal adag oo leh tiro-koob iyo malo-awaal. Marka daaqadaha macnaha guud ay u koraan malaayiin calaamado ah, maamulka KV ee hufan ee bogag leh ayaa xitaa udub dhexaad u noqda ilaalinta u adeegida la awoodi karo.
Dhaqangelinta Adduunka-dhabta ah
Martigelinta ilo furan LLM API halkaasoo vLLM ugu adeegto isticmaaleyaal badan oo wada sheekaysi isla socda oo ka socda hal GPU oo leh wax soo saar sare.
Wadaagista nidaam dheer oo isla kumanaan codsi ah iyada oo la adeegsanayo prefix caching si loo habeeyo hal mar, maahan ku celcelin
Raadinta laydhka ordaya ama dhamaystirka muunad badan oo la wadaaga baloogyada KV isla markiiba
Ka-goynta qashinka xusuusta GPU-da ee kala-jajabinta si adeeg-bixiyuhu uu ugu xidhi karo kulamo badan oo isku mar ah isla qalab isku mid ah
Khatarta & Dariiqyada Ilaalada
Hagaajinta hal bartilmaameed waxay qarin kartaa daciifnimada nidaamka ballaaran.
Kaabayaasha dhaqaalaha iyo dayactirka inta badan waa la dhayalsadaa.
Nabadgelyada iyo daldaloolada u fiirsashada ayaa kori kara marka nidaamyadu noqdaan kuwo aad u adag.
Qorshe Hawleedka Dhaqangelinta
Qeex daahida, tayada, iyo bartilmaameedyada qiimaha ka hor inta aan la hirgelin.
Benchmark marka la eego culeyska dhabta ah iyo xaaladaha xogta.
La socodka qalabka khaladaadka, leexashada, iyo saamaynta isticmaalaha.
U diyaari dib-u-noqoshada iyo dariiqyada jawaab-celinta dhacdada ka hor inta aanad miisaan.
Sii wad Sahaminta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the PagedAttention and vLLM quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Hagaha xiga
Asturnaanta Kala Duwan
Su'aalaha soo noqnoqda
What is PagedAttention and vLLM?
PagedAttention waa farsamada maaraynta xusuusta oo ku kaydisa kaydka dareenka moodka luqadda baloogyo yar yar oo dib loo isticmaali karo halkii ay ka ahaan lahayd qayb weyn oo iskuxiran. Waxay awood u siinaysaa vLLM, mishiin u adeega il furan oo si wayn u kordhiya inta codsi ee hal GPU uu qaadi karo.
Muxuu kaydiyaa 'KV cache' inta lagu jiro qoraalka qoraalka?
Kaydka KV waxa uu hayaa furaha iyo qiyamka vectors calaamad kasta oo hore, taas oo u oggolaanaysa moodalku inuu si buuxda u eego macnaha guud iyada oo aan dib loo xisaabin tallaabo kasta.
Waa maxay fikradda nidaamka hawlgalka dhiirigeliyay PagedAttention?
PagedAttention waxay soo amaahataa bogag xusuusta dalwaddii ah: khasnadda KV waxay u qaybsantaa bogag cabbir go'an oo meel kasta ku noolaan kara, oo lagu sawiray miis block ah.
Dhibaato noocee ah ayaa ku jirta qoondaynta kaydka KV ee dhaqameed ee PagedAttention xalliyo?
Kaydinta hal sariirood oo weyn codsi kasta, oo lagu qiyaaso dhererka ugu badan, waxay lumisay xaddi badan oo xusuusta GPU ah. Boggu wuxuu qoondeeyaa baloogyo yar yar marka loo baahdo, qashinka qashinka.
Sidee PagedAttention u ogolaataa soosaarayaal badan inay wadaagaan xusuusta degdeg guud?
Horgalayaasha isku midka ah (dardaarannada la wadaago ama laamaha raadinta isha) waxay tixraacayaan isla boggaga KV ee jireed, koobiyeynta kaliya marka laantu kala baxdo.
Halkee laga sameeyay vLLM iyo PagedAttention markii hore?
vLLM iyo PagedAttention algorithm waxay ka soo baxeen UC Berkeley warqad 2023 waxayna si dhakhso ah u noqdeen matoorka adeega il furan oo si weyn loo isticmaalo.