Kwenzekeni
Abacwaningi bahlole izindlela ezimbili zokukhulula ingcindezi yenkumbulo lapho kusetshenziswa amamodeli olimi amakhulu: ukusabalalisa izisindo kanye nenqolobane ye-KV kuma-GPU amaningi, noma ukucindezela nokukhipha inqolobane ye-KV ngokukhetha. Besebenzisa isifanisi esinephrofayili esilinganiselwe ku-A100, A40, ne-H100 hardware, baqhathanisa izindleko ngesigidi samathokheni nokubambezeleka kuwo wonke amamodeli we-Llama-2 kumapharamitha angu-7B no-70B.
Iphepha lihlola ibhodlela elithile kumodeli yolimi olukhulu: indawo enganele yenani elingukhiye, noma i-KV, inqolobane. Ngokusho kwababhali, i-tensor parallelism ibhekana nenkinga ngokuhlukanisa izisindo zemodeli kanye nenqolobane ye-KV kumadivayisi amabili, amane noma ayisishiyagalombili. Leyo ndlela idala i-headroom eyengeziwe kodwa idinga ukusebenza kokunciphisa konke kuzo zonke izendlalelo futhi ikhulise ibhili yehadiwe njengoba inani lamadivayisi likhula.
Okunye okufundwe ukunciphisa inqolobane ngokwayo. Ababhali bahlola ubuningi be-KV kumazinga angu-16-, 8-, kanye no-4-bit, kanye nezilinganiso zokugcina zehle ziye ku-0.25, okusho ukuthi ukulungiselelwa okuhloliwe kugcina izingxenyana ezihlukene zenqolobane. Iphepha libeka womabili amasu ku-eksisi yezindleko ezabiwe: izindleko ngesigidi samathokheni uma kuqhathaniswa ne-latency, kunokuqhathanisa izilinganiso zememori namajika okukhipha ngokuhlukana.
Ukuhlaziywa kusebenzisa isifanisi esinephrofayili esilinganiselwe ku-A100, A40, ne-H100 hardware. Ihlanganisa izinhlobo ze-Llama-2 ezinamapharamitha ayizigidi eziyizinkulungwane ezingu-7 nezingu-70, amadigri ama-tensor-parallel ukusuka kweyodwa kuya kweyisishiyagalombili, kanye nezilungiselelo zokucindezela ezihloliwe. Ababhali babika ukuthi abatholanga i-crossover yokulingana kwezindleko kulezo zivivinyo: ukucindezela bekuphakathi kwe-1.20 ne-2.00 izikhathi ezishibhile kukho konke ukucupha abakwakhile.
Umngcele obikiwe uncike ebudlelwaneni phakathi kosayizi wemodeli nememori yedivayisi. Ngedivayisi engu-80 GB, ababhali bathi imodeli ye-7B ayikwazi ukusebenzisa isabelomali sayo se-KV ngaphakathi kwewindi lokuqukethwe kwayo, kuyilapho umngcele wesinqumo uvela cishe kumapharamitha angu-36B. Ngaphansi kwalelo phuzu, iphepha libika ukuthi ukucindezela kubusa ngokwezomnotho. Ngaphezulu kwakho, ukufana kwe-tensor kubhekana nomkhawulo omkhulu lapho izisindo zemodeli zingakwazi ukulingana kudivayisi eyodwa; ababhali banikeza i-Llama-2 70B ku-A100 eyodwa njengesibonelo esihlala singenakwenzeka kungakhathaliseki ukulungiselelwa kwe-KV.
Imininingwane yomthombo: arxiv.org ↗
Kungani kubalulekile
Iphepha lifaka uhlaka olusebenzayo lwengqalasizinda yabasebenzisi be-AI. Imiphumela yakhona iphakamisa ukuthi ukucindezela kunganikeza umthamo wokuphakela owengeziwe ngedola ngalinye kumamodeli alingana nezisindo zawo kudivayisi eyodwa, kuyilapho ukufana kwe-tensor kuba kudingekile lapho izisindo zemodeli ngokwazo zeqa inkumbulo etholakalayo. I-tradeoff iwukubambezeleka okuphezulu ngaphansi kokucindezelwa ekusetheni okuhloliwe.
Inani elingokoqobo lephepha ukuthi liqhathanisa izinqumo ezimbili zengqalasizinda okuvame ukuxoxwa ngazo kusetshenziswa izinyathelo ezihlukene. Ithimba elinquma ukuthi liyisebenzela kanjani i-LLM kufanele lilinganise amandla enkumbulo, ukubambezeleka, kanye nezindleko. Ngokuveza ezinye izindlela njengezindleko zamathokheni ayisigidi ngokumelene nokubambezeleka, ucwaningo lunikeza uhlaka oluvamile lokucabanga ngaleso sinqumo, nakuba imiphumela ihlala ingeyokucushwa kwephepha okumodeliwe kanye nephrofayili.
Inzuzo yezomnotho ebikiwe iqine kakhulu kumamodeli anesisindo esivele silingana kudivayisi eyodwa. Kuleso simo, ukuncipha inqolobane ye-KV kungakhuphula inani lomsebenzi osingathwa yi-hardware etholakalayo ngaphandle kokudinga iqoqo elikhulu le-GPU. Ababhali babika ukuphindaphinda kwe-16.5-power-dollar ngayinye yokucindezelwa, uma kuqhathaniswa nokuphindwe ka-1.21 ngokunyuka okuphindwe kasishiyagalombili kwezindleko ze-GPU. Lena imiphumela ebikiwe yephepha, akusona isiqinisekiso esijwayelekile sakho konke ukuthunyelwa.
Ukubambezeleka kuyizindleko ezimaphakathi zesu lokuminyanisa ocwaningweni. Ababhali babika ukuthi ukucindezela kukhuphuke nge-token latency ngo-8% kuya ku-93%, abathi kungenxa yombango we-batching. I-tensor parallelism yiyona kuphela i-lever ehloliwe eyathuthukisa ukubambezeleka. Lokhu kudala i-tradeoff ecacile yokusebenza: ukucindezela kungase kube ngcono lapho ukudlula noma umthamo wedola ngalinye kubaluleke kakhulu, kuyilapho ama-GPU engeziwe angase athethelelwe lapho isikhathi sokuphendula siyisidingo esiyinhloko.
Leli phepha liphinde licacise ukuthi kungani umthetho owodwa wakho konke ukuthunyelwa kwe-LLM ungadukisa. Ukucindezelwa kwe-KV akunciphisi inkumbulo ehlala izisindo zemodeli, ngakho-ke akukwazi ukwenza imodeli enkulukazi ilingane kudivayisi eyodwa. Ngokuphambene, ukungeza ama-GPU kungabhekana nesisindo kodwa kungase kudlulele kumodeli encane inkinga yayo eyinhloko iwukusetshenziswa kwenqolobane. Ngakho-ke umnikelo wocwaningo uyisincomo esinemibandela esiboshelwe kunsiza yenkumbulo ebophayo, kunesimangalo sokuthi indlela eyodwa ihlala ithatha indawo enye.
I-Interactive Mechanism: Indlela Esebenza Ngayo Ngempela
Hlola ubuchwepheshe obuyisisekelo ngemuva kwalokhu kuthuthukiswa ngokuhlanganyela.
Which component of an AI application is the machine-learning model itself?
Ongakubuka ngokulandelayo
Okutholiwe kudinga ukuhlolwa ngokumelene nomthwalo wokukhiqiza, imindeni yamamodeli, izidingo zekhwalithi, nezintengo zangempela zamafu noma zehadiwe. I-abstract ayibiki imiphumela enembayo enemininingwane evela ekwakhiweni kwenani noma ekukhishweni, imiphumela yokuqinisekisa yesifanisi, noma ukuthi umngcele obikiwe we-crossover ubambe ngale kwamamodeli ahloliwe we-Llama-2 nezinhlobo ze-GPU.
I-abstract ayinikezi izintengo zamafu eziyisisekelo, ukuqagela kokusetshenziswa kwezingxenyekazi zekhompuyutha, ingxube yomthwalo womsebenzi, osayizi beqoqo, noma okuqondiwe kokubambezeleka okusetshenziselwa ukubala izindleko ngesigidi samathokheni. Leyo mininingwane izonquma ukuthi opharetha bangakwazi kanjani ukuhumusha kalula izilinganiso ezibikiwe zibe yisabelomali sabo sokusebenza. Umphumela osuselwe kuphrofayela eyodwa yentengo noma yokusetshenziswa ungase ushintshe lapho lokho okokufaka kushintsha.
Izindleko zekhwalithi zokucindezelwa nazo azicaciswanga ngokugcwele emthonjeni. Leli phepha lichaza ukulinganisa nokukhishwa kwabantu njengokuchitha “ikhwalithi encane,” kodwa i-abstract ayinikezi amamethrikhi ekhwalithi elinganisiwe, imisebenzi, ububanzi bokwehliswa kwesithunzi, noma imikhawulo esetshenziselwa ukunquma ukuthi ukulungiselelwa kwamukelekile yini. Amaqembu okukhiqiza azodinga lolo lwazi ngaphambi kokuphatha imiphumela yezindleko njengesincomo sokugcina.
Ucwaningo lukhawulelwe emthonjeni osayizi ababili be-Llama-2 nezinhlobo ezintathu ze-GPU, nakuba luveza umngcele wesinqumo ovamile cishe kumapharamitha angu-36B wekhadi elingu-80 GB. I-abstract ayiqinisekisi ukuthi lowo mngcele uyaqhubeka yini kwezinye izakhiwo, ubude bomongo, imiklamo yokunaka, imindeni yamamodeli, noma izingxenyekazi zekhompuyutha ezintsha. Futhi ayisho ukuthi iziphetho zishintsha kanjani lapho izisindo zemodeli zilinganiswa noma lapho amasistimu enikezela esebenzisa amanye amasu okuphatha inkumbulo.
Ukuqinisekisa okwengeziwe kufanele kugxile ekusetshenzisweni kwangempela nasekulinganisweni kwesifanisi. Umthombo ukhomba isifanisi njengephrofayili ngokumelene nehadiwe ye-A100, A40, ne-H100, kodwa i-abstract ayibiki ukuqinisekiswa okuzimele ngokumelene nezilinganiso zokupha bukhoma. Kuphinde kushiye kuvuliwe ukuthi ukuminyanisa nokufana kwe-tensor kusebenza kanjani uma kuhlangene, noma ngabe ukujeziswa kwe-latency kuyehluka ngamaphethini wethrafikhi, nokuthi abasebenzisi bekhwalithi engakanani abangahweba ngezinzuzo zamandla ezibikiwe.