Ukuqopha Okucatshangelwayo
Ukukhipha amakhodi okuqagela kwenza amamodeli wolimi amakhulu akhiqize umbhalo ngokushesha ngokusebenzisa imodeli 'esalungiswa' encane, esheshayo ukuqagela amathokheni ambalwa ngaphambili, bese kuthi imodeli enkulu iwaqinisekise wonke ngesikhathi esisodwa.
Uhlolojikelele
It speeds up inference 2-3x with identical output quality.
I-Deep Dive
Ngokuvamile i-LLM ikhiqiza umbhalo wethokheni eyodwa ngesikhathi: ithokheni ngayinye idinga ukudlula okugcwele ukuya phambili ngemodeli enkulukazi, futhi awukwazi ukuqala elandelayo kuze kuphele eyamanje. Lokhu kuhamba kancane ngoba iboshelwe kwinkumbulo, ayibophezelekile kukhompuyutha - i-GPU ichitha isikhathi sayo esiningi ilayisha izisindo, ingenzi izibalo. Ukukhipha amakhodi okuqagela kwephula umgoqo. Imodeli encane, eshibhile yokusalungiswa iphakamisa isiqephu, masithi, amathokheni ekhandidethi amahlanu. Imodeli enkulu 'yethagethi' ibe isicubungula zonke ezinhlanu ngendlela yokudlula eyodwa ehambisanayo bese iyawahlola. Amathokheni ahambisana nalokho ebingakuveza ayamukelwa; ekungavumelanini kokuqala liyalungisa bese lilahla okunye. Ngenxa yokuthi ukuqinisekisa amathokheni amaningi kubiza cishe ngokufana nokukhiqiza eyodwa, ukuqagela okwamukelwe cishe kumahhala.
I-Technical Insight
Ingxenye ehlakaniphile umthetho wokunqatshelwa kwesampula oqinisekisa ukuthi ukusabalalisa okukhiphayo kuyafana ngezibalo nokusebenzisa imodeli eqondiwe iyodwa — ngakho ikhwalithi ayilinganiselwa, inembile. Izinga lokwamukela liqhuba ukusheshisa: ngcono imodeli encane ibikezela enkulu, amathokheni amaningi anamathela esinyathelweni ngasinye sokuqinisekisa. Izinhlobonhlobo ezifana ne-Medusa zengeza amakhanda okubikezela engeziwe kumodeli eqondiwe ngokwayo, kanye nokusalungiswa kwe-EAGLE endaweni yesici, kususe isidingo semodeli ehlukile yokusalungiswa.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa Lokuqagela Okuqagelayo
Ukukhipha amakhodi okuqagela kuba okuzenzakalelayo ekunikezeni izitaki ezifana ne-vLLM ne-TensorRT-LLM. Lindela izindlela zokuzibhala ngokwakho (i-Medusa, UKHOZI, i-Lookahead) ukuthi zilawule njengoba zigwema ukugcina imodeli yesibili, kanye nokuqagela okusekelwe esihlahleni okuqinisekisa amagatsha amaningi ekhandidethi ngesinyathelo ngasinye. Njengoba amamodeli ekhula, i-bottleneck eboshelwe inkumbulo iba yimbi, okwenza ukuqagela kube yigugu nakakhulu, futhi abasalungiswa abaqaphela ihadiwe bazocindezela ukushesha komhlaba wangempela phezulu.
Ukuqaliswa Komhlaba Wangempela
Imodeli yokusalungiswa ye-7B ephakamisa amathokheni emodeli yengxoxo engu-70B ukuze kunqanyulwe ukubambezeleka kwempendulo kumsizi wokukhiqiza.
Amakhanda e-Medusa aboshelwe ku-LLM ngakho-ke ibikezela amathokheni ambalwa esikhathi esizayo ngesikhathi esisodwa ngaphandle kwemodeli yokusalungiswa ehlukile.
I-vLLM inika amandla ukuqoshwa okucatshangelwayo ukuze kukhuphule ukuphuma kwamathokheni ngesekhondi ngalinye kuqoqo lokuphakelayo
UKHOZI lubhala esikhaleni sesici esifihliwe semodeli ukuze kukhuliswe izinga lokwamukelwa kanye nesivinini sisonke
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukuqopha Okuqagelwayo nge-EAGLE
Imibuzo evame ukubuzwa
What is Speculative Decoding?
Ukukhipha amakhodi okuqagela kwenza amamodeli wolimi amakhulu akhiqize umbhalo ngokushesha ngokusebenzisa imodeli 'esalungiswa' encane, esheshayo ukuqagela amathokheni ambalwa ngaphambili, bese kuthi imodeli enkulu iwaqinisekise wonke ngesikhathi esisodwa. Isheshisa ukubikezela okungu-2-3x ngekhwalithi efanayo yokukhipha.
Uyini umqondo owumongo wokuqagela okuqagelayo?
Imodeli yokusalungiswa esheshayo iphakamisa amathokheni amaningi, futhi imodeli enkulu eqondiwe iwahlola wonke ngephasi eyodwa ehambisanayo.
Kungani isizukulwane se-LLM esivamile se-token-at-a-time sihamba kancane?
Isinyathelo ngasinye silayisha ikakhulukazi ama-matrices esisindo esikhulu ngekhanda kunokwenza ukubala okukhulu, ngakho i-GPU isetshenziswa kancane.
Kwenzekani ekuqaleni lapho okusalungiswa namamodeli okuhlosiwe angavumelani khona?
Amathokheni afinyelela ekungavumelanini ayamukelwa; kusukela ekungafanini kuye phambili ziyanqatshwa futhi ithokheni elungile yemodeli eqondiwe iyagcinwa.
Ukukhipha amakhodi okuqagela kuyithinta kanjani ikhwalithi yokuphumayo?
Umthetho wokunqatshelwa kwesampula uqinisekisa ukusatshalaliswa kokugcina kufana ncamashi nemodeli eqondiwe, ngakho ikhwalithi ayishintshile.
Yini enquma ngokuqondile ukusheshisa kusukela ekuqoqweni okucatshangelwayo?
Uma amathokheni asalungiswa kakhulu imodeli eqondiwe yamukela isinyathelo ngasinye sokuqinisekisa, kukhulu ukusheshisa.