Ukuqopha Okuqagelwayo nge-EAGLE
Ukukhipha amakhodi okuqagela kusheshisa ukuchazwa kwemodeli enkulu yolimi ngokuvumela imodeli encane engakapheli iqagele amathokheni ambalwa ngaphambili, imodeli enkulu bese iwaqinisekisa ngephasi eyodwa.
Uhlolojikelele
EAGLE is a state-of-the-art version that drafts at the feature level rather than the token level, delivering 2-4x speedups with zero loss in output quality.
I-Deep Dive
Isizukulwane se-LLM esivamile si-autoregressive: imodeli ikhiqiza ithokheni eyodwa, iphinde iwunike ukudla, futhi iphinda, ngakho-ke ithokheni ngayinye idinga ukudlula okugcwele phambili ngezigidigidi zamapharamitha. Ukukhipha amakhodi okuqagela kwephula lo mgomo. Umhleli oshibhile uphakamisa ingxenye yamathokheni ekhandidethi, futhi imodeli ebizayo eqondiwe iwaqinisekisa wonke ngephasi eyodwa ehambisanayo, yamukele isiqalo esilungile eside kunazo zonke. UKHOZI (I-Extrapolation Algorithm Yempumelelo Yemodeli Yolimi Olukhulu) ithuthuka ezindleleni zangaphambili ngokubhala esikhaleni sesici esifihliwe semodeli futhi ibuyisele ukushumeka kwangempela kwethokheni yangaphambili ukuze kuncishiswe ukungaqiniseki. I-EAGLE-2 yengeza isihlahla esisalungiswa esiguqukayo, futhi UKHOZI-3 lwehlisa isici sokubikezela isici ukuze sikhule kangcono. Okubaluleke kakhulu, ukuqinisekiswa kuqinisekisa ukuthi okukhiphayo kuyefana nalokho imodeli eqondiwe ebingakukhiqiza iyodwa.
I-Technical Insight
UKHOZI luqeqesha inhloko encane ezenzakalelayo ebikezela isici sesimo esilandelayo semodeli eqondiwe, bese iphinda isebenzise ikhanda le-LM eliqondiwe ukuze liguqule izici zibe amakhandidethi amathokheni. Ngokumisa ukulandelana kwethokheni eguquliwe kanye nezici zangaphambili, kunciphisa ukungaqondakali okuhlasele ukubhalwa kwesici kuphela. Isihlahla sabantu abazongenela ukhetho siqinisekiswa kanye nje; ukusatshalaliswa kwemodeli eqondiwe kugcinwa kahle ngoba amathokheni amukelwayo kufanele afane nesampuli yawo noma ukukhetha kwe-argmax, okwenza i-speedup ingalahleki.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa Lokuqagela Okuqagelayo noKHOZI
Ukukhipha amakhodi okuqagela kuba yingqalasizinda ezenzakalelayo ekunikezeni izitaki ezifana ne-vLLM ne-TensorRT-LLM. Lindela ukuhlanganiswa okuqinile nge-batching kanye nokwabelana kwenqolobane ye-KV, amamodeli azidwebela wona angadingi uhlaka oluhlukile, kanye ne-hardware co-design ethatha ukuqinisekiswa okufanayo. Ukwakhiwa kwesici sesitayela sokhozi kunwetshwa kumamodeli ezinto eziningi kanye nokucabanga, lapho amaketango amade omcabango enza izindleko zethokheni ngayinye zibe buhlungu kakhulu, kanye nokucabanga okukudivayisi lapho ukubambezeleka kubaluleke kakhulu.
Ukuqaliswa Komhlaba Wangempela
Ukunciphisa ukubambezeleka kubasizi bengxoxo ukuze izimpendulo zisakaze ngokushesha izikhathi ezingu-2-3 ngaphandle kokushintsha izimpendulo zemodeli
Ukunciphisa izindleko zokusebenzisa i-GPU kubahlinzeki bevolumu ephezulu ye-API ngokukhiqiza amathokheni engeziwe ngokudlula phambili ngakunye
Ukusheshisa amamodeli okucabanga ochungechunge olude lapho kukhiqizwa izinkulungwane zamathokheni ngombuzo ngamunye
Ukusheshisa amathuluzi okugcwalisa ikhodi lapho ukulandelana kwamathokheni okubikezelwayo, okuphindaphindwayo kukhiqiza amanani aphezulu okuvuma okusalungiswa
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding with EAGLE quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
Ukuqopha Okucatshangelwayo
Imibuzo evame ukubuzwa
What is Speculative Decoding with EAGLE?
Ukukhipha amakhodi okuqagela kusheshisa ukuchazwa kwemodeli enkulu yolimi ngokuvumela imodeli encane engakapheli iqagele amathokheni ambalwa ngaphambili, imodeli enkulu bese iwaqinisekisa ngephasi eyodwa. I-EAGLE inguqulo yesimanjemanje ebhala ezingeni lesici esikhundleni seleveli yamathokheni, iletha ama-speedups angu-2-4x ngokulahleka okuyiziro kukhwalithi yokuphumayo.
Uyini umqondo oyinhloko ngemuva kokuqanjwa kokuqagela?
Umhleli omncane uqagela amathokheni ambalwa ngaphambili, futhi imodeli enkulu eqondiwe iwaqinisekisa ndawonye, amukele isiqalo esilungile eside kunazo zonke.
I-EAGLE ihluke kanjani ezindleleni zangaphambili zokuqagela zokuqopha?
UKHOZI lubikezela izici ezifihliwe zemodeli eqondiwe futhi luphinde lusebenzise ikhanda layo le-LM, elikhiqiza okusalungiswa okunembe kakhudlwana kunezindlela zamathokheni kuphela.
Kungani ukuqopha okucatshangelwayo kuthathwa 'njengokungalahleki'?
Ngenxa yokuthi imodeli eqondiwe ihlola ithokheni ngayinye esalungiswa ngokusatshalaliswa kwayo, okukhiphayo okwamukelwe yilokho kanye okuhlosiwe obekungakukhiqiza kukodwa.
Iyiphi inkinga ekudwetshweni kwesici se-EAGLE ukusiza ukubuyisela ukushumeka kwethokheni yangaphambilini ukuyixazulula?
Ukubeka isimo kuthokheni yangempela yangaphambilini kunciphisa ukungaqondakali kokubikezela isici esilandelayo esifihliwe, kuthuthukisa ukunemba kokusalungiswa.
I-EAGLE-2 yengeze ini ngaphezu koKHOZI lwangempela?
I-EAGLE-2 yethule isihlahla esisalungiswa esiguqukayo esiqaphela umongo, sinweba amagatsha athembisayo ukuze kwenyuse izinga lokwamukelwa.