Gukoresha Decoding hamwe na EAGLE
Gukoresha decoding yihutisha imvugo nini yerekana imvugo ureka akantu gato kerekana icyitegererezo ukeka ibimenyetso byinshi imbere, iyo moderi nini noneho ikagenzura muri pass imwe.
Incamake
EAGLE is a state-of-the-art version that drafts at the feature level rather than the token level, delivering 2-4x speedups with zero loss in output quality.
Kwibira cyane
Igisekuru gisanzwe cya LLM ni autoregressive: icyitegererezo gitanga ikimenyetso kimwe, kikagaburira inyuma, kandi kigasubiramo, buri kimenyetso rero gisaba kunyura imbere yuzuye binyuze muri miliyari y'ibipimo. Gukoresha decoding idasanzwe bimena iki cyuho. Igishushanyo gihenze gitanga igitekerezo cyabakandida ibimenyetso, kandi intego ihenze igenzura byose hamwe mumurongo umwe ugereranije, ukemera prefix ndende ikosora. EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) itezimbere muburyo bwambere mugutegura ahantu hihishe imiterere yikitegererezo no kugaburira ibimenyetso byabanjirije gushira kugirango ugabanye gushidikanya. EAGLE-2 yongeramo igiti cyingirakamaro, kandi EAGLE-3 igabanya imiterere-yo guhanura kugirango igere neza. Byibanze, verisiyo yemeza ko ibisohoka bisa nibyo icyitegererezo cyaba cyarabyaye wenyine.
Ubushishozi
EAGLE itoza umutwe muto wigenga uteganya icyerekezo cyicyerekezo gikurikiraho cyihishe-leta, hanyuma ukongera gukoresha umutwe bwite wa LM kugirango uhindure ibiranga abakandida ba token. Mugutondekanya kumurongo wahinduwe wikimenyetso wongeyeho ibintu byabanje, bigabanya ibidasobanutse byugarije imiterere-gusa. Igiti cy'abakandida kigenzurwa icyarimwe; intego yicyitegererezo cyo gukwirakwiza irabitswe neza kuko ibimenyetso byemewe bigomba guhuza icyitegererezo cyayo cyangwa argmax, bigatuma umuvuduko utagira igihombo.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Igihe kizaza cyo gushushanya hamwe na EAGLE
Gukoresha decoding yibikorwa birahinduka ibikorwa remezo mugutanga ibice nka vLLM na TensorRT-LLM. Tegereza kwishyira hamwe gukomeye hamwe no gusangira na KV-cache kugabana, kwishushanya-moderi idakeneye uwashushanyije, hamwe nibyuma bifatanyiriza hamwe bifata ibyemezo bisa. Imiterere ya EAGLE yerekana uburyo bwateguwe bugera kuri moderi nyinshi kandi zitekereza, aho iminyururu miremire yibitekerezo ituma ikiguzi kuri buri kimenyetso kibabaza cyane, kandi no kumurongo wibikoresho aho ubukererwe bufite akamaro kanini.
Gushyira mu bikorwa Isi
Gukata ubukererwe mubafasha kuganira kuburyo ibisubizo bitemba 2-3x byihuse udahinduye ibisubizo byicyitegererezo
Kugabanya ibiciro bya GPU kubiciro byinshi byabatanga API mugutanga ibimenyetso byinshi kuri pass imbere
Kwihutisha urunigi rurerure-rwibitekerezo byerekana ibitekerezo aho ibihumbi nibimenyetso bikorerwa kuri buri kibazo
Kwihutisha ibikoresho-kurangiza ibikoresho aho byateganijwe, gusubiramo ibimenyetso bikurikirana bitanga umushinga wo hejuru-wo kwemerwa
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding with EAGLE quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Kugereranya
Ibibazo bikunze kubazwa
What is Speculative Decoding with EAGLE?
Gukoresha decoding yihutisha imvugo nini yerekana imvugo ureka akantu gato kerekana icyitegererezo ukeka ibimenyetso byinshi imbere, iyo moderi nini noneho ikagenzura muri pass imwe. EAGLE nuburyo bugezweho bwa verisiyo ishushanya kurwego rwibiranga aho kuba urwego rwikimenyetso, itanga umuvuduko wa 2-4x hamwe nigihombo cya zeru mubisohoka.
Ni ikihe gitekerezo cyibanze inyuma ya decoding?
Gitoya ishushanya ikeka ibimenyetso byinshi imbere, kandi intego nini yerekana igenzura hamwe, ikemera prefix ndende ikosora.
Nigute EAGLE itandukaniye nuburyo bwambere bwo gushushanya decoding?
EAGLE ihanura intego yicyitegererezo yibintu byihishe kandi ikoresha umutwe wa LM, itanga ibishushanyo mbonera kuruta uburyo bwa token gusa.
Ni ukubera iki gushishoza bifatwa nk '' igihombo '?
Kuberako intego yicyitegererezo igenzura buri cyapa cyateguwe kirwanya kugabana kwayo, ibisohoka byemewe nibyo rwose intego yaba yarabyaye wenyine.
Ni ikihe kibazo mubikorwa byateguwe na EAGLE kugaburira ibimenyetso byabanje gushira bifasha gukemura?
Gutondekanya ku kimenyetso kibanziriza iki kigabanya kudasobanuka guhanura ibikurikira byihishe, kunoza umushinga wukuri.
Niki EAGLE-2 yongeyeho kuri EAGLE yumwimerere?
EAGLE-2 yashyizeho imiterere-yimikorere yingirakamaro igiti, yagura amashami yizeza kuzamura igipimo cyo kwemerwa.