Ubuyobozi bwa tekiniki

Gukoresha Decoding hamwe na EAGLE

Gukoresha decoding yihutisha imvugo nini yerekana imvugo ureka akantu gato kerekana icyitegererezo ukeka ibimenyetso byinshi imbere, iyo moderi nini noneho ikagenzura muri pass imwe.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Igihe kizaza cyo gushushanya hamwe na EAGLE
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

EAGLE ni verisiyo igezweho ikora inyandiko ku rwego rw'ibiranga ikintu (feature level) aho kuba ku rwego rw'ikimenyetso (token level), itanga umuvuduko wihuse inshuro 2-4 nta gihombo na gito mu bwiza bw'ibyasohotse.

Kwibira cyane

Igisekuru gisanzwe cya LLM ni autoregressive: icyitegererezo gitanga ikimenyetso kimwe, kikagaburira inyuma, kandi kigasubiramo, buri kimenyetso rero gisaba kunyura imbere yuzuye binyuze muri miliyari y'ibipimo. Gukoresha decoding idasanzwe bimena iki cyuho. Igishushanyo gihenze gitanga igitekerezo cyabakandida ibimenyetso, kandi intego ihenze igenzura byose hamwe mumurongo umwe ugereranije, ukemera prefix ndende ikosora. EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) itezimbere muburyo bwambere mugutegura ahantu hihishe imiterere yikitegererezo no kugaburira ibimenyetso byabanjirije gushira kugirango ugabanye gushidikanya. EAGLE-2 yongeramo igiti cyingirakamaro, kandi EAGLE-3 igabanya imiterere-yo guhanura kugirango igere neza. Byibanze, verisiyo yemeza ko ibisohoka bisa nibyo icyitegererezo cyaba cyarabyaye wenyine.

Gushyira mu bikorwa Isi

Gukata ubukererwe mubafasha kuganira kuburyo ibisubizo bitemba 2-3x byihuse udahinduye ibisubizo byicyitegererezo

Kugabanya ibiciro bya GPU kubiciro byinshi byabatanga API mugutanga ibimenyetso byinshi kuri pass imbere

Kwihutisha urunigi rurerure-rwibitekerezo byerekana ibitekerezo aho ibihumbi nibimenyetso bikorerwa kuri buri kibazo

Kwihutisha ibikoresho-kurangiza ibikoresho aho byateganijwe, gusubiramo ibimenyetso bikurikirana bitanga umushinga wo hejuru-wo kwemerwa

Igihe kizaza cyo gushushanya hamwe na EAGLE

Gukoresha decoding yibikorwa birahinduka ibikorwa remezo mugutanga ibice nka vLLM na TensorRT-LLM. Tegereza kwishyira hamwe gukomeye hamwe no gusangira na KV-cache kugabana, kwishushanya-moderi idakeneye uwashushanyije, hamwe nibyuma bifatanyiriza hamwe bifata ibyemezo bisa. Imiterere ya EAGLE yerekana uburyo bwateguwe bugera kuri moderi nyinshi kandi zitekereza, aho iminyururu miremire yibitekerezo ituma ikiguzi kuri buri kimenyetso kibabaza cyane, kandi no kumurongo wibikoresho aho ubukererwe bufite akamaro kanini.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Kora ikizamini cya Speculative Decoding na EAGLE

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Speculative Decoding na EAGLE ni iki?

Gukoresha decoding yihutisha imvugo nini yerekana imvugo ureka akantu gato kerekana icyitegererezo ukeka ibimenyetso byinshi imbere, iyo moderi nini noneho ikagenzura muri pass imwe. EAGLE nuburyo bugezweho bwa verisiyo ishushanya kurwego rwibiranga aho kuba urwego rwikimenyetso, itanga umuvuduko wa 2-4x hamwe nigihombo cya zeru mubisohoka.

Ni ikihe gitekerezo cyibanze inyuma ya decoding?

Gitoya ishushanya ikeka ibimenyetso byinshi imbere, kandi intego nini yerekana igenzura hamwe, ikemera prefix ndende ikosora.

Nigute EAGLE itandukaniye nuburyo bwambere bwo gushushanya decoding?

EAGLE ihanura intego yicyitegererezo yibintu byihishe kandi ikoresha umutwe wa LM, itanga ibishushanyo mbonera kuruta uburyo bwa token gusa.

Kuki speculative decoding ifatwa nk'idatakaza amakuru (lossless) iyo ishyizwe mu bikorwa hakoreshejwe kugenzura neza (exact verification)?

Speculative sampling nyayo ibungabunga imiterere y'ibisubizo byitezwe; ibi ntibivuze ko buri gikorwa cy'amahirwe kigomba gusohora urukurikirane rw'ibimenyetso rumwe iyo hakoreshejwe uburyo butandukanye bw'amahirwe.

Ni ikihe kibazo mubikorwa byateguwe na EAGLE kugaburira ibimenyetso byabanje gushira bifasha gukemura?

Gutondekanya ku kimenyetso kibanziriza iki kigabanya kudasobanuka guhanura ibikurikira byihishe, kunoza umushinga wukuri.

Niki EAGLE-2 yongeyeho kuri EAGLE yumwimerere?

EAGLE-2 yashyizeho imiterere-yimikorere yingirakamaro igiti, yagura amashami yizeza kuzamura igipimo cyo kwemerwa.