Kugereranya
Gukoresha decoding yerekana ururimi runini rwerekana inyandiko byihuse ukoresheje akantu gato, byihuse 'umushinga' wo gukeka ibimenyetso byinshi biri imbere, hanyuma kugira moderi nini igenzura byose icyarimwe.
Incamake
It speeds up inference 2-3x with identical output quality.
Kwibira cyane
Mubisanzwe LLM itanga inyandiko ikimenyetso kimwe icyarimwe: buri kimenyetso gisaba kunyura imbere yuzuye binyuze muri moderi nini, kandi ntushobora gutangira igikurikira kugeza ubu icyarangiye. Ibi biratinda kuko birahujwe no kwibuka, ntabwo bibarwa - GPU imara umwanya munini wikoreza ibiro, idakora imibare. Gukoresha decoding yibice bimena icyuho. Gitoya, ihendutse umushinga w'icyitegererezo utanga igice cya, tuvuge, ibimenyetso bitanu by'abakandida. Icyitegererezo kinini 'intego' noneho gitunganya byose uko ari bitanu muburyo bumwe buringaniye imbere hanyuma ikabigenzura. Tokens ihuye nibyo yaba yarabyaye iremewe; ku kutumvikana kwambere gukosora no guta ibisigaye. Kuberako kugenzura ibimenyetso byinshi bigura hafi kimwe no kubyara kimwe, ibitekerezo byemewe ni ubuntu.
Ubushishozi
Igice cyubwenge ni itegeko ryo kwangwa-gutoranya byemeza ko ibisohoka bisaranganywa bihwanye no gukoresha icyitegererezo cyonyine - bityo ubuziranenge ntibugereranijwe, ni ukuri. Igipimo cyo kwakirwa gitwara umuvuduko: nibyiza icyitegererezo gito gihanura kinini, ibimenyetso byinshi bifata kuri intambwe yo kugenzura. Ibihinduka nka Medusa byongeweho imitwe yinyongera kubigenewe intego ubwayo, hamwe na EAGLE ishushanya mumwanya wihariye, ikuraho ibikenewe byimishinga itandukanye.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Kazoza Kudasanzwe
Gukoresha decoding yibintu birahinduka mugutanga ibice nka vLLM na TensorRT-LLM. Tegereza uburyo bwo kwishushanya (Medusa, EAGLE, Lookahead) kuganza kuko birinda gukomeza icyitegererezo cya kabiri, hiyongereyeho ibiti bishingiye ku biti bigenzura amashami menshi y'abakandida kuri buri ntambwe. Mugihe icyitegererezo kigenda gikura, ububiko-buke bwibuke bugenda bwiyongera, bigatuma ibitekerezwa birushaho kuba byiza, kandi abashushanya ibyuma-bizamura ibyuma bizamura umuvuduko wisi-mwinshi.
Gushyira mu bikorwa Isi
Igishushanyo mbonera cya 7B cyerekana ibimenyetso byerekana 70B yo kuganira kugirango ugabanye ubukererwe bwumufasha wumusaruro
Imitwe ya Medusa ihindagurika kuri LLM kuburyo iteganya ibimenyetso byinshi bizaza icyarimwe nta mushinga wihariye
vLLM ituma decoding yibitekerezo yo kuzamura ibimenyetso-kumasegonda byinjira kumurongo ukorera
EAGLE itegura icyitegererezo cyihishe-kiranga umwanya wo kuzamura igipimo cyo kwakirwa n'umuvuduko rusange
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Gukoresha Decoding hamwe na EAGLE
Ibibazo bikunze kubazwa
What is Speculative Decoding?
Gukoresha decoding yerekana ururimi runini rwerekana inyandiko byihuse ukoresheje akantu gato, byihuse 'umushinga' wo gukeka ibimenyetso byinshi biri imbere, hanyuma kugira moderi nini igenzura byose icyarimwe. Yihutisha umwanzuro 2-3x hamwe nubwiza busohoka.
Ni ikihe gitekerezo cyibanze cyo gushushanya ibintu?
Igishushanyo cyihuse cyerekana ibimenyetso byinshi, kandi intego nini yerekana igenzura byose muburyo bumwe.
Ni ukubera iki isanzwe imwe-ikimenyetso-icyarimwe LLM itinda?
Buri ntambwe yikoreza cyane matrices yuburemere kuva yibuka aho gukora comptabilite iremereye, bityo GPU idakoreshwa.
Bigenda bite ku kimenyetso cya mbere aho umushinga na moderi bigamije kutemeranya?
Tokens kugeza kubutumvikana iremewe; guhera kudahuye imbere baranze kandi icyerekezo cyicyitegererezo cyukuri kibitswe.
Nigute decoding yibintu bigira ingaruka nziza kubisohoka?
Amategeko yo kwangwa-icyitegererezo yemeza ko isaranganya ryanyuma rihuye neza nicyitegererezo, bityo ubwiza ntibuhinduka.
Ni iki kigena mu buryo butaziguye umuvuduko uva kuri decoding?
Nibindi byateguwe byerekana intego intego yemera kuri buri ntambwe yo kugenzura, niko byihuta.