Inofungidzira Decoding Draft Models
Kudhirodha kwekufungidzira kunoshandisa diki, inokurumidza 'dhizaini' modhi kufungidzira akati wandei ari kuuya ma tokeni ayo hombe modhi inozo ongorora mupasi rimwe.
Pfupiso
It speeds up text generation 2-3x with no change to the output.
Kudzika Kwakadzika
Mienzaniso mikuru yemitauro inoburitsa chinyorwa chimwe chete panguva, uye nhanho imwe neimwe inoda kuenderera mberi kwakazara nemabhiriyoni emaparamita - inononoka uye inosungwa mundangariro. Yekufungidzira decoding inorwisa izvi nekubatanidza iyo hombe 'chinangwa' modhi neyakachipa 'dhizaini' modhi. Iyo yekudhirowa modhi inokurumidza kukurudzira chunk ye, toti, 4-8 mumiriri tokens. Iyo hombe modhi inozoagadzirisa ese mune imwechete yakafanana kuenda mberi uye inotarisa imwe neimwe. Zviratidzo zvinopindirana neizvo mufananidzo mukuru ungadai wakagadzira zvinogamuchirwa; yekutanga kusawirirana inogadziriswa uye imwe yese inoraswa. Nekuti kuonesa tokeni akati wandei kamwechete kunodhura zvakangofanana nekugadzira imwe, inotambirwa inomhanya inenge yemahara. Zvine hutsinye, nhanho yekuramba-sampling inovimbisa kugovera kwekupedzisira kwakafanana nekumhanyisa iyo hombe modhi chete - kumhanya pasina kurasikirwa kwemhando.
Technical Insight
Iyo yakakosha trick ndeye yakagadziridzwa yekuramba-sampling bvunzo. Pachiratidzo chega chega chakanyorwa, mukana weiyo inotarirwa modhi inofananidzwa neyeiyo dhizaini modhi. Kana chinangwa chinopa mukana wakaenzana kana wepamusoro, chiratidzo chinogamuchirwa; zvikasadaro inogamuchirwa ine mukana wakaenzana nereshiyo, uye pakurambwa chiratidzo chakagadziriswa chinotorwa kubva kune yakagadziridzwa yasara kugovera. Masvomhu aya anoita kuti zvinobuda zvienzane nesampling zvakananga kubva kune hombe modhi.
Strategic Impact
Kumhanya uye chiyero
Mutauro workflows inogona kufamba nekukurumidza pasina kupira kuenderana.
Svika uye svika
Inopamhidzira kupinda mumitauro yese nemataera ekutaurirana.
Sarudzo dzakajeka
Zvikwata zvinogona kupedza nguva yakawanda pakutonga uku otomatiki ichibata kudzokorora.
Ramangwana reKufungidzira Decoding Draft Models
Tarisira kudhirowa modhi kuti ive yakajairika masisitimu mumaseva ekufungidzira sevLLM uye TensorRT-LLM. Kuzvifungira kwakasiyana (Medusa, EAGLE) inodonhedza iyo yakaparadzana dhizaini modhi zvachose nekuwedzera isingaremi kufembera misoro, uye miti-yakavakirwa dhizaini inosimbisa kuenderera kwevamiriri kamwechete. Sezvo mahwindo anokura uye mutengo wekushandira unotonga, hungwaru, modhi-inofananidzwa dhizaini uye hardware-inoziva verification inosundira kugamuchirwa mitengo uye kuburikidza nepamusoro.
Real-World Implementation
Anthropic, OpenAI, uye Google vanoshandisa decoding yekufungidzira kuderedza kunonoka uye kushandisa mutengo kune vabatsiri vekutaura vanoshandira mamirioni evashandisi.
vLLM uye NVIDIA TensorRT-LLM ngarava yakavakirwa-mukati yekufungidzira decoding kuitira kuti ivo vanozvitambira vagone kukurumidzira Llama kana Mistral deployments.
Kubatanidza 7B dhizaini modhi ine 70B yakanangana (semuenzaniso, Llama-3 mhuri) kune inokwana kaviri tokens-pasekondi pane imwe GPU.
Maturusi ekupedzisa kodhi anoshandisa diki dhizaini modhi kuratidza boilerplate iyo yakakura modhi inosimbisa, ichichengeta mazano ari nyore mupepeti.
Njodzi & Guardrails
Chokwadi chehuroyi chinogona kupinda chinyararire mishumo, kuyerera kwetsigiro, kana tsvakiridzo.
Kunzwa nekukasira kunogona kugadzira mhedzisiro isingaenderane pane zvikumbiro zvakafanana.
Sensitive text data inogona kuburitswa kana zvidhiraivho zvisina kusimba.
Implementation Roadmap
Tsanangura chimiro chekubuda, toni, uye mhando zviyero usati waburitsa.
Mhinduro dzepasi neakavimbika masosi pese pazvine basa.
Chengetedza ongororo yekuongorora yemunhu kune yakakwira-stake zvinobuda.
Tevera maitiro ekutadza uye dzidzisazve kukurudzira kana mafambiro ebasa nguva nenguva.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding Draft Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Kunofungira Magadzirirwo eMakodhi Models
Mibvunzo inowanzo bvunzwa
What is Speculative Decoding Draft Models?
Kudhirodha kwekufungidzira kunoshandisa diki, inokurumidza 'dhizaini' modhi kufungidzira akati wandei ari kuuya ma tokeni ayo hombe modhi inozo ongorora mupasi rimwe. Inomhanyisa chizvarwa chemavara 2-3x pasina shanduko kune inobuda.
Nderipi basa rekutanga remuenzaniso we'draft' mukufungidzira decoding?
Iyo diki yekudhirowa modhi inofungidzira zvakachipa ma tokens, ayo hombe inotarirwa modhi inozotarisa mune imwechete yakafanana pass.
Sei kuongorora akawanda akadhindwa tokeni panguva imwe chete kuchichengetedza nguva?
Chizvarwa chakasungwa mundangariro; kutarisa ma tokens akati wandei mune imwe batched pass yakachipa senhanho imwe, saka inogamuchirwa inomhanya inenge yemahara.
Chii chinoitika kune akadhirowewa tokeni mushure mekutanga chiratidzo chechiratidzo charamba?
Kubvuma kunoenderera kusvikira kusawirirana kwekutanga; chiratidzo ichocho chinogadziriswa uye ese anotevera akanyorwa tokeni anoraswa.
Ko kufungidzira decoding inoita sei kuti mhando yekubuda haina kuderedzwa?
Iyo yakagadziridzwa yekuramba-sampling bvunzo inovimbisa yekupedzisira kugovera tokeni kunoenderana nesampling zvakananga kubva kune yakananga modhi.
Ndeipi yeiyi nzira 'yekuzvifungira' inonzvenga yakasarudzika modhi?
Medusa uye EAGLE wedzera huremu hwekuwedzera misoro yekufungidzira kune iyo huru modhi kuti ikwanise kunyora ayo emangwana tokens.