InoteveraGaidhi rinotevera
Multi-Token Prediction Training
Mutauro AI
Nhungamiro yehunyanzvi
Kufungidzira Kutenderera uye Multi-Token Prediction
Kutepfenyura uye kufanotaura kwezviratidzo zvakawanda kumhanyisa chizvarwa chemhando yemutauro nekufungidzira akati wandei emangwana tokeni kamwechete uye nekuaona nekupfuura kumwe chete, pane kugadzira chiratidzo chimwe panguva.
Teerera kugwaro iriAI-yakarondedzerwa · ~2 min
Pa peji ino2 min verenga
Pfupiso
They cut latency without changing the text the model would have written.
Kudzika Kwakadzika
Yakajairika autoregressive decoding inononoka nekuti chiratidzo chega chega chinoda kupfuudza kumberi uye ma tokeni anogadzirwa akanyatso mumwe mushure memumwe, achisiya GPU isingashandiswe. Kudhirodha kwekufungidzira kunogadzirisa izvi nedhizaini rakachipa iro rinokurudzira chunk yematokeni evamiriri, iyo iyo hombe yaivavarirwa modhi yobva yaona zvakafanana; chero prefix inofanana neicho chinangwa chingave chaburitsa chinogamuchirwa mahara, uye yekutanga kusawirirana inogadziriswa. Kutepfenyura kutenderera uye Medusa-maitiro akawanda-tokeni kufanotaura peta dhizaini mumodhi pachayo: yakawedzera kureruka misoro yekufungidzira (kana rwizi rwekufungidzira tokens) rega modhi imwe inyore uye simbisa, ichinzvenga yakaparadzana dhizaini modhi. Nekuti verification ndeyechokwadi, kugovera kwakabuda kwakafanana neyakajairwa decoding, iwe unongowana 2 kusvika ku3 kakapetwa mashoma anotevedzana matanho.
Kuitwa muNyika yeChokwadi
Kucheka mhinduro latency yemubatsiri wekutaura ne2 kusvika 3x uchishandisa Medusa-maitiro ekuwedzera ekufanotaura misoro.
Kuwedzera-yekufungidzira decoding kune inference server saka hapana yakaparadzana dhizaini modhi inoda kutambirwa
Kumhanyisa kupedzisa kodhi apo refu, inofanotaurwa tokeni inotambirwa muzvikamu zvakakura
Kuderedza mutengo weGPU pachikumbiro nekuburitsa mamwe ma tokens kubva kune yega yega-inosungwa yekupfuura pass
Ramangwana reKufungidzira Kutenderera uye Multi-Token Prediction
Nzira dzekungofungidzira dzisingade dzakapatsanurwa dhizaini modhi dzave kuita dzisiri muinjini dzekufungidzira, uye tsvakiridzo iri kusundidzira mitengo yekugamuchirwa yakakwira nemisoro yedhirowa iri nani, vagadziri vemiti, uye kudzidzisa iyo base modhi pamwe chete kune akawanda-token kufanotaura (ayo anogona zvakare kuvandudza mhando). Tarisira aya matekiniki kuti abatanidze nequantization uye batching saka vanopindirana vabatsiri vanonzwa nekukasira kunyangwe mamodheru achikura.
Kuzvishandisa
Nei zvichikosha
- Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
- Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
- Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.
Chenjera
- Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
- Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
- Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.
Matanho anotevera
- Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
- Benchmark pasi pechokwadi mutoro uye data mamiriro.
- Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
- Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.
Ramba Uchiongorora
Tsamba yemahara
Tora iyo yemazuva ese AI pfupiso
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Take the Speculative Streaming and Multi-Token Prediction quiz
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Mibvunzo inowanzo bvunzwa
What is Speculative Streaming and Multi-Token Prediction?
Kutepfenyura uye kufanotaura kwezviratidzo zvakawanda kumhanyisa chizvarwa chemhando yemutauro nekufungidzira akati wandei emangwana tokeni kamwechete uye nekuaona nekupfuura kumwe chete, pane kugadzira chiratidzo chimwe panguva. Ivo vanocheka latency vasina kushandura zvinyorwa izvo modhi ingadai yakanyora.
Nei yakajairwa autoregressive decoding ichiwanzononoka paGPU?
Chiratidzo chega chega chinoda chayo chekupfuura uye chakanyatso kutevedzana, saka iyo GPU ndeyeyeuko-bandwidth inosungwa uye isingashandiswe panguva yekudhirodha.
Ko 'mudhirowa' anoita sei mukufungidzira decoding?
Iyo yakachipa dhizaini inokurudzira chunk yevamiriri tokeni iyo hombe yainotarirwa modhi yobva yaona zvakafanana.
Hunhu hwekubuda hunochengetedzwa sei mukufungidzira decoding?
Verification (kukara kuenzanisa kana kurambwa sampling) inova nechokwadi chekuti tokeni dzinogamuchirwa dzinotevera kugovera chaiko kwaizoburitswa modhi inotangwa, saka zvinobuda hazvichinji.
Chii chinosiyanisa Medusa-maitiro akawanda-token kufanotaura kubva kukirasi yekufungidzira decoding?
Medusa-maitiro nzira dzinopeta dhizaini mune modhi ine mamwe ekufanotaura misoro, kudzivirira kukosha kwekugamuchira yakaparadzana dhizaini modhi.
Sei transformer ichigonesa zvigaro zvakawanda zvevamiriri zvakachipa seimwe panguva yekunyora?
Panguva yekudhirodha iyo bhodhoro iri kufambisa uremu kubva mundangariro, saka kuhwina zvimwe zvinzvimbo mune imwechete pass kunowedzera mari shoma yekombuta.
Ramba uchidzidza
Mazano anoenderana
Mamwe madhairekitori akasarudzirwa nyaya iyi