Ubuyobozi bwa tekiniki
Gutondeka gutemba no guhanura byinshi
Gukwirakwiza ibintu hamwe no guhanura byinshi byihutisha ururimi rwicyitegererezo mugukeka ibimenyetso byinshi bizaza icyarimwe hanyuma ukabigenzura mumurongo umwe, aho gutanga ikimenyetso kimwe icyarimwe.
Umva iki gitaboAI-byavuzwe · ~2 min
Kuriyi page2 min soma
Incamake
They cut latency without changing the text the model would have written.
Kwibira cyane
Ubusanzwe autoregressive decoding iratinda kuko buri kimenyetso gisaba pass yuzuye imbere kandi ibimenyetso byakozwe neza nyuma yikindi, bigatuma GPU idakoreshwa. Decoding idasanzwe ikosora ibi hamwe nigishushanyo gihenze gitanga agace kerekana ibimenyetso byabakandida, iyo moderi nini igereranya hanyuma ikagereranya; imbanzirizamushinga iyo ari yo yose ihuye nicyo intego yaba yarabyaye yemerwa kubuntu, kandi idahuye ryambere rirakosorwa. Gukwirakwiza ibintu hamwe na Medusa-yuburyo bwinshi-bwo guhanura bikubye igishushanyo mbonera ubwacyo: imitwe yoroheje yo guhanura imitwe yoroheje (cyangwa umurongo wibimenyetso byerekana) reka icyitegererezo kimwe cyateguwe kandi kigenzurwe, wirinde umushinga wihariye. Kuberako verisiyo isobanutse neza, ibisohoka bisaranganya bisa na decoding isanzwe, ubona gusa inshuro 2 kugeza kuri 3 intambwe zikurikiranye.
Gushyira mu bikorwa Isi
Kugabanya ubukererwe bwibisubizo byumufasha wibiganiro kuri 2 kugeza 3x ukoresheje Medusa-yuburyo bwinyongera bwo guhanura
Ongeraho kwishushanya-decoding kuri seriveri yerekana ko ntamushinga wihariye ugomba kwakirwa
Kwihutisha code kurangiza aho birebire, byateganijwe token ikora byemewe mubice binini
Kugabanya igiciro cya GPU kubisabwa ukuramo ibimenyetso byinshi muri buri kwibuka-byerekanwe imbere
Ejo hazaza h'ibishushanyo mbonera no guhanura byinshi
Uburyo bwo kwibeshya budakeneye umushinga wuburyo butandukanye burimo guhinduka muburyo budasanzwe muri moteri zifatika, kandi ubushakashatsi butera igipimo cyo kwemerwa hejuru hamwe nimitwe myiza yimitwe myiza, abakandida bubakiwe n’ibiti, no guhugura icyitegererezo fatizo hamwe no guhanura ibimenyetso byinshi (bishobora no kuzamura ireme). Tegereza ubwo buhanga guhuza hamwe no kugereranya no gutobora kugirango abafasha bakorana bumve ako kanya nubwo moderi ikura.
Kubishyira mubikorwa
Impamvu ari ngombwa
- Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
- Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
- Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Witondere
- Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
- Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
- Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Intambwe ikurikira
- Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
- Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
- Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
- Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Ibaruwa y'amakuru y'ubuntu
Ibuka amakuru ya buri munsi ya AI
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Take the Speculative Streaming and Multi-Token Prediction quiz
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Ibibazo bikunze kubazwa
What is Speculative Streaming and Multi-Token Prediction?
Gukwirakwiza ibintu hamwe no guhanura byinshi byihutisha ururimi rwicyitegererezo mugukeka ibimenyetso byinshi bizaza icyarimwe hanyuma ukabigenzura mumurongo umwe, aho gutanga ikimenyetso kimwe icyarimwe. Bagabanye ubukererwe badahinduye inyandiko icyitegererezo cyaba cyanditse.
Ni ukubera iki decode isanzwe ya autoregressive akenshi itinda kuri GPU?
Buri kimenyetso gikeneye icyerekezo cyacyo cyimbere kandi kirakurikiranye cyane, GPU rero nibuka-umurongo mugari kandi ntibikoreshwa mugihe cya decoding.
Niki 'uwashushanyije' akora muri decoding spekulifike?
Igishushanyo gihenze gitanga igitekerezo cyabakandida ibimenyetso byerekana intego nini hanyuma igenzura muburyo bubangikanye.
Nigute ubuziranenge bwibisohoka bubitswe muri decoding?
Kugenzura (umururumba uhuza cyangwa kwangwa icyitegererezo) byemeza ibimenyetso byemewe gukurikiza igabanywa nyaryo intego yagenewe, bityo umusaruro ntuhinduka.
Niki gitandukanya Medusa-yuburyo bwinshi bwo guhanura hamwe na decoding ya classique ya classique?
Uburyo bwa Medusa-bukubye gushushanya muburyo bw'imitwe hamwe n'imitwe y'inyongera yo guhanura, wirinda gukenera kwakira umushinga wihariye.
Kuki transformateur ishobora kugenzura imyanya myinshi yabakandida hafi ya bihendutse nkimwe mugihe cya decoding?
Mugihe cyo gutobora icupa ryimuka riva mububiko, bityo gutsinda amanota yinyongera muri pass imwe byongera igiciro gito cyo kubara.
Komeza wige
Kuyobora
Abandi bayobozi batoranijwe kuriyi ngingo