Gutondeka gutemba no guhanura byinshi
Gukwirakwiza ibintu hamwe no guhanura byinshi byihutisha ururimi rwicyitegererezo mugukeka ibimenyetso byinshi bizaza icyarimwe hanyuma ukabigenzura mumurongo umwe, aho gutanga ikimenyetso kimwe icyarimwe.
Incamake
They cut latency without changing the text the model would have written.
Kwibira cyane
Ubusanzwe autoregressive decoding iratinda kuko buri kimenyetso gisaba pass yuzuye imbere kandi ibimenyetso byakozwe neza nyuma yikindi, bigatuma GPU idakoreshwa. Decoding idasanzwe ikosora ibi hamwe nigishushanyo gihenze gitanga agace kerekana ibimenyetso byabakandida, iyo moderi nini igereranya hanyuma ikagereranya; imbanzirizamushinga iyo ari yo yose ihuye nicyo intego yaba yarabyaye yemerwa kubuntu, kandi idahuye ryambere rirakosorwa. Gukwirakwiza ibintu hamwe na Medusa-yuburyo bwinshi-bwo guhanura bikubye igishushanyo mbonera ubwacyo: imitwe yoroheje yo guhanura imitwe yoroheje (cyangwa umurongo wibimenyetso byerekana) reka icyitegererezo kimwe cyateguwe kandi kigenzurwe, wirinde umushinga wihariye. Kuberako verisiyo isobanutse neza, ibisohoka bisaranganya bisa na decoding isanzwe, ubona gusa inshuro 2 kugeza kuri 3 intambwe zikurikiranye.
Ubushishozi
Urufunguzo ni uko transformateur ishobora gutsinda imyanya myinshi mumurongo umwe wambere uhendutse nkumwe, kubera ko ari memoire-umurongo mugari, ntabwo ubarwa, mugihe cya decoding. Imitwe myinshi yo guhanura isohora abakandida ibimenyetso kumyanya ikurikira; igiti cyangwa urutonde rwabakandida bigenzurwa hamwe, kandi kwemererwa gukoresha icyitegererezo cyo kwangwa (cyangwa guhuza umururumba) bityo ibimenyetso byemewe bikurikiza intego yo kugabura. Uburebure bwemewe kuri buri ntambwe bugena umuvuduko.
Ingaruka z'Ingamba
Igiciro na bije
Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
Ibyemezo bisobanutse
Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
Kugenzura ubuziranenge
Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.
Ejo hazaza h'ibishushanyo mbonera no guhanura byinshi
Uburyo bwo kwibeshya budakeneye umushinga wuburyo butandukanye burimo guhinduka muburyo budasanzwe muri moteri zifatika, kandi ubushakashatsi butera igipimo cyo kwemerwa hejuru hamwe nimitwe myiza yimitwe myiza, abakandida bubakiwe n’ibiti, no guhugura icyitegererezo fatizo hamwe no guhanura ibimenyetso byinshi (bishobora no kuzamura ireme). Tegereza ubwo buhanga guhuza hamwe no kugereranya no gutobora kugirango abafasha bakorana bumve ako kanya nubwo moderi ikura.
Gushyira mu bikorwa Isi
Kugabanya ubukererwe bwibisubizo byumufasha wibiganiro kuri 2 kugeza 3x ukoresheje Medusa-yuburyo bwinyongera bwo guhanura
Ongeraho kwishushanya-decoding kuri seriveri yerekana ko ntamushinga wihariye ugomba kwakirwa
Kwihutisha code kurangiza aho birebire, byateganijwe token ikora byemewe mubice binini
Kugabanya igiciro cya GPU kubisabwa ukuramo ibimenyetso byinshi muri buri kwibuka-byerekanwe imbere
Ingaruka & Kurinda
Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.
Igishushanyo mbonera
Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
Tegura inzira yo gusubiza ibyabaye mbere yo gupima.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Streaming and Multi-Token Prediction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Amahugurwa menshi yo guhanura
Ibibazo bikunze kubazwa
What is Speculative Streaming and Multi-Token Prediction?
Gukwirakwiza ibintu hamwe no guhanura byinshi byihutisha ururimi rwicyitegererezo mugukeka ibimenyetso byinshi bizaza icyarimwe hanyuma ukabigenzura mumurongo umwe, aho gutanga ikimenyetso kimwe icyarimwe. Bagabanye ubukererwe badahinduye inyandiko icyitegererezo cyaba cyanditse.
Ni ukubera iki decode isanzwe ya autoregressive akenshi itinda kuri GPU?
Buri kimenyetso gikeneye icyerekezo cyacyo cyimbere kandi kirakurikiranye cyane, GPU rero nibuka-umurongo mugari kandi ntibikoreshwa mugihe cya decoding.
Niki 'uwashushanyije' akora muri decoding spekulifike?
Igishushanyo gihenze gitanga igitekerezo cyabakandida ibimenyetso byerekana intego nini hanyuma igenzura muburyo bubangikanye.
Nigute ubuziranenge bwibisohoka bubitswe muri decoding?
Kugenzura (umururumba uhuza cyangwa kwangwa icyitegererezo) byemeza ibimenyetso byemewe gukurikiza igabanywa nyaryo intego yagenewe, bityo umusaruro ntuhinduka.
Niki gitandukanya Medusa-yuburyo bwinshi bwo guhanura hamwe na decoding ya classique ya classique?
Uburyo bwa Medusa-bukubye gushushanya muburyo bw'imitwe hamwe n'imitwe y'inyongera yo guhanura, wirinda gukenera kwakira umushinga wihariye.
Kuki transformateur ishobora kugenzura imyanya myinshi yabakandida hafi ya bihendutse nkimwe mugihe cya decoding?
Mugihe cyo gutobora icupa ryimuka riva mububiko, bityo gutsinda amanota yinyongera muri pass imwe byongera igiciro gito cyo kubara.