Igishushanyo mbonera cyo gushushanya
Gukoresha decoding ikoresha akantu gato, byihuse 'umushinga' icyitegererezo cyo gukeka ibimenyetso byinshi bizaza urugero runini noneho rugenzura muri pass imwe.
Incamake
It speeds up text generation 2-3x with no change to the output.
Kwibira cyane
Ururimi runini rwerekana inyandiko imwe ikimenyetso icyarimwe, kandi buri ntambwe isaba inzira yuzuye kunyura miriyari y'ibipimo - bitinda kandi byibukwa. Gukoresha decoding yibitero muguhuza icyitegererezo kinini 'intego' nicyitegererezo gihenze. Umushinga w'icyitegererezo utanga igitekerezo cyihuse igice, twavuga, 4-8 ibimenyetso by'abakandida. Icyitegererezo kinini noneho gitunganya byose muburyo bumwe bubangikanye imbere kandi bugenzura buri kimwe. Tokens ihuye nicyitegererezo kinini cyaba cyaremewe iremewe; kudahuza kwambere gukosorwa naho ibindi bikajugunywa. Kuberako kugenzura ibimenyetso byinshi icyarimwe bigura hafi kimwe no kubyara kimwe, kwiruka byemewe ni ubuntu. Icy'ingenzi, intambwe yo kwangwa-icyitegererezo yemeza ko isaranganya rya nyuma risa no gukora moderi nini yonyine - umuvuduko udafite igihombo cyiza.
Ubushishozi
Amayeri yingenzi ni ikizamini cyahinduwe cyo kwangwa. Kuri buri cyapa cyateguwe, intego yicyitegererezo ishobora kugereranywa ninteguro yicyitegererezo. Niba intego igenera amahirwe angana cyangwa arenze, ikimenyetso cyemewe; bitabaye ibyo byemewe hamwe nibishoboka bingana na ratio, naho kwangwa ikimenyetso cyakosowe cyatoranijwe kuva kugabanwa gusigara kugabanijwe. Iyi mibare ituma ibisohoka bihwanye no gutoranya biturutse kuri moderi nini.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Ejo hazaza h'ibishushanyo mbonera byo gushushanya
Tegereza umushinga wicyitegererezo uhinduke ibikorwa remezo mubisanzwe muri seriveri nka vLLM na TensorRT-LLM. Kwiyitirira kwihinduranya (Medusa, EAGLE) guta umushinga wicyitegererezo utandukanye rwose wongeyeho imitwe yoroheje yo guhanura, kandi igishushanyo mbonera cyibiti kigenzura abakandida benshi icyarimwe. Mugihe imiterere ya Windows ikura kandi igatanga ikiguzi cyiganje, ubwenge, icyitegererezo-gishushanya abashushanya hamwe no kugenzura ibyuma bizwi neza bizatera igipimo cyo kwemerwa no kwinjiza hejuru.
Gushyira mu bikorwa Isi
.
vLLM na NVIDIA TensorRT-LLM ubwato bwubatswe muri decoding spekulifike kugirango abikorera ubwabo bashobore kwihutisha ibikorwa bya Llama cyangwa Mistral.
Guhuza icyitegererezo cya 7B gifite intego ya 70B (urugero, umuryango wa Llama-3) kugirango ugereranye ibimenyetso bibiri-isegonda kuri GPU imwe.
Ibikoresho byo kurangiza-kode ikoresha umushinga muto wicyitegererezo kugirango utange ibyuka byerekana ko binini binini bigenzura, bikomeza ibitekerezo byoroshye mubanditsi.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding Draft Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Guhindura Ibishushanyo Kuri Kode Model
Ibibazo bikunze kubazwa
What is Speculative Decoding Draft Models?
Gukoresha decoding ikoresha akantu gato, byihuse 'umushinga' icyitegererezo cyo gukeka ibimenyetso byinshi bizaza urugero runini noneho rugenzura muri pass imwe. Byihuta kubyara inyandiko 2-3x nta gihindutse kubisohoka.
Ni uruhe ruhare rwibanze rw 'umushinga' muburyo bwo gushushanya?
Gitoya yerekana icyitegererezo ihendutse gukeka ibimenyetso bizaza, iyo nini nini yerekana intego hanyuma igenzura muri parike imwe.
Kuki kugenzura ibimenyetso byinshi byateguwe icyarimwe bibika umwanya?
Igisekuru kijyanye no kwibuka; kugenzura ibimenyetso byinshi murwego rumwe rwuzuye ni bihendutse nkintambwe imwe, bityo kwiruka byemewe ni ubuntu.
Bigenda bite kubimenyetso byateguwe nyuma yikimenyetso cya mbere icyitegererezo cyanze?
Kwakira birakomeza kugeza igihe ubwumvikane buke bwa mbere; icyo kimenyetso cyakosowe kandi ibimenyetso byose byateguwe nyuma bijugunywa kure.
Nigute gushishoza byerekana neza ko umusaruro usohoka utateshejwe agaciro?
Ikizamini cyahinduwe cyo kwangwa-icyitegererezo cyemeza ibimenyetso byanyuma byo gukwirakwiza bihuye neza nicyitegererezo.
Nubuhe muribwo buryo bwo 'kwiyitirira' bwirinda umushinga wihariye?
Medusa na EAGLE bongeramo imitwe yoroheje yo guhanura imitwe kurugero nyamukuru kugirango ishobore gutegura ibimenyetso byayo bizaza.