Kurangiza
Kode ya Lookahead yihutisha ibisekuruza bya LLM nta yandi mashusho yinyongera mugukeka no kugenzura ibimenyetso byinshi bizaza mugihe kimwe ukoresheje n-garama icyitegererezo gitanga muguruka.
Incamake
It breaks the strict one-token-at-a-time bottleneck.
Kwibira cyane
Byatangijwe nabashakashatsi bo muri UC Berkeley mu 2023, decodehead yihuta yihutisha gufata umwanzuro ukoresheje icyitegererezo cyonyine - nta moderi ya kabiri kandi nta mahugurwa y'abafasha. Ivugurura ibisekuruza nkugukemura sisitemu yo kugereranya idafite umurongo ukoresheje uburyo bubangikanye bwitwa Jacobi itera. Kuri buri ntambwe icyitegererezo gikoresha amashami abiri icyarimwe: ishami 'rireba' rinonosora ibitekerezwa kumyanya myinshi yikimenyetso kizaza, hamwe nishami rya 'verisiyo' igenzura ibyiringiro byinshi-n-garama byakusanyirijwe muri pisine. Kugenzura n-garama icyitegererezo cyemeranijweho icyarimwe icyarimwe, bityo ibimenyetso byinshi birashobora kwemerwa kuri buri ntambwe. Kuberako yishingikiriza gusa kuri moderi yimbere yimbere, ibisohoka bikomeza kuba ibyo umururumba cyangwa icyitegererezo cya decoding byatanga umusaruro, mugihe ugabanya umubare wintambwe zikenewe zikenewe.
Ubushishozi
Igitekerezo cyibanze kiguza Jacobi / Gauss-Seidel itondekanya-ingingo itera: decode ya autoregressive ifatwa nkugushakisha ingingo ihamye yerekana ikarita yerekana idirishya ryibimenyetso bizaza. Ibigereranyo bisa neza biratunganijwe neza, kandi n-garama pisine cashe yerekana ibimenyetso byerekana ibimenyetso mugihe cyo gusubiramo. Igenzura ryemeza niba buri cashe n-garama ihuye nicyitegererezo cyukuri gikurikiraho, kureka ibimenyetso byinshi bigatera imbere muri pass imwe idafite umushinga wihariye.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka Lookahead Decoding
Kode ya Lookahead irashimishije kuko idakeneye moderi yinyongera yo guhugura, kohereza, cyangwa kugumya kwibuka - koroshya kwakirwa kubantu bonyine. Tegereza kwishyira hamwe mubikorwa byinshi hamwe no guhuza hamwe na decoding idasanzwe hamwe na KV-cache optimizasiyo. Ubushakashatsi burimo guhuza ingano yidirishya hamwe nubuyobozi bwa n-garama ya pisine kumurimo utandukanye wakazi, no gushakisha uburyo tekinike yapimye hamwe nigihe kirekire kandi ikorera hamwe aho compte ya GPU ubundi idakoreshwa.
Gushyira mu bikorwa Isi
Kwiyakira-moderi ifunguye nka Llama cyangwa Vicuna hamwe nubukererwe bwihuse nta mahugurwa cyangwa gupakira icyitegererezo cyabafasha.
Kugabanya umubare wikurikiranya ryikurikiranya ryigihe kirekire-nkibisekuru cyangwa code, aho flops ari nyinshi ariko intambwe ninzitizi.
Kwinjiza mumasomero yerekana (gusohora kwambere byohereje FlashAttention-ihuza ishyirwa mubikorwa) kugirango uzamure ibicuruzwa kuri GPUs zisanzwe.
Kwihutisha ibyiciro byakorewe kubikoresho bidakoreshwa mugucuruza ibigereranyo bisa na compte nkeya ya moderi ikurikiranye.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Lookahead Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Skeleton-y-Igitekerezo Iringaniza Decoding
Ibibazo bikunze kubazwa
What is Lookahead Decoding?
Kode ya Lookahead yihutisha ibisekuruza bya LLM nta yandi mashusho yinyongera mugukeka no kugenzura ibimenyetso byinshi bizaza mugihe kimwe ukoresheje n-garama icyitegererezo gitanga muguruka. Ivunagura cyane icyerekezo kimwe-icyarimwe.
Niki gitandukanya decode yo kureba hamwe na decoding isanzwe?
Lookahead decoding yihutisha ibisekuru ikoresheje gusa intego yicyitegererezo yimbere yimbere, nta rezo ifasha umuyoboro.
Nubuhe buryo bwo kubara bushigikira decoding?
Ivugurura decode ya autoregressive nka sisitemu idafite umurongo yakemuwe na Jacobi-stil igereranya-ingingo-itondekanya hejuru y'ibimenyetso bizaza.
Ni ayahe mashami abiri abangikanye akora kuri buri ntambwe?
Ishami rya lookahead risobanura gukeka kumwanya uzaza mugihe ishami ryigenzura rigenzura umukandida n-garama kuva muri pisine.
Ni iki kibitswe muri 'p-pisine'?
Ikidendezi gifata umukandida n-garama zakozwe mugihe cyo gusubiramo kugirango zisuzumwe kandi zishobora kuba zakozwe mu ntambwe iri imbere.
Nigute decoding ya decahead igira ingaruka kumusaruro ugereranije na decode isanzwe?
Kuberako intego yicyitegererezo yonyine pass ikoreshwa kandi ikagenzurwa, ibisubizo bihuye nibyo decoding isanzwe yatanga.