Lelee isi ngbanwe
Ndozi anya na-agba ọsọ ọsọ ọgbọ LLM na-enweghị ụdị ọ bụla ọzọ site na ịkọ nkọ na nyochaa ọtụtụ akara ngosi n'ọdịnihu n'otu aka ahụ site na iji n-gram nke ihe nlereanya ahụ na-ebute na ofufe.
Nchịkọta
It breaks the strict one-token-at-a-time bottleneck.
Ime miri emi
Ndị nyocha webatara na UC Berkeley na 2023, nhọpụta anya isi na-eme ka ntinye aka na-eji naanị ihe atụ ebumnuche n'onwe ya - enweghị ụdị nke abụọ yana enweghị ọzụzụ inyeaka. Ọ na-emezi ọgbọ ka ọ na-edozi usoro nha anya na-abụghị nke ntanetị site na iji usoro yiri nke a na-akpọ Jacobi iteration. Na nzọụkwụ nke ọ bụla ihe nlereanya ahụ na-agba alaka abụọ n'otu oge: alaka 'lookahead' nke na-emezi atụmatụ maka ọtụtụ ọnọdụ token n'ọdịnihu n'otu aka ahụ, yana alaka 'nkwenye' nke na-enyocha n-gram na-ekwe nkwa multi-token anakọtara na ọdọ mmiri. N-gram egosipụtara nke ihe nlereanya ahụ kwenyere na a na-eme ihe niile n'otu oge, yabụ enwere ike ịnakwere ọtụtụ akara n'otu nzọụkwụ. N'ihi na ọ na-adabere naanị na ngafe n'ihu nke ihe nlereanya ahụ, mmepụta na-anọgide kpọmkwem ihe anyaukwu ma ọ bụ ihe ngbanwe ga-emepụta, na-ebelata ọnụ ọgụgụ usoro usoro achọrọ.
Nghọta nka nka
Echiche bụ isi na-agbaziri Jacobi/Gauss-Seidel nrụtụ-aka a kapịrị ọnụ: a na-ahụta nhọpụta autoregressive dị ka ịchọta ebe edobere nke eserese ngosi ihe nlegharị anya n'elu windo nke akara ngosi n'ọdịnihu. A na-anụcha atụmatụ ndị yiri ya, na ọdọ mmiri n-gram na-echekwa usoro akara ngosi pụtara n'oge mmeghari ndị a. Nkwenye na-akwado ma n-gram ọ bụla nke echekwara dabara na ezi mpụta na-esote ihe nlereanya ahụ, na-ahapụ ọtụtụ token ịga n'ihu n'otu ngafe na-enweghị netwọkụ dị iche.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke ngbanwe nke Lookhead
Ndozi nke Lookhead na-adọrọ adọrọ n'ihi na ọ chọghị ihe nlereanya ọzọ iji zụọ, ibuga, ma ọ bụ idobe ebe nchekwa - na-ebelata nkuchi maka ndị na-anabata onwe. Na-atụ anya mwekota n'ime usoro ndị ọzọ na-eje ozi yana nchikota ya na nhọpụta ntule yana nkwalite cache KV. Nchọpụta na-emezi nha windo na njikwa ọdọ mmiri n-gram maka ọrụ dị iche iche, yana ịchọpụta ka usoro ahụ si agba ogologo ogologo yana ije ozi ebe GPU compute na-ejighị ya eme ihe.
Mmejuputa n'ezie n'ụwa
Ịkwado ihe ngosi mepere emepe dị ka Llama ma ọ bụ Vicuna na-eji nwayọ nwayọ na-enweghị ọzụzụ ma ọ bụ na-ebunye ụdị ihe enyemaka ọ bụla.
Mbelata ọnụ ọgụgụ nke usoro ngbanwe usoro maka ọgbọ dị ogologo dị ka edemede ma ọ bụ koodu, ebe flops bara ụba mana nzọụkwụ bụ ihe mgbochi.
Ntinye n'ime ọba akwụkwọ ntinye aka (mwepụta mbụ butere mmejuputa FlashAttention dakọtara) iji kwalite mmepụta na GPU ndị dị adị.
Na-eme ka ozi ngwa ngwa na ngwaike ejighị ya rụọ ọrụ site n'ịzụ ahịa myirịta agbakwunyere maka ngafe usoro dị ole na ole.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Lookahead Decoding quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ọkpụkpụ-nke-Echere Ndozi ndakọrịta
Ajụjụ a na-ajụkarị
What is Lookahead Decoding?
Ndozi anya na-agba ọsọ ọsọ ọgbọ LLM na-enweghị ụdị ọ bụla ọzọ site na ịkọ nkọ na nyochaa ọtụtụ akara ngosi n'ọdịnihu n'otu aka ahụ site na iji n-gram nke ihe nlereanya ahụ na-ebute na ofufe. Ọ na-agbaji otu ihe nrịbama-n'otu oge siri ike.
Kedu ihe dị iche n'ichepụta isi isi na ngbanwe ọkọlọtọ ọkọlọtọ?
Ndozi Lookhead na-eme ka ọgbọ na-aga n'ihu na-eji naanị ụdị ngafe aga n'ihu nke ihe atụ, na-enweghị netwọkụ inyeaka.
Kedu usoro ọnụọgụ na-akwado ngbanwe isi anya?
Ọ na-emegharị ngbanwe akpaaka dị ka sistemu na-adịghị adị n'ịntanetị nke ejiri ụdị Jacobi na-arụkọ ọrụ ọnụ karịa akara n'ọdịniihu.
Gịnị bụ alaka abụọ yiri ibe ha na-agba na nzọụkwụ nke ọ bụla?
Alaka ụlọ ọrụ na-emezi atụmatụ maka ọkwa ga-abịa n'ihu ebe ngalaba nkwenye na-enyocha n-gram ndị a na-aga ime site na ọdọ mmiri ahụ.
Kedu ihe echekwara na ọdọ mmiri n-gram?
Ọdọ mmiri ahụ na-echekwa n-gram ndị a na-eme ntuli aka emepụtara n'oge a na-emegharị ya ka e wee nwee ike nyochaa ya ma nwee ike itinye ya na nzọụkwụ n'ọdịnihu.
Kedu ka nhọpụta lookahead si emetụta ogo mmepụta ma e jiri ya tụnyere ntọpụta nkịtị?
N'ihi na ọ bụ naanị ihe nfefe nke ihe nlereanya a ka a na-eji wee nyochaa ya, nsonaazụ ya dabara na ngbanwe ọkọlọtọ ga-ewepụta.