Iji EAGLE emezi ihe nlere anya
Ndozi nhụsianya na-eme ka ntụnye ụdị asụsụ buru ibu dị ngwa site n'ikwe ka obere ihe ngosi mara ọtụtụ akara n'ihu, nke nnukwu ihe nlereanya ahụ na-enyocha n'otu ngafe.
Nchịkọta
EAGLE is a state-of-the-art version that drafts at the feature level rather than the token level, delivering 2-4x speedups with zero loss in output quality.
Ime miri emi
Ọgbọ LLM nkịtị bụ autoregressive: ihe nlereanya ahụ na-emepụta otu token, na-azụ ya, ma na-emegharị ya, yabụ akara ngosi ọ bụla chọrọ ịgafe n'ihu n'ụzọ zuru ezu site na ijeri paramita. Ndozi nhụsianya na-agbaji ọkpọ a. Onye na-edepụta ihe dị ọnụ ala na-atụpụta ọnụ ọgụgụ nke akara akwụkwọ anamachọihe, na ụdị ebumnuche dị oke ọnụ na-enyocha ha niile n'otu ngafe yiri ya, na-anabata prefix ziri ezi kacha ogologo. EAGLE (Extrapolation Algorithm for Greater Language-Model Efficiency) na-akawanye mma na ụzọ ndị mbụ site n'ịdepụta oghere njiri mara ihe nlereanya ahụ na iweghachite ezi ntinye akara nke gara aga iji belata ejighị n'aka. EAGLE-2 na-agbakwụnye osisi dị ike, na EAGLE-3 na-atụba mmachi amụma-mmetụta ka ọ dịkwuo mma. N'ụzọ dị mkpa, nkwenye na-ekwe nkwa na mmepụta ahụ bụ otu ihe nlereanya a ga-emepụta naanị ya.
Nghọta nka nka
EAGLE na-azụ obere isi na-agba ọsọ nke na-ebu amụma ihe atụ nzuzo ga-esote ọnọdụ steeti, wee jiri isi LM nke ebumnuche ya tụgharịa atụmatụ ka ọ bụrụ ndị na-eme ntuli aka. Site n'ịkwado n'usoro akara ngbanwe gbakwunyere atụmatụ ndị bu ụzọ, ọ na-ebelata ihe mgbagwoju anya nke butere naanị atụmatụ. A na-enyocha osisi ndị na-eme ntuli aka ozugbo; A na-echekwa nkesa nke ihe atụ ahụ kpọmkwem n'ihi na akara ngosi anabatara ga-adabarịrị na nhọrọ ya nke sampled ma ọ bụ argmax, na-eme ka ọsọ ọsọ na-efunahụ ya.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke ịkọwapụta nkọwa na EAGLE
Ndozi nleba anya na-aghọ akụrụngwa ndabara n'ịkwado ngwugwu dị ka vLLM na TensorRT-LLM. Na-atụ anya mwekota siri ike na nkesa batching na KV-cache, ụdị ịdepụta onwe nke na-achọghị onye na-edepụta ihe dị iche, yana ihe nrụnye ngwaike na-ewere nkwenye yiri ya. A na-agbatị usoro nhazi ụdị EAGLE gaa n'ụdị multimodal na ụdị ntụgharị uche, ebe ogologo ogologo echiche na-eme ka ọnụ ahịa otu n'ime ihe na-egbu mgbu karịsịa na ntinye aka na ngwaọrụ ebe nkwụsịtụ kacha dị mkpa.
Mmejuputa n'ezie n'ụwa
Mbelata latency na ndị enyemaka nkata ka nzaghachi na-asọfe 2-3x ọsọ ọsọ na-agbanweghị azịza ihe nlereanya.
Mbelata ụgwọ ọrụ GPU maka ndị na-eweta API dị elu site n'ịwepụta ọtụtụ token kwa ngafe n'ihu
Ụdị echiche echiche na-eme ngwa ngwa ebe a na-emepụta puku kwuru puku token n'otu ajụjụ
Ịmelite ngwa ọrụ mmecha koodu ebe enwere ike ịkọ amụma, usoro akara ugboro ugboro na-enye ọnụego nnabata dị elu.
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding with EAGLE quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nchọpụta nhụta anya
Ajụjụ a na-ajụkarị
What is Speculative Decoding with EAGLE?
Ndozi nhụsianya na-eme ka ntụnye ụdị asụsụ buru ibu dị ngwa site n'ikwe ka obere ihe ngosi mara ọtụtụ akara n'ihu, nke nnukwu ihe nlereanya ahụ na-enyocha n'otu ngafe. EAGLE bụ ụdị ọgbara ọhụrụ nke na-edepụta na ọkwa njirimara karịa ọkwa akara, na-ebuga ọsọ ọsọ 2-4x na enweghị efu na mmepụta mmepụta.
Kedu ihe bụ isi echiche dị n'azụ ngbanwe ntule?
Obere onye na-ese ihe na-eche ọtụtụ akara n'ihu, na nnukwu ihe nlereanya a na-atụ anya na-enyocha ha ọnụ, na-anabata prefix ziri ezi kacha ogologo.
Kedu ka EAGLE si dị iche na ụzọ ngbanwe nke mbụ?
EAGLE na-ebu amụma atụmatụ zoro ezo nke ihe nlereanya ahụ wee jirikwa isi LM ya, nke na-ewepụta usoro ziri ezi karịa usoro naanị token.
Kedu ihe kpatara eji ewere ngbanwe ntule dị ka 'enweghị efu'?
N'ihi na ihe atụ a na-atụ anya na-enyocha akara ngosi nke ọ bụla edepụtara megide nkesa nke ya, ihe anabatara bụ kpọmkwem ihe ebumnuche ahụ gaara ewepụta naanị ya.
Kedu nsogbu dị na nhazi atụmatụ EAGLE na-enye aka dozie ihe ntinye akara ngosi gara aga?
Ịkwado na akara ngosi gara aga na-ebelata enweghị mgbagwoju anya nke ịkọ atụmatụ ezoro ezo na-esote, na-emeziwanye nhazi nhazi.
Kedu ihe EAGLE-2 gbakwunyere n'elu EAGLE mbụ?
EAGLE-2 webatara ihe gbara ya gburugburu maara osisi dị ike, na-agbasa alaka ndị na-ekwe nkwa iji bulie ọnụego nnabata.