Modeldị Draft Ndozi Apụrụiche
Ndozi nleba anya na-eji ụdị 'draft' dị obere, ngwa ngwa iji maa ọtụtụ akara na-abịa na nnukwu ihe nlereanya wee nyochaa n'otu ngafe.
Nchịkọta
It speeds up text generation 2-3x with no change to the output.
Ime miri emi
Ụdị asụsụ buru ibu na-emepụta ederede otu akara n'otu oge, nzọụkwụ ọ bụla na-achọkwa ịgafe n'ihu n'ụzọ zuru ezu site na ọtụtụ ijeri paramita - nwayọọ na ebe nchekwa. Ndozi nhụsianya na-awakpo nke a site na ijikọta nnukwu ihe nlereanya 'edobere' na ụdị 'draft' dị ọnụ ala. Ihe nlereanya a na-atụpụta ngwa ngwa na-atụpụta mkpirisi, sịnụ, akara ndị ndoro-ndoro ochichi 4-8. Nnukwu ihe nlereanya ahụ na-ahazi ha niile n'otu ngafe na-aga n'ihu ma lelee nke ọ bụla. A na-anabata akara ndị dabara na ihe nnukwu ihe nlereanya gaara emepụta; a na-edozi ndakọrịta nke mbụ ma tụfuo ndị ọzọ. N'ihi na ịlele ọtụtụ token n'otu oge na-efu otu ihe dị ka ịmepụta otu, ịgba ọsọ anabatara fọrọ nke nta ka ọ bụrụ n'efu. N'ụzọ dị mkpa, usoro nlele-ajụ na-ekwe nkwa na nkesa ikpeazụ bụ otu ihe na-agba ọsọ nnukwu ihe nlereanya naanị - ọsọ na-enweghị ezigbo mfu.
Nghọta nka nka
Isi aghụghọ bụ ule nlere ajụ gbanwetụrụ. Maka akara ngosi ọ bụla edepụtara, a na-atụnyere ihe gbasara nke puru omume n'ụdị ihe atụnyere nke ihe nlereanya. Ọ bụrụ na ebumnuche ekenye nha nha ma ọ bụ karịa, a na-anabata akara ngosi ahụ; ma ọ bụghị ya, a na-anabata ya na ihe gbasara nke puru omume ha nhata, na n'ajụjụ a, a na-esetịpụ akara ngosi emeziri site na nkesa mpụta emeziri. Mgbakọ mgbakọ na mwepụ a na-eme ka mmepụta ya bụrụ nhata na nlele ozugbo site na nnukwu ụdị.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke Ụdị Ndozi Ndozi Ọdịnihu
Na-atụ anya ụdị akwụkwọ ka ọ bụrụ akụrụngwa ọkọlọtọ na sava ntinye dị ka vLLM na TensorRT-LLM. Ụdị ntule onwe onye (Medusa, EAGLE) tụfuo usoro dị iche iche kpamkpam site n'ịgbakwunye isi amụma dị fechaa, na ịdepụta osisi na-egosi na ọtụtụ ndị ga-aga n'ihu n'otu oge. Ka windo ndị gbara ya gburugburu na-eto ma na-akwụ ụgwọ ọrụ na-achịkwa, ndị nwere ọgụgụ isi, ndị na-edepụta ihe dabara adaba na nkwenye ngwaike ga-akwalite ọnụego nnabata na ntinye dị elu.
Mmejuputa n'ezie n'ụwa
Anthropic, OpenAI, na Google na-eji nhọpụta nleba anya iji belata latency na ụgwọ ọrụ maka ndị enyemaka nkata na-ejere nde ndị ọrụ ozi.
vLLM na NVIDIA TensorRT-LLM ụgbọ mmiri arụnyere n'ime ngbanwe nkọwapụta ka ndị na-akwado onwe ha nwee ike mee ka Llama ma ọ bụ Mistral rụọ ọrụ ngwa ngwa.
Ijikọta ihe atụ 7B nwere ebumnuche 70B (dịka, ezinụlọ Llama-3) ka ọ bụrụ akara ngosi okpukpu abụọ kwa nkeji na otu GPU.
Ngwa mmecha koodu na-eji obere ihe nlere na-atụpụta efere efere ka nnukwu ihe nlereanya na-enyocha, na-edobe aro na-atọ ụtọ na onye ndezi.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Decoding Draft Models quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ndozi nkọwa maka ụdị koodu
Ajụjụ a na-ajụkarị
What is Speculative Decoding Draft Models?
Ndozi nleba anya na-eji ụdị 'draft' dị obere, ngwa ngwa iji maa ọtụtụ akara na-abịa na nnukwu ihe nlereanya wee nyochaa n'otu ngafe. Ọ na-agba ọsọ ọgbọ ederede 2-3x na-enweghị mgbanwe na mmepụta.
Kedu ọrụ bụ isi nke ihe nlere 'draft' ahụ n'ime nhọpụta ntule?
Obere ihe nlereanya a na-eche n'ọnụ ọnụ ala na-abịa, bụ nke nnukwu ihe nleba anya na-enyocha na otu ngafe yiri ya.
Kedu ihe kpatara ịnwapụta ọtụtụ akara ngosi otu oge na-echekwa oge?
Ọgbọ bụ ebe nchekwa; ịlele ọtụtụ akara ngosi n'otu ngafe batched fọrọ nke nta ka ọ dị ọnụ ala ka otu nzọụkwụ, yabụ ịgba ọsọ anabatara fọrọ nke nta ka ọ bụrụ n'efu.
Kedu ihe ga-eme akara ngosi ndị e depụtara ka akara mbụ nke ihe nlereanya a jụrụ?
Nnabata na-aga n'ihu ruo mgbe esemokwu mbụ; A na-edozi akara ahụ ma tụfuo akara ngosi niile e depụtara na-esote.
Kedu ka ndoputa ntule si achọpụta na adịghị eweda ogo mmepụta?
Nnwale nlere jụrụ ajụ gbanwetụrụ na-ekwe nkwa na nkesa akara ikpeazụ ga-adakọ na nlele ozugbo site na ụdị ebumnuche.
Kedu n'ime ihe ndị a bụ ụzọ 'ịkọ nkọ n'onwe' nke na-ezere usoro nhazi dị iche?
Medusa na EAGLE na-agbakwunye isi amụma amụma dị fechaa na ụdị isi ka o wee nwee ike depụta akara nke ya n'ọdịnihu.