Ntụgharị egwu egwu na amụma Multi-Token
Ịtụ egwu egwu na amụma ọtụtụ token na-eme ka ọgbọ nlereanya asụsụ dị ngwa site na ịkọ ọtụtụ akara n'ọdịnihu n'otu oge ma nyochaa ha n'otu ngafe, kama ịmepụta otu akara n'otu oge.
Nchịkọta
They cut latency without changing the text the model would have written.
Ime miri emi
Ndozi autoregressive nkịtị na-adị ngwa ngwa n'ihi na akara ngosi ọ bụla chọrọ ngafe n'ihu zuru oke yana a na-emepụta akara n'otu n'otu, na-ahapụ ejighị GPU. Ndozi nleba anya na-edozi nke a na onye na-edepụta ihe dị ọnụ ala nke na-atụpụta ọnụ ọgụgụ nke akara akwụkwọ anamachọihe, nke nnukwu ihe nlere anya wee gosipụta ya n'otu aka ahụ; A na-anabata prefix ọ bụla dabara na ihe ebumnuche gaara ewepụta n'efu, a na-edozikwa ndakọrịta nke mbụ. Mgbasa egwu na-atụgharị na amụma ụdị ọtụtụ ụdị Medusa na-atụgharị onye na-edepụta ihe n'ime ihe nlereanya n'onwe ya: ndị isi amụma dị fechaa (ma ọ bụ iyi nke nleba anya) hapụ otu ihe atụ ma chepụta ma nyochaa, na-ezere ụdị usoro dị iche. N'ihi na nkwenye ziri ezi, nkesa mmepụta bụ otu na nhazi nhazi ọkọlọtọ, naanị ị ga-enweta 2 ruo 3 ugboro ole na ole usoro usoro.
Nghọta nka nka
Isi ihe bụ na ihe ntụgharị nwere ike ị nweta ọtụtụ ọkwa n'otu ngafe na-aga n'ihu dị ọnụ ala ka otu, ebe ọ bụ ebe nchekwa-bandwidth, ọ bụghị ịgbakọ, n'oge ngbanwe. Ọtụtụ ndị isi amụma na-ewepụta akara ndoro-ndoro anya maka ọnọdụ dị iche iche na-esote; A na-enyocha osisi ma ọ bụ usoro nke ndị na-aga ime ọnụ, na nnabata na-eji nlele ajụ (ma ọ bụ anyaukwu matching) ya mere akara ndị anabatara na-agbaso nkesa nkesa. Ogologo anabatara kwa nzọụkwụ na-ekpebi ọsọ ọsọ.
Mmetụta atụmatụ
Ọnụ ego na mmefu ego
Mkpebi ihe owuwu ụlọ na-akwalite arụmọrụ yana ọnụ ahịa ọrụ ruo ọtụtụ afọ.
Mkpebi doro anya
Nkà mmụta nka na-enyere ndị otu egwuregwu aka ịhọrọ nchịkọta ziri ezi, ọ bụghị naanị nke kachasị ọhụrụ.
Quality akara
Nhọrọ injinia ka mma na-ebelata ihe omume ntụkwasị obi na mmepụta.
Ọdịnihu nke gụgharia nlebara anya na amụma ọtụtụ Token
Usoro nke onwe nke na-achọghị ụdị usoro dị iche iche na-aghọ ihe ndabara na engines inference, na nyocha na-akwalite ọnụego nnabata dị elu na ndị isi akwụkwọ ka mma, ndị a na-ahazi osisi, na ịzụ ihe nlereanya ntọala maka ọnụ ọgụgụ dị ukwuu nke amụma amụma (nke nwekwara ike imeziwanye mma). Na-atụ anya ka usoro ndị a ga-ejikọta na quantization na batching ka ndị na-enyere aka na-emekọrịta ihe nwee mmetụta ozugbo ọbụlagodi ka ụdị na-etolite.
Mmejuputa n'ezie n'ụwa
Ibelata nzaghachi nzaghachi nke onye enyemaka nkata site na 2 ruo 3x site na iji isi amụma amụma ụdị Medusa.
Na-agbakwụnye nhọpụta nke onwe na ihe nkesa ntinye ka ọ nweghị ihe ngosi dị iche iche kwesịrị ịkwado
Na-eme ka mmecha koodu ọsọ ọsọ ebe a na-anabata ihe nrịbama a na-atụghị anya ya na nnukwu iberibe
Mbelata ọnụ ahịa GPU kwa arịrịọ site na ịwepụ ọtụtụ token na ngafe n'ichekwa ọ bụla
Ihe ize ndụ & okporo ụzọ nche
Ịkwalite otu akara ngosi nwere ike zoo adịghị ike sistemụ sara mbara.
A na-eledakarị ihe akụrụngwa na ụgwọ ọrụ anya.
Ọdịiche nchekwa na nleba anya nwere ike itolite ka sistemu na-adịwanye mgbagwoju anya.
Map mmejuputa
Kọwaa latency, ịdịmma na ebumnuche ọnụ ahịa tupu mmejuputa ya.
Benchmark n'okpuru ibu dị adị na ọnọdụ data.
Nleba anya akụrụngwa maka mperi, ịkpafu na mmetụta onye ọrụ.
Kwadebe ụzọ nzaghachi azụghachi azụ na ihe omume tupu ịchachaa.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Speculative Streaming and Multi-Token Prediction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ọzụzụ amụma amụma ọtụtụ Token
Ajụjụ a na-ajụkarị
What is Speculative Streaming and Multi-Token Prediction?
Ịtụ egwu egwu na amụma ọtụtụ token na-eme ka ọgbọ nlereanya asụsụ dị ngwa site na ịkọ ọtụtụ akara n'ọdịnihu n'otu oge ma nyochaa ha n'otu ngafe, kama ịmepụta otu akara n'otu oge. Ha na-egbutu latency na-agbanweghị ederede ihe nlereanya ahụ gaara ede.
Kedu ihe kpatara ngbanwe ọkọlọtọ autoregressive na-ejikarị ngwa ngwa na GPU?
Token nke ọ bụla chọrọ ngafe nke ya ma ha na-agakwa n'usoro, yabụ GPU na-ejikọta bandwit na ebe nchekwa na ejighị ya n'oge ngbanwe.
Kedu ihe 'drafter' na-eme n'ichepụta ntule?
Onye na-edepụta ihe dị ọnụ ala na-atụpụta ọnụ ọgụgụ nke akara ngosi nke nnukwu ihe nlere anya ahụ na-enyocha n'otu aka ahụ.
Kedu otu esi echekwa ogo mmepụta na ndopụta ntule?
Nnwale (nnwale anyaukwu dakọtara ma ọ bụ nleba anya ajụ) na-eme ka akara ngosi anabatara soro nkesa nke ụdị ebumnuche ga-arụpụta, yabụ na mmepụta agbanwebeghị.
Kedu ihe dị iche na amụma ọtụtụ akara ụdị ụdị Medusa na ngbanwe nhụsianya mara mma?
Ụzọ ụdị Medusa na-atụgharị n'ime ihe nlereanya nwere isi amụma ndị ọzọ, na-ezere mkpa ọ dị ịnabata ụdịdị dị iche iche.
Kedu ihe kpatara na transformer ga-eji nyochaa ọtụtụ ọkwa ndoro-ndoro anya ọ fọrọ nke nta ka ọ dị ọnụ ala ka nke a na-eme n'oge ngbanwe?
Mgbe a na-emezi ihe n'olu karama na-ebupụ ibu site na ebe nchekwa, yabụ ị nweta ọkwa n'otu ngafe ahụ na-agbakwunye obere ọnụ ahịa mgbakọ.