Ọzụzụ amụma amụma ọtụtụ Token
Kama ịkọ naanị akara na-esote, a zụrụ ihe nlereanya ahụ ịkọ ọtụtụ akara n'ọdịnihu n'otu oge.
Nchịkọta
This sharpens learning signals and unlocks faster inference through self-speculative decoding.
Ime miri emi
A na-eji amụma akara na-esote zụọ ụdị asụsụ ọkọlọtọ: nyere nkọwa, buru amụma otu akara na-esote. Multi-token amụma (MTP), nke akwụkwọ 2024 Meta na-ewu ewu ma nakweere na DeepSeek-V3, na-agbakwunye isi mmepụta dị fechaa ka ihe nlereanya ahụ n'otu oge na-ebu amụma akara na-esote gbakwunyere 2nd, 3rd, and 4th tokens n'ihu site na otu ọnọdụ zoro ezo. Nke a na-amanye netwọk ahụ ime atụmatụ n'ihu n'ọdịnihu ma mebie mgbaàmà ọzụzụ - ọnọdụ ọ bụla na-enye ugbu a ọtụtụ okwu ọnwụ. Meta kọrọ ọkachasị nnukwu uru na ntinye koodu na ntụgharị uche, yana ụdị buru ibu na-erite uru karịa. N'ụzọ dị mkpa, enwere ike tụfuo isi ndị ọzọ mgbe ọzụzụ gasịrị, yabụ nha nlereanya na mbugharị ekwesịghị itolite.
Nghọta nka nka
MTP attaches n nnwere onwe amụma isi n'elu nke òkè transformer ogwe; isi k na-ebu amụma akara na ọnọdụ t+k site na nnọchite anya na ọnọdụ t. A na-agbakọta mfu ndị ahụ n'oge ọzụzụ. Na ntinye, ndị isi inyeaka na-eme ka nhọpụta nke onwe: ihe nlereanya ahụ na-atụ aro ọtụtụ tokens n'otu ngafe, wee nyochaa ha, na-enweta ihe ruru ọgbọ 3x ngwa ngwa na-enweghị ịgbanwe nkesa mmepụta.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke Ọzụzụ amụma amụma Multi-Token
MTP na-aghọ ihe ndabere na usoro ọzụzụ ọzụzụ n'ihu n'ihi na ọ na-eme ka ọ dị mma yana ọsọ inference na obere ọnụ ahịa. Na-atụ anya mwekota siri ike site na nhọpụta ntule, amụma amụma miri emi, ma jiri dị ka ebumnuche inyeaka na-eme ka atụmatụ ogologo oge dịkwuo mma. Ejikọtara ya na ụdị echiche, ịkọ ọtụtụ nzọụkwụ dị n'ihu nwere ike inyere ụdị aka ime ka ọ pụta ìhè tupu ha azaa azịza.
Mmejuputa n'ezie n'ụwa
DeepSeek-V3 na-eji ebumnobi MTP n'oge ọzụzụ ọzụzụ iji kwalite arụmọrụ data yana mee ka nhọpụta ntule.
Ụdị koodu ọgbọ nke Meta na-egosi uru ziri ezi na HumanEval na MBPP site na ịkọ ọtụtụ token.
Ndozi nke onwe: na-edepụta akara ngosi 3-4 n'otu ngafe wee nyochaa ngwa ngwa, na-echekwa nkesa.
Mezue ngwa ngwa n'ime ndị enyemaka koodu ebe a na-atụpụta ọtụtụ akara ngosi ma lelee ya n'otu nzọụkwụ.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Multi-Token Prediction Training quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ntụgharị egwu egwu na amụma Multi-Token
Ajụjụ a na-ajụkarị
What is Multi-Token Prediction Training?
Kama ịkọ naanị akara na-esote, a zụrụ ihe nlereanya ahụ ịkọ ọtụtụ akara n'ọdịnihu n'otu oge. Nke a na-amụba akara mmụta ma na-emeghe ntinye ngwa ngwa site na ngbanwe nke onwe.
Kedu ihe amụma ọtụtụ token na-agbakwunye ma e jiri ya tụnyere ọzụzụ ọkọlọtọ na-esote?
MTP na-agbakwụnye isi mmepụta ndị ọzọ ka ihe nlereanya na-ebu amụma akara na-esote yana ọtụtụ akara n'ihu n'ihu site na otu ọnọdụ zoro ezo.
Kedu ụdị nke ejiri ebumnobi amụma ọtụtụ token mee ihe n'ịzụ ụzọ?
DeepSeek-V3 nakweere ebumnobi ọzụzụ MTP iji melite arụmọrụ data yana mee ka ngbanwe nke nkọlọ.
Kedu ihe kpatara MTP ji na-agbagha mgbama ọzụzụ ahụ?
Ịma amụma ọtụtụ akara n'ọdịnihu pụtara ọnọdụ akara ọ bụla na-arụpụta ọtụtụ mfu amụma, na-enyekwu akara mmụta mmụta kwa akara ngosi.
Kedu uru inference ndị isi amụma ndị ọzọ na-enyere?
Ndị isi inyeaka nwere ike ịdepụta ọtụtụ tokens kwa ngafe nke enyochaziri, na-eme ngwa ngwa ọgbọ na-agbanweghị nkesa mmepụta.
Kedu ihe na-eme ndị isi inyeaka mgbe ọzụzụ gasịrị ma ọ bụrụ na ịchọghị ọsọ ọsọ?
Isi ndị agbakwunyere bụ nhọrọ na ntinye; ịtụfu ha na-edobe ihe nlereanya otu nha ka ọkọlọtọ na-esote akara ngosi.