Amụma Token na-esote
Amụma akara na-esote bụ ebumnobi dị mfe aghụghọ dị n'azụ ụdị ụdị GPT: enyere ihe niile dị ugbu a, chee akụkụ ederede na-esote.
Nchịkọta
Repeated billions of times, this single task produces models that write, reason, and converse.
Ime miri emi
Amụma akara na-esote na-azụ ihe atụ iji kenye ihe gbasara omume na token na-esote enyere akara niile bu ụzọ. A na-ebu ụzọ gbajie ederede ka ọ bụrụ token (iberibe obere okwu) site n'aka ihe tokenizer dị ka ngbanwe byte-pair. Onye ntụgharị naanị-ngbanwe na-agụ usoro aka ekpe gaa n'aka nri wee wepụta nkesa puru omume n'ime okwu niile maka ọnọdụ na-esote. N'oge ọzụzụ, ihe nlereanya a na-egosi nnukwu ederede corpora na ntaramahụhụ mgbe ọ bụla ọ na-ekenye ohere dị ala na akara na-esote n'ezie. N'oge ọgbọ, ihe nlereanya ma ọ bụ anyaukwu na-ewere akara, tinye ya, ma na-emegharị akaghị a na-akpaghị aka. Otu ebumnobi a tụrụ n'ụzọ dị ịrịba ama: GPT-2, GPT-3, na ndị nọchiri anya niile mụtara ụtọ asụsụ, eziokwu, ntụgharị asụsụ na ịtụgharị uche naanị site n'ịma nke ọma n'ibu amụma na-esote.
Nghọta nka nka
Usoro isi bụ ihe kpatara (masked) nlebara anya onwe onye: mgbe ị na-ebu amụma ọnọdụ N, ihe nlereanya ahụ nwere ike ịga naanị n'ọkwa 1 ruo N-1, ọ bụghị ọdịnihu. Ngwunye mmepụta na-arụpụta ọnọdụ zoro ezo ikpeazụ n'ime okwu ma tinye softmax iji nweta ihe gbasara omume. Ọzụzụ na-ebelata cross-entropy, dabara na iwelite ohere nke ederede hụrụ. Njikwa nlele dị ka ọnọdụ okpomọkụ na n'elu-p na-emegharị nkesa ahụ site n'ịzụta mmepụta ihe megide ntụkwasị obi.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Usoro ọrụ asụsụ nwere ike ịga ngwa ngwa n'achụghị nkwụsi ike.
Nweta na iru
Ọ na-agbasawanye ohere n'ofe asụsụ na ụdị nzikọrịta ozi.
Mkpebi doro anya
Ndị otu nwere ike itinyekwu oge na ikpe ebe akpaaka na-ejikwa nkwughachi.
Ọdịnihu nke amụma Token na-esote
Amụma akara na-esote na-akwado n'ụzọ bụ isi ụdị asụsụ ọgbara ọhụrụ niile ma ga-anọgide na-abụ ọkpụkpụ azụ nke AI na-emepụta ihe. Nchọpụta na-agbatị ya site na windo okirikiri dị ogologo, ntule na ngbanwe maka ọsọ, yana ebumnuche amụma ọtụtụ token nke na-eche ọtụtụ akara n'ọdịnihu n'otu oge. Nkwalite mmụta site na nzaghachi nzaghachi mmadụ n'elu iji kwado nsonazụ. Akụkụ ahụ na-eme ka otu ebumnobi dị mfe dị ọnụ ala, ngwa ngwa na nke a na-ejikwa ya n'ọ̀tụ̀tụ̀ buru ibu mgbe ọ bụla.
Mmejuputa n'ezie n'ụwa
Na-enye ike ChatGPT na ndị enyemaka yiri ya iji wepụta nzaghachi mkparịta ụka otu akara n'otu oge.
Mezue na ntinye koodu na ngwaọrụ dị ka GitHub Copilot ka ị na-apịpị.
Ịdepụta ozi-e, akụkọ, na nnomi ahịa site na obere ngwa ngwa.
Ọgbọ ederede n'ezie na ndị enyemaka ederede na-emecha ahịrịokwu gị.
Ihe ize ndụ & okporo ụzọ nche
Eziokwu ndị e chepụtara echepụta nwere ike jiri nwayọ tinye akụkọ, nkwado nkwado, ma ọ bụ nsonaazụ nyocha.
Mmetụta ngwa ngwa nwere ike ịmepụta nsonaazụ na-ekwekọghị ekwekọ n'ofe arịrịọ ndị yiri ya.
Enwere ike ikpughe data ederede nwere mmetụta ma ọ bụrụ na njikwa ohere adịghị ike.
Map mmejuputa
Kọwaa usoro mmepụta, ụda, na ụkpụrụ ịdịmma tupu ibugharị.
Weghachite nzaghachi site na isi mmalite ntụkwasị obi mgbe ọ bụla izi ezi dị mkpa.
Debe ebe nleba anya mmadụ maka mpụta dị elu.
Sochie ụkpụrụ ọdịda ma na-azụghachi mkpali ma ọ bụ usoro ọrụ mgbe niile.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Next-Token Prediction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ọzụzụ amụma amụma ọtụtụ Token
Ajụjụ a na-ajụkarị
What is Next-Token Prediction?
Amụma akara na-esote bụ ebumnobi dị mfe aghụghọ dị n'azụ ụdị ụdị GPT: enyere ihe niile dị ugbu a, chee akụkụ ederede na-esote. Ọtụtụ ijeri ugboro ugboro, otu ọrụ a na-emepụta ụdị ndị na-ede ihe, na-atụgharị uche, na mkparịta ụka.
Kedu ihe nlere amụma amụma na-esote token na-ewepụta n'ọkwa ọ bụla?
Ihe nlereanya a na-emepụta ihe gbasara nke puru omume maka akara ngosi ọ bụla ga-ekwe omume, mgbe ahụ, a na-ahọrọ otu site na nlele ma ọ bụ oke anyaukwu.
Kedu ụdị nlebara anya ka onye nrụpụta ụdị GPT na-eji?
Mkpuchi ihe kpatara na-eme ka ọnọdụ N nwere ike ịhụ ọnọdụ n'ihu ya, na-echekwa ntọlite amụma aka ekpe gaa aka nri.
Kedu ihe ọgbọ 'autoregressive' pụtara ebe a?
Ọgbọ na-agba ọsọ na-ewepụta otu akara, na-agbakwunye ya na gburugburu, ma na-emegharị akaghị.
Kedu ọrụ mfu a na-ebelatakarị n'oge ọzụzụ?
Cross-entropy na-ata ahụhụ ihe nlere anya maka ikenye ohere dị ala n'ezi akara na-esote, nke dabara na ịbawanye ohere.
Kedu ihe iwelite okpomọkụ n'oge nlele na-eme?
Igwe ọkụ dị elu na-eme ka nkesa nke puru omume, na-abawanye randomness; Okpomọkụ dị ala na-eme ka nhọrọ dịkwuo mkpa.