Ibikurikira-byavuzwe
Ibikurikira-token guhanura ni uburiganya bworoshye intego ya GPT yuburyo bwa moderi: ukurikije ibintu byose kugeza ubu, tekereza igice gikurikira cyinyandiko.
Incamake
Repeated billions of times, this single task produces models that write, reason, and converse.
Kwibira cyane
Ibikurikira-token guhanura bitoza icyitegererezo cyo gutanga ibishoboka kurikimenyetso gikurikira cyatanzwe ibimenyetso byose bibanziriza. Inyandiko ibanza gucikamo ibimenyetso (subword ibice) na tokenizer nka byte-couple kodegisi. Impinduramatwara-gusa Transformer isoma urukurikirane rwibumoso ujya iburyo kandi rusohora ibishoboka byo gukwirakwiza hejuru yamagambo yose kumwanya ukurikira. Mugihe cyamahugurwa, icyitegererezo cyerekanwe inyandiko nini corpora kandi igahanwa igihe cyose igenera amahirwe make kukimenyetso gikurikira. Mugihe cyibisekuru, icyitegererezo ntangarugero cyangwa umururumba utoranya ikimenyetso, ukagerekaho, kandi ugasubiramo iyi loop mu buryo bwikora. Umunzani umwe ufite intego: GPT-2, GPT-3, hamwe nabasimbuye bose bize ikibonezamvugo, ukuri, guhindura, no gutekereza gusa muburyo bwiza bwo guhanura ikimenyetso gikurikira.
Ubushishozi
Uburyo bwingenzi nimpamvu (mask) kwiyitaho: mugihe uhanura umwanya N, icyitegererezo gishobora kwitabira imyanya 1 kugeza kuri N-1, ntabwo bizaza. Ibisohoka byerekana umushinga wanyuma wihishe kumagambo kandi ugakoresha softmax kugirango ubone ibishoboka. Amahugurwa agabanya cross-entropy, bihwanye no kugabanya amahirwe menshi yinyandiko yagaragaye. Gutoranya kugenzura nkubushyuhe hamwe na top-p ivugurura ibyo gukwirakwiza hifashishijwe ubucuruzi bwo guhanga udushya.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Ejo hazaza h'ibikurikira-byavuzwe
Ibikurikira-token guhanura bishimangira cyane imiterere yindimi nini igezweho kandi bizakomeza kuba inkingi ya AI ibyara umusaruro. Ubushakashatsi burimo kuyagura hamwe na Windows ndende yerekana imiterere, gushushanya no kugereranya decode yihuta, hamwe nintego nyinshi zo guhanura zikeka ibimenyetso byinshi bizaza icyarimwe. Kwiga gushimangira uhereye kubitekerezo byabantu hejuru kugirango uhuze ibisubizo. Imipaka ikora intego imwe yoroheje ihendutse, yihuta, kandi irashobora kugenzurwa murwego runini.
Gushyira mu bikorwa Isi
Guha imbaraga ChatGPT hamwe nabafasha basa kugirango batange ibisubizo byibiganiro ikimenyetso kimwe icyarimwe.
Autocomplete na code ibyifuzo mubikoresho nka GitHub Copilot nkuko wandika.
Gutegura imeri, ingingo, hamwe na kopi yo kwamamaza uhereye mugihe gito.
Igihe-nyacyo cyo kwandika inyandiko mubafasha kwandika barangiza interuro yawe.
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Next-Token Prediction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Amahugurwa menshi yo guhanura
Ibibazo bikunze kubazwa
What is Next-Token Prediction?
Ibikurikira-token guhanura ni uburiganya bworoshye intego ya GPT yuburyo bwa moderi: ukurikije ibintu byose kugeza ubu, tekereza igice gikurikira cyinyandiko. Gusubiramo miriyari inshuro, iki gikorwa kimwe gitanga icyitegererezo cyandika, impamvu, no kuganira.
Niki gikurikira-ikimenyetso cyerekana ubuhanuzi gisohoka kuri buri ntambwe?
Icyitegererezo gitanga ibishoboka kuri buri kimenyetso gishoboka gikurikiraho, hanyuma imwe yatoranijwe ikoresheje icyitegererezo cyangwa umururumba.
Ni ubuhe bwoko bwo kwitondera GPT-yuburyo bwa decoder ikoresha?
Impamvu ya masking yemeza imyanya N ishobora kubona imyanya mbere yayo, ikabika ibumoso-iburyo.
Igisekuru 'autoregressive' gisobanura iki hano?
Autoregressive generation itanga ikimenyetso kimwe, ikongeramo imiterere, kandi igasubiramo ikizunguruka.
Ni ubuhe buryo bwo gutakaza busanzwe bugabanywa mugihe cy'amahugurwa?
Kwambukiranya-entropiya bihana icyitegererezo cyo gutanga amahirwe make ku kimenyetso gikurikiraho, bihwanye no kwerekana byinshi bishoboka.
Kuzamura ubushyuhe mugihe cyo gutoranya bikora iki?
Ubushyuhe bwo hejuru buringaniza ibishoboka, gukwirakwiza; ubushyuhe bwo hasi butuma amahitamo arengera.