Gaba-Token Hasashen
Hasashen alama na gaba shine maƙasudin sauƙi mai yaudara a bayan ƙirar salon GPT: idan aka ba da komai ya zuwa yanzu, yi tsammani gunkin rubutu na gaba.
Dubawa
Repeated billions of times, this single task produces models that write, reason, and converse.
Zurfafa nutsewa
Hasashen alama na gaba yana horar da ƙira don sanya yuwuwar zuwa alama ta gaba da aka ba duk alamun da suka gabata. An fara karya rubutu zuwa alamomi (yanayin ƙananan kalmomi) ta hanyar tokenizer kamar ɓoyayyen byte-pair. Mai canzawa-kawai Mai Canjawa yana karanta jerin hagu zuwa dama kuma yana fitar da rabon yuwuwar akan duk ƙamus na matsayi na gaba. A yayin horo, ana nuna samfurin babban haɗin gwiwar rubutu kuma ana azabtar da shi a duk lokacin da ya sanya ƙarancin yuwuwar zuwa ainihin alama ta gaba. A lokacin tsarawa, ƙirar samfuri ko zari suna ɗaukar alama, saka shi, kuma ta maimaita wannan madauki ta atomatik. Wannan haƙiƙa guda ɗaya tana da ma'auni mai ma'ana: GPT-2, GPT-3, da magada duk sun koyi nahawu, gaskiya, fassarar, da tunani zalla ta hanyar samun ƙwarewa sosai wajen tsinkayar alama ta gaba.
Fahimtar Fasaha
Maɓalli mai mahimmanci shine dalili (masked) hankalin kai: lokacin tsinkayar matsayi N, ƙirar na iya halartar matsayi 1 zuwa N-1 kawai, ba gaba ɗaya ba. Layin fitarwa yana aiwatar da yanayin ɓoye na ƙarshe akan ƙamus kuma yana amfani da softmax don samun yuwuwar. Horon yana rage girman giciye-entropy, daidai da haɓaka yuwuwar rubutun da aka lura. Samfuran sarrafawa kamar zafin jiki da sake fasalin sama-p waccan rarraba bisa ga ra'ayi don cinikin kashe kerawa akan dogaro.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Hasashen-Token na gaba
Hasashen alama na gaba yana tallafawa da gaske duk manyan yare na zamani kuma zai kasance ƙashin bayan AI. Bincike yana tsawaita shi tare da tagogin mahallin mai tsayi, hasashe da daidaitawa don saurin gudu, da maƙasudin tsinkaya iri-iri waɗanda ke hasashen alamu da yawa na gaba lokaci guda. Ƙarfafa koyo daga matakan martani na ɗan adam a sama don daidaita abubuwan da aka fitar. Iyakar tana sanya manufa mai sauƙi iri ɗaya mai rahusa, sauri, da ƙarin sarrafawa a mafi girman sikeli.
Aiwatar da Gaskiyar Duniya
Ƙaddamar da ChatGPT da makamantan mataimakan don samar da martanin tattaunawa alama ɗaya a lokaci guda.
Cika atomatik da shawarwarin lamba a cikin kayan aikin kamar GitHub Copilot yayin da kuke bugawa.
Zana imel, labarai, da kwafin tallace-tallace daga ɗan gajeren lokaci.
Ƙirƙirar rubutu na ainihi a cikin mataimakan rubuce-rubuce waɗanda suka gama jimlolin ku.
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Next-Token Prediction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Horon Hasashen Multi-Token
Tambayoyin da ake yawan yi
What is Next-Token Prediction?
Hasashen alama na gaba shine maƙasudin sauƙi mai yaudara a bayan ƙirar salon GPT: idan aka ba da komai ya zuwa yanzu, yi tsammani gunkin rubutu na gaba. Maimaita biliyoyin sau, wannan ɗawainiyar guda ɗaya tana samar da samfura waɗanda ke rubutu, tunani, da tattaunawa.
Menene fitar da samfurin tsinkayar alamar gaba a kowane mataki?
Samfurin yana samar da yuwuwar ga kowane alama na gaba mai yiwuwa, sannan an zaɓi ɗaya ta hanyar samfur ko zaɓe na zari.
Wani nau'in kulawa ke amfani da na'urar dikodi irin ta GPT?
Makullin dalili yana tabbatar da matsayi N kawai zai iya ganin matsayi a gabansa, yana adana saitin hasashen hagu-zuwa-dama.
Me ake nufi da tsara 'autoregressive' a nan?
Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwararren Ƙwaƙwalwa na Ƙaƙwalwa na Ƙaƙwalwa na Ƙaƙwalwa ) ya yi, yana ƙara shi zuwa mahallin, kuma yana maimaita madauki.
Wane aikin asara ne yawanci aka rage yayin horo?
Cross-entropy yana azabtar da ƙima don sanya ƙarancin yuwuwar ga alama ta gaba ta gaskiya, daidai da haɓaka yuwuwar.
Menene haɓaka zafin jiki yayin yin samfur yake yi?
Maɗaukakin zafin jiki yana ƙaddamar da rarraba yiwuwar, ƙara bazuwar; ƙananan zafin jiki yana sa zaɓin ya fi mazan jiya.