RWKV Hankalin Litattafai
RWKV gine-gine ne wanda ke horarwa kamar Transformer amma yana gudanar da bincike kamar cibiyar sadarwa mai maimaitawa, yana ba da lokaci-lokaci, tsarar ƙwaƙwalwar ajiya akai-akai.
Dubawa
It reformulates attention so there is no quadratic cost and no growing key-value cache.
Zurfafa nutsewa
RWKV (lafazin 'RwaKuv') yana nufin Karɓa, Nauyi, Maɓalli, Ƙimar, ainihin abubuwan sa guda huɗu. An ƙirƙira shi ne a matsayin buɗaɗɗen aiki, aikin al'umma wanda Bo Peng ke jagoranta. Manufar ita ce a ci gaba da daidaiton horo na Transformers yayin da ake kawar da ra'ayinsu mai tsada. Daidaitaccen kulawa yana adana maɓalli mai ƙima wanda ke girma tare da kowane alama kuma yana kwatanta kowace sabuwar alama da duk waɗanda suka gabata. RWKV maimakon haka yana ɗaukar ƙaramar ƙayyadaddun ƙayyadaddun yanayin ɓoye gaba, yana sabunta shi tare da tsarin lalata lokaci don haka tsofaffin bayanai suna shuɗewa. A lokacin horo ana iya buɗe shi a cikin nau'i mai kama da juna; a lokacin tsara yana aiki azaman RNN yana samar da alama ɗaya a lokaci ɗaya akan farashi akai-akai. Wannan yana sa ya zama abin sha'awa ga dogon yanayi da iyakantaccen kayan aiki.
Fahimtar Fasaha
RWKV yana maye gurbin softmax dot-samfurin hankali tare da maimaita salo-salon hankali- madaidaiciya. Nauyin lalacewa na kowane tashoshi (W) wanda aka koya yana sarrafa yadda saurin maɓallan da suka shuɗe ke rasa tasiri, ƙofar karɓa (R) tana yanke shawarar adadin jihar da za a karanta, kuma maɓalli/daraja masu ƙima suna ciyar da adadi mai nauyi. Domin kowane mataki ya dogara ne kawai akan yanayin da ya gabata, ƙwaƙwalwar ajiya tana tsayawa tsayin daka kuma aikin kowace alama baya girma tare da tsayin jeri.
Dabarun Tasiri
Gudu da sikelin
Gudun aikin harshe na iya tafiya da sauri ba tare da sadaukar da daidaito ba.
Shiga ku isa
Yana faɗaɗa damar shiga cikin harsuna da salon sadarwa.
Shawarwari masu haske
Ƙungiyoyi za su iya ciyar da ƙarin lokaci akan hukunci yayin da aiki da kai ke sarrafa maimaitawa.
Makomar Hankalin Linear RWKV
RWKV ya ci gaba da sauri ta hanyar nau'ikan (v4, v5 Eagle, v6 Finch da bayan), yana rage gibin inganci tare da Masu Canjawa yayin kiyaye farashin layi. Yi tsammanin ci gaba da girma a cikin buɗaɗɗen nau'ikan harsuna da yawa, gefen da tura CPU inda akai-akai abubuwan ƙwaƙwalwar ajiya, da ƙirar ƙira. Cikakkun bayanin sa na maimaitawa yana sa ya zama ɗan takara mai ƙarfi don aikace-aikacen yawo da tsayin daka sosai inda maɓallan maɓalli na ƙima zasu fashe.
Aiwatar da Gaskiyar Duniya
Samfuran taɗi masu iya buɗewa akan CPUs ko ƙananan na'urorin ƙwaƙwalwar ajiya tare da ƙididdigan ƙwaƙwalwar ajiya kowace alama
Ƙirƙirar rubutu mai yawo inda ake samar da alamu ɗaya bayan ɗaya ba tare da girmar cache ba
Dogayen daftarin aiki inda maɓallin ƙimar maɓalli na Transformer zai zama haramun babba
Ayyukan ƙirar al'umma da harsuna da yawa waɗanda ke buƙatar ingantaccen gine-gine mai lasisi a bayyane
Hatsari & Tsare-tsare
Abubuwan da aka ruɗe suna iya shigar da rahotanni cikin nutsuwa, kwararar tallafi, ko abubuwan bincike.
Hankali na gaggawa na iya ƙirƙirar sakamako mara daidaituwa a cikin buƙatun iri ɗaya.
Za a iya fallasa bayanan rubutu mai ma'ana idan ikon samun dama yana da rauni.
Taswirar Hanya
Ƙayyade tsarin fitarwa, sautin, da ma'auni masu inganci kafin fitowa.
Amsa a ƙasa tare da amintattun tushe a duk lokacin da daidaito ya shafi mahimmanci.
Ajiye wurin binciken ɗan adam don abubuwan da ake samu masu girma.
Bibiyar tsarin gazawar kuma sake horar da tsokaci ko tafiyar aiki akai-akai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the RWKV Linear Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Hankalin Layi na Layi da Ƙwayoyin Ƙwararru
Tambayoyin da ake yawan yi
What is RWKV Linear Attention?
RWKV gine-gine ne wanda ke horarwa kamar Transformer amma yana gudanar da bincike kamar cibiyar sadarwa mai maimaitawa, yana ba da lokaci-lokaci, tsarar ƙwaƙwalwar ajiya akai-akai. Yana sake fasalin hankali don haka babu farashi mai ƙididdigewa kuma babu haɓakar maɓalli mai ƙima.
Menene babban kanun labarai na RWKV?
Manufar ƙira ta RWKV ita ce horon nau'in Transformer iri ɗaya haɗe tare da na yau da kullun, ƙimar farashi.
Menene haruffa a cikin RWKV suka tsaya ga?
Ana kiran RWKV ne bayan abubuwan da ke tattare da shi guda hudu: Karɓa, Nauyi, Maɓalli, da Ƙimar.
Ta yaya RWKV ke kiyaye ƙwaƙwalwar ajiya akai yayin tsarawa?
Kamar RNN, RWKV yana sabunta ƙayyadadden matsayi kowane mataki, don haka ƙwaƙwalwar ajiya ba ta girma da tsayin jeri.
Wace rawa nauyin ruɓewar lokaci (W) ke takawa?
Nauyin lalata kowane tashoshi da aka koya yana ƙayyade yadda saurin maɓallan da suka shuɗe ke shuɗewa a cikin yanayin gudu.
Idan aka kwatanta da hankali na softmax, menene komawar jan hankali na layin RWKV ke gujewa?
Ta amfani da maimaituwa, RWKV yana kwatanta kowane alama da kowace alama, cire farashi mai ƙima.