RWKV Itondekanya
RWKV ni imyubakire itoza nka Transformer ariko ikoresha umwanzuro nkurusobe rusubirwamo, rutanga umurongo-umwanya, uhoraho-wibuke.
Incamake
It reformulates attention so there is no quadratic cost and no growing key-value cache.
Kwibira cyane
RWKV (bisobanurwa ngo 'RwaKuv') bisobanura Kwakira, Uburemere, Urufunguzo, Agaciro, ibintu bine byingenzi. Yakozwe ahanini nkumushinga ufunguye, uyobowe nabaturage bayobowe na Bo Peng. Intego nugukomeza guhuza imyitozo ya Transformers mugihe ukuraho umwanzuro wabo uhenze. Ibitekerezo bisanzwe bibika urufunguzo-agaciro cache ikura hamwe na buri kimenyetso kandi igereranya buri kimenyetso gishya nicyabanjirije. RWKV ahubwo itwara akantu gato gafite ubunini-bwihishe leta imbere, kuyivugurura hamwe nigihe cyo kubora kugirango amakuru ashaje ashire neza. Mugihe cyamahugurwa irashobora gufungurwa muburyo bubangikanye; mugihe cyibisekuruza ikora nka RNN itanga ikimenyetso kimwe icyarimwe kubiciro bihoraho. Ibi bituma bikurura ibintu birebire hamwe nibikoresho bitagabanijwe.
Ubushishozi
RWKV isimbuza softmax akadomo-ibicuruzwa byitondewe hamwe numurongo-wo-kwitondera-uburyo bwo kwisubiramo. Wize kuri buri muyoboro umwanya-wangirika (W) ugenzura uburyo urufunguzo rwashize rwihuta gutakaza imbaraga, irembo ryakira (R) rihitamo umubare wigihugu cyegeranijwe cyo gusoma, kandi urufunguzo / agaciro rugaburira ibiryo biremereye. Kuberako buri ntambwe iterwa gusa na reta ibanza, kwibuka bigumaho kandi akazi kuri token ntigakura hamwe nuburebure bwikurikiranya.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
Kugera no kugera
Yagura uburyo bwindimi nuburyo bwo gutumanaho.
Ibyemezo bisobanutse
Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.
Kazoza ka RWKV Umurongo Witondewe
RWKV yagarutse byihuse binyuze muri verisiyo (v4, v5 Eagle, v6 Finch na nyuma yayo), igabanya ikinyuranyo cyiza hamwe na Transformers mugihe ikomeza igiciro cyumurongo. Witege ko uzakomeza gukura muburyo bwo gufungura indimi nyinshi, impande zose hamwe na CPU aho buri gihe bibuka ibintu byingenzi, hamwe n'ibishushanyo mbonera. Umwanzuro wacyo usubiramo rwose bituma uba umukandida ukomeye mugutondekanya porogaramu hamwe nigihe kirekire cyane aho urufunguzo-agaciro cashe ubundi yaturika.
Gushyira mu bikorwa Isi
Gukoresha ubushobozi bwo gufungura-isoko yo kuganira kuri CPU cyangwa ibikoresho-byibukwa buke hamwe nububiko buhoraho kuri buri kimenyetso
Streaming text generation aho ibimenyetso bikorerwa icyarimwe icyarimwe nta cache ikura
Gutunganya inyandiko ndende aho Transformer urufunguzo-agaciro cache yaba nini cyane
Umuganda hamwe nindimi nyinshi zicyitegererezo zikeneye imyubakire ikora neza, yemewe kumugaragaro
Ingaruka & Kurinda
Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.
Igishushanyo mbonera
Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
Komeza kugenzura abantu kugenzura ibisubizo byinshi.
Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the RWKV Linear Attention quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Imirongo Yitondewe hamwe nudukoryo
Ibibazo bikunze kubazwa
What is RWKV Linear Attention?
RWKV ni imyubakire itoza nka Transformer ariko ikoresha umwanzuro nkurusobe rusubirwamo, rutanga umurongo-umwanya, uhoraho-wibuke. Ivugurura ibitekerezo kugirango hatabaho ikiguzi cya quadratic kandi nta terambere ryingenzi-agaciro cache.
Nuwuhe mutwe wumutwe wa RWKV?
Intego yo gushushanya ya RWKV irasa na Transformer-yuburyo bwamahugurwa hamwe nibisubirwamo, guhora-ibiciro.
Inyuguti ziri muri RWKV zisobanura iki?
RWKV yitiriwe ibice bine byayo: Kwakira, Uburemere, Urufunguzo, nagaciro.
Nigute RWKV ituma kwibuka bihoraho mugihe cyibisekuru?
Kimwe na RNN, RWKV ivugurura imiterere-yubunini ihagaze buri ntambwe, bityo kwibuka ntibikura hamwe nuburebure bwikurikiranya.
Ni uruhe ruhare uburemere bwangirika (W) bugira?
Kwiga kuri buri muyoboro kubora byerekana uburyo urufunguzo rwashize rugenda rwangirika muri reta ikora.
Ugereranije na softmax kwitabwaho, ni iki umurongo wa RWKV wongeye kwirinda?
Ukoresheje gusubiramo, kuruhande rwa RWKV ugereranya buri kimenyetso nikindi kimenyetso, ukuraho igiciro cya quadratic.