HejuruUbuyobozi bukurikira
LLM Yerekana Inzira no Kuringaniza Umutwaro
Tekiniki
Ubuyobozi bwa tekiniki
Kwinjira no kunyura reka urusobe rw'imitsi rukora gusa ibice rukeneye kuri buri cyinjiriro aho gukora moderi yose igihe cyose.
Umva iki gitaboAI-byavuzwe · ~2 min
Isohora ingano yicyitegererezo uhereye kubiciro bya compte, igafasha moderi nini ziguma byihuse kandi bihendutse gukora.
Kubara ibintu bisobanuye bisobanura urusobe rufata ibyemezo-bishingiye ku makuru yerekeranye na sub-module yo gukoresha. Umuyoboro muto wize 'gating' cyangwa 'router' umuyoboro ureba buri cyinjijwe (akenshi buri kimenyetso) kandi gitanga amanota ahitamo 'abahanga' kubyohereza. Mu ruvange-rw-Impuguke (MoE), urwego rwinzobere cyangwa amagana rwinzobere zirahari, ariko router itoranya imwe yambere cyangwa ibiri hejuru yikimenyetso, kubwibyo abahanga benshi baguma badafite icyo bakora kubitekerezo byose byatanzwe. Igisubizo nicyitegererezo gifite umubare munini wibipimo bibarwa ariko bito bikora, bitanga imbaraga zo guhagararira moderi nini ku giciro cyo gukora cyoroheje cyane. Nuburyo moderi nka Hindura Transformer, GLaM, hamwe nimbibi nyinshi zururimi rwimbibi zingana kugeza kuri trillioni yibipimo bihendutse.
Guhindura Transformer iyobora buri kimenyetso kumuhanga umwe, gipima hejuru ya tiriyari imwe mugihe ugumije kuri token compte hasi.
Imipaka minini yururimi rwimbere ukoresheje Uruvange-rw-Impuguke kuburyo igice gusa cyibipimo bikora kuri buri kimenyetso.
Gusohora hakiri kare amashusho atondekanya ahagarara murwego ruto kugirango amashusho yoroshye kandi akore byimbitse gusa kubikomeye.
Indimi nyinshi Indimi zifite inzira zo kohereza ibimenyetso biva mu ndimi zitandukanye kubahanga batandukanye.
Irembo rito ubu ni ingenzi cyane mu kwerekana imipaka yerekana imipaka, kandi icyerekezo ni icyerekezo cyinzobere nziza, router nziza, hamwe no kunyura mubice byinshi. Tegereza tekiniki nziza zamahugurwa ahamye, kugabanya itumanaho hejuru mugihe impuguke zikwirakwijwe kwihuta, hamwe nisesengura ryinzobere kugirango wumve icyo buri mpuguke yiga. Ibarura risabwa naryo rikwirakwira kurenga MoE mumiyoboro yo gusohoka hakiri kare na dinamike-yimbitse ikoresha compte nyinshi gusa kubintu byinjira cyane.
Ibaruwa y'amakuru y'ubuntu
Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.
Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.
Isuzume
Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.
Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano
Kwinjira no kunyura reka urusobe rw'imitsi rukora ibice rukeneye kuri buri cyinjiriro aho gukora moderi yose igihe cyose. Iyi decuples ingano yicyitegererezo uhereye kubiciro bya compte, ituma moderi nini ziguma vuba kandi zihendutse gukora.
The guide defines data-dependent selection of submodules as conditional computation.
The guide says a learned router selects experts based on its input.
Sparse routing can activate only part of the total parameter set per input.
The guide explains that most experts can remain idle for a particular token.
The guide notes routers can favor a few experts, leaving others underused.
Komeza wige
Abandi bayobozi batoranijwe kuriyi ngingo
HejuruUbuyobozi bukurikira
LLM Yerekana Inzira no Kuringaniza Umutwaro
Tekiniki