Ubuyobozi bwa tekiniki

Ubusanzwe

Ubusanzwe urwego rushyiraho amahugurwa mugukuraho ibikorwa muri buri rugero rwihariye kuburyo bafite zeru zeru nibice bitandukanye.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Ejo hazaza h'ibisanzwe
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Nibintu bituje ariko byingenzi bituma impinduka zimbitse zitozwa.

Kwibira cyane

Yatangijwe na Ba, Kiros, na Hinton mu mwaka wa 2016, ubusanzwe (LayerNorm) bukemura ikibazo ko ibikorwa biri mumurongo wimbitse bishobora gutemba ku munzani itandukanye kuko ibimenyetso binyura mubice byinshi, bigatinda cyangwa bigahagarika imyigire. Bitandukanye nicyiciro gisanzwe, gisanzwe buri kintu murugero muri mini-batch, LayerNorm isanzwe kurwego rwibintu bimwe. Ibi bituma itigenga kubunini bwicyiciro kandi ikoreshwa kimwe mumahugurwa no gufata umwanzuro, kandi ikora muburyo busanzwe hamwe nuburebure bwikurikiranya, niyo mpamvu yabaye ihame rya transformateur ikoresha imiterere yindimi zigezweho. Nyuma yo gukora ibisanzwe, ikoresha igipimo cyiga (gamma) na shift (beta) kugirango urusobe rushobore kugarura ibyo rukeneye byose.

Gushyira mu bikorwa Isi

Gutezimbere buri gihinduramatwara muburyo bwururimi nka GPT na BERT.

Gushoboza RMSNorm nkuburyo bworoshye bwo guhitamo imbere muri moderi ya Llama-umuryango.

Guhindura ibipimo-birebire bikurikirana amakuru mumvugo hamwe nubusemuzi aho ingano yicyiciro itandukanye.

Emerera imyitozo yizewe hamwe nubunini bwicyiciro kimwe, nko mubice bimwe byo gushimangira imyigire.

Ejo hazaza h'ibisanzwe

Ubusanzwe burimo kunozwa kugirango bikore neza murwego. RMSNorm yasimbuye ahanini LayerNorm muburyo bushya bw'ururimi runini kuko bihendutse kandi ikora kimwe, kandi gushyira-pre-norm isanzwe ubu nibisanzwe kubintu byimbitse. Abashakashatsi bakomeje gushakisha inyubako zidafite ubusanzwe zikoresha uburyo bwitondewe bwo gutangiza cyangwa gupima amayeri aho, bagamije guca hejuru mugihe bagumya guhugura imyitozo isanzwe itanga.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the Layer Normalization quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Ubusanzwe ni iki?

Ubusanzwe urwego rushyiraho amahugurwa mugukuraho ibikorwa muri buri rugero rwihariye kuburyo bafite zeru zeru nibice bitandukanye. Nibintu bituje ariko byingenzi bituma impinduka zimbitse zitozwa.

Hirya no hino kurwego rusanzwe rugereranya iki kandi rutandukanye?

LayeriNorm isanzwe hejuru yimiterere yikigereranyo cyurugero rumwe, bigatuma itigenga nizindi ngero mugice.

Ni ukubera iki ibisanzwe bisanzwe bikundwa kuruta icyiciro gisanzwe muri transformateur?

Kuberako imibare yacyo ituruka kurugero rumwe, LayerNorm yitwara neza ititaye ku bunini bwicyiciro kandi ikwiranye nimpinduka-ndende yinyandiko ikurikiranye.

Niki ibipimo byiga gamma na beta reka LayerNorm ikora?

Nyuma yo gukora ibisanzwe kuri zeru bisobanura no gutandukanya ibice, umunzani wa gamma na beta bihindura ibisubizo kugirango moderi idahatirwa kugabanwa kugenwe.

Nigute RMSNorm itandukaniye na Layeri isanzwe?

RMSNorm ikuraho intambwe igereranya intambwe nubunzani ukoresheje imizi-bivuze-kare-ya-ibikorwa, bihendutse kandi bikoreshwa muri moderi nka Llama.

Ni ikihe kibazo gisanzwe gisanzwe gifasha gukemura mumiyoboro yimbitse?

Nkuko ibimenyetso binyura mubice byinshi igipimo cyabo gishobora guturika cyangwa kugabanuka; ibisanzwe bikomeza ibikorwa murwego ruhamye kugirango gradients yitware neza.