Normalisasioŋ bu kuréel gi
Normalisation groupe xarala la buy dakkal tàggat reso neuronal ci normalise ay man-man ci biir groupe chaine yu ndaw, misaal bu nekk moom boppam.
Résumé
It matters because, unlike Batch Normalization, it works well even when batches are tiny.
Plongeur bu xóot
Normalisation layers yi dañuy tëye nimero yi di jaar ci reso bu baax, loolu mooy gaawloo ak dakkal tàggat yaram. Batch Normalization dafay def loolu ci xayma moyenne ak variance bu bépp man-man ci biir mini-batch bi yépp, waaye loolu dafay tax mu yomba dagg sudee batch yi dañu tuuti, ndax lim yi dañuy bari bruit te wóoru ñu. Normalisation groupe, bi Wu ak He dugal ci 2018, dindi na batch bi ci equation bi lëmm. Ci misaal bu nekk, dafay xaaj chaine yi ay groupe yuñ fiks, ba noppi normalisee groupe bu nekk ci jëfandikoo valeur yu misaal boobu kese. Ndax xayma bi musul aju ci yeneen misaal yi ci batch bi, performance bi dafay des ci anam wu amul benn werante, muy batch bi yor 32 nataal wala benn, moo tax mu siiw ci gis, segmentation, ak liggéey yu diis ci memory.
Gis-gis xarala
Groupe Norm dafay xayma moyenne ak variance ci dimension spatial yi ak ci chaine yi ci biir groupe bu nekk, ci misaal bu nekk. Ginaaw loolu mu normalise ko ci nul moyenne ak variance unitaire ba noppi jëfandikoo echelu chaine bu nekk (gamma) ak shift (beta). Dafay yamale yeneen pexe: ak benn mbooloo mu nekk Normalisation Layer, ak benn chaine ci mbooloo mu nekk Normalisation Instance. Limu grupu yi dafay nuru hiperparametre, ñu koy faral di def ci 32.
njeextalu pexe
dogal yu gëna leer
Daf lay jàppale nga tàqale kàddu yu leer ci wàllu xarala ak làkku fësal njaay.
Njëgg ak budget
Mën nga laaj laaj yu gëna baax ci samp gi balaa ngay dugal xaalis wala sa jotu liggéey.
Ekip ak def liggéey
Ekip yi bokk xam-xam ñoo gëna mëna jël yenn dogal ci wàllu produit, politik ak jàng.
Ëlëgu Normalisation Groupe
Normalisation groupe mooy nekk tànneef bi gëna mag fépp fu lote yi wara nekk yu ndaw, lu melni gis-gis ak segmentation bu am resolusioŋ bu kawe, model 3D ak wideo, ak tàggat yaram bu yam ci mémoire. Dafa nekk itam ci architecture generatif yu ñuy jëfandikoo bu bari lu ci melni U-Nets ci biir xeetu diffusion. Bi model yi di màgg, te pression mémoire bi di wàññi tolluwaayu lots yi, normalisateurs yi moom seen bopp ci lots yi, Groupe Norm ci ñoom ci wetu Layer Norm, mën nañu des ci tabax yi ñuy tabax, ak gëstu yu wéy ci hybrid yi ak yeneen yu amul normalisation.
Doxal ci àdduna dëgg
Gis mbir ak xaaj misaal (lu melni, modelu stil Mask R-CNN) ñu tàggate ci ay lots yu ndaw lool ci GPU bu nekk.
Yaxu ndigg U-Net ci biir generatëru nataalu diffusion, foofu la Groupe Norm di dakkal balansu man-man yi.
Reseau 3D ak wideo yu jëfandikoo memory bu bari dañuy wàññi dayo batch yi ba benn wala ñaar.
Reglage xeetu gis-gis bu yaatu ci hardware bu néew, fu lots yu ndaw yi tax batch Norm amul benn wóolu.
Risk yi ak balustrade yi
Ekip yu bari mën nañu jëfandikoo benn baat ci anam wu wuute, kon teela leeral yaatuwaayam.
Benchmark yi mën nañu nuru lu am doole waaye performance yi ci àdduna bi duñu tolloo.
Bëgg kalite done ak palaŋu jàngat dafay faral di jur njariñ yu yomba dagg.
Roadmap ngir samp gi
Tàmbaleel ci joxe leeral ci làkk wu leer ci njariñ li nga soxla.
Tannal benn metric bu baax ak benn anam bu baaxul balaa ngay saytu.
Doxal ab pilote bu ndaw ak ay done yu representatif, du ab demo bu leer.
Dokumenteer fi Normalisation Groupe di jàppale ak fi pexe yu gëna yomba gëna baax.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Group Normalization quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Normalisation bu neexal buñ boole ci RLHF
Laaj yi ñuy faral di laaj
What is Group Normalization?
Normalisation groupe xarala la buy dakkal tàggat reso neuronal ci normalise ay man-man ci biir groupe chaine yu ndaw, misaal bu nekk moom boppam. Dafa am solo ndax wuute na ak Batch Normalization, dafay dox bu baax doonte ay lots dañu tuuti.
Lan mooy njariñu normalisation groupe ci normalisation batch?
Ndax Groupe Norm dafay xayma lim ci misaal bu nekk te du ci lote bi yépp, lote yu ndaw yi duñu ko yàq, te wuute na ak Batch Norm.
Ci lan la Normalisation Groupe xayma moyenne ak variance bi?
Dafay xaaj chaine yi ay groupe, ba noppi normalise groupe bu nekk ci jëfandikoo benn misaal bi ci valeur yi ak barab yi ci barab bi, baña bàyyi xel ci yeneen groupe yi.
Normalisation groupe nekk na normalisation layer ci ban anam wu amul fenn?
Ak benn groupe, chaine yépp dañuy boole normalise ci misaal, loolu mooy normalisation Layer.
Normalisation groupe nekk na normalisation instance ci ban anam wu amul fenn?
Sudee grupu bu nekk amna benn chaine, chaine bu nekk dafay normalise boppam ci misaal, mooy Normalisation Instance.
Ginaaw ñu normalisee, lan la Groupe Norm di jëfandikoo ngir defaraat neexaayu representation bi?
Groupe Norm ni yeneen layers normalisasioŋ yi dañuy àndaale ak eskaal yuñ mëna jàng ak paramet shift suko defee reso bi mëna dindi wala defar normalisasioŋ bi ni ko soxlaa.