Barlow Twins and Redundancy Rage
Barlow Twins hanya ce mai kulawa da kai wacce ke koyon wakilci ta hanyar yin ma'aunin haɗin kai tsakanin ra'ayoyi guda biyu da aka haɓaka kusa da matrix na ainihi.
Dubawa
It avoids collapse through a redundancy-reduction principle rather than negatives or momentum encoders.
Zurfafa nutsewa
Facebook AI ne ya gabatar da shi a cikin 2021 kuma mai suna H. Barlow's reundancy-reduction ka'idar, Barlow Twins yana ciyar da karkatattun ra'ayoyi guda biyu na hoto ta hanyar cibiyoyin sadarwa iri ɗaya don samar da batches biyu na abubuwan haɗawa. It computes the cross-correlation matrix between the components of these two embedding vectors, measured across the batch. Manufar ita ce ta tura wannan matrix zuwa ga ainihi: shigarwar diagonal ya kamata ya zama 1 (kowace siffa ba ta da bambanci ga haɓakawa) kuma shigarwar da ba ta dace ba yakamata ta zama 0 ( fasalulluka daban-daban suna da alaƙa da ado, rage raguwa). Kalmar kan-diagonal tana aiwatar da sabani; kalmar rage rage-diagonal a zahiri tana hana rushewa saboda abubuwan da ke da alaƙa ba za su iya zama iri ɗaya ba. Ba kamar BYOL ba yana buƙatar asymmetry, tsinkaya, ko tsayawa-gradient, kuma ba kamar SimCLR ba yana buƙatar nau'i-nau'i mara kyau, kodayake yana fa'ida daga manyan abubuwan haɓakawa.
Fahimtar Fasaha
Asarar tana da sassa biyu da aka tattara akan matrix-daidaitacce C: jimlar (1 - C_ii)^2 sharuɗɗan saɓani akan diagonal, da adadin lambda-nauyin C_ij^2 sharuddan redundancy off-diagonal. Saboda matrix ɗin an daidaita shi akan tsari, hanyar tana da ƙarfi da ƙarfi don girman tsari, fa'ida mai amfani akan hanyoyin banbance-banbance waɗanda ke yunwar manyan batches na rashin ƙarfi. Ma'auni na aiki tare da haɓaka girma, don haka majigi suna da faɗi sosai.
Dabarun Tasiri
Kudin da kasafin kuɗi
Hukunce-hukuncen gine-gine suna haifar da aiki da tsadar aiki na shekaru.
Shawarwari masu haske
Ilimin fasaha yana taimaka wa ƙungiyoyi su zaɓi tari mai kyau, ba kawai sabon abu ba.
Kula da inganci
Zaɓuɓɓukan injiniya mafi kyau suna rage abin dogaro a cikin samarwa.
Makomar Barlow Twins da Rage Ragewa
Barlow Twins sun taimaka wajen haifar da bayanan-ka'idar iyali na hanyoyin kulawa da kai, musamman VICReg, wanda ke raba bambance-bambance, saɓani, da sharuɗɗan haɗin kai a sarari. Yi tsammanin raguwa-raguwa da maƙasudin adon fasali don ci gaba da sanar da yadda muke horar da encoders waɗanda ke samar da ƙayyadaddun siffofi, waɗanda ba su da yawa, da kuma haɓaka sama da hotuna zuwa saitunan multimodal da jeri-lokaci inda aka yi ado, ƙaƙƙarfan wakilci na taimaka wa samfuran ƙasa su koyi daga ƙayyadaddun lakabi.
Aiwatar da Gaskiyar Duniya
Haɓaka maɓallan hoto waɗanda ke samar da fasalulluka na ado masu amfani don rarrabuwar ƙasa tare da ƙayyadaddun bayanai masu alaƙa.
Horarwa akan kayan aiki masu matsakaici inda manyan batches marasa amfani ba su da amfani, tunda Barlow Twins ba su da hankali.
Ƙirƙirar ƙaƙƙarfan, abubuwan da ba a sake yin amfani da su ba don tari ko gano ɓarna a cikin hoton firikwensin masana'antu.
Yin hidima azaman tushen kulawa da kai a cikin bincike kwatanta dabarun gujewa rugujewa a cikin SimCLR, BYOL, da VICReg.
Hatsari & Tsare-tsare
Haɓaka ma'auni ɗaya na iya ɓoye manyan raunin tsarin.
Sau da yawa ana raina kayan more rayuwa da kuma kuɗin kulawa.
Tsaro da gibin lura na iya girma yayin da tsarin ke ƙara haɓaka.
Taswirar Hanya
Ƙayyade latency, inganci, da maƙasudin farashi kafin aiwatarwa.
Alamar ma'auni a ƙarƙashin ainihin kaya da yanayin bayanai.
Kula da kayan aiki don kurakurai, ɗigo, da tasirin mai amfani.
Shirya bijirowa da hanyoyin mayar da martani kafin sikeli.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Barlow Twins and Redundancy Reduction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Sarkar-Tabbatarwa don Rage Hallucination
Tambayoyin da ake yawan yi
What is Barlow Twins and Redundancy Reduction?
Barlow Twins hanya ce mai kulawa da kai wacce ke koyon wakilci ta hanyar yin ma'aunin haɗin kai tsakanin ra'ayoyi guda biyu da aka haɓaka kusa da matrix na ainihi. Yana guje wa rugujewa ta hanyar ka'ida-raguwa-raguwa maimakon maras kyau ko masu saɓani.
Wane matrix Barlow Twins ke ƙoƙarin turawa zuwa ga ainihi?
Barlow Twins yana tafiyar da matrix ɗin haɗin kai tsakanin abubuwan haɗa ra'ayoyin biyu zuwa matrix na ainihi.
Menene shigarwar diagonal na matrix ɗin da aka yi niyya ke ƙarfafawa?
Shigar da diagonal kusa da 1 yana nufin kowane fasalin yana amsa iri ɗaya ga duka ra'ayoyi da aka haɓaka, yana tilasta rashin daidaituwa.
Menene abubuwan shigarwar da ake turawa zuwa sifili suke cim ma?
Sifili off-diagonal yana nufin fasali daban-daban suna da alaƙa da ado, waɗanda duka biyun suna rage raguwa kuma suna dakatar da rugujewa mara nauyi.
Ka'idar kimiyyar kwakwalwa ta wane ne ya karfafa sunan hanyar da tunaninsa?
Ana kiran hanyar ne bayan H. Barlow da ka'idodinsa na raguwa-rage na ingantaccen coding jijiyoyi.
Idan aka kwatanta da BYOL, menene Barlow Twins BAYA buƙata?
Barlow Twins suna amfani da cibiyoyin sadarwa iri ɗaya ba tare da tsinkaya ba, tsayawa-gradient, ko maƙasudin EMA, suna dogaro a maimakon raguwa-raguwa.