Amawele e-Barlow kanye Nokwehliswa Kwe-Redundancy
I-Barlow Twins iyindlela ezigadwayo efunda izethulo ngokwenza i-matrix yokuhlobana phakathi kokubuka okubili okungeziwe eduze kwe-matrix yobunikazi.
Uhlolojikelele
It avoids collapse through a redundancy-reduction principle rather than negatives or momentum encoders.
I-Deep Dive
Ihlongozwe i-Facebook AI ngo-2021 futhi yaqanjwa ngesazi sezinzwa u-H. Barlow isimiso sokunciphisa ukungasasebenzi, i-Barlow Twins iphakela ukubukwa okubili okuhlanekezelwe kwesithombe ngamanethiwekhi afanayo ukuze kukhiqizwe amaqoqo amabili okushumeka. Ibala i-matrix yokuhlobana phakathi kwezingxenye zalawa ma-vector amabili ashumekayo, akalwa kuyo yonke inqwaba. Injongo iphushela le matrix kubunikazi: okufakiwe okudayagonali kufanele kube ngu-1 (isici ngasinye asiguquki ekwandiseni) futhi okufakiwe okungaphandle kwediagonal kufanele kube ngu-0 (izici ezihlukile zihlotshisiwe, zehlisa ukuphindaphindeka). Itemu eliku-diagonal liphoqelela ukuguquguquka; igama lokunciphisa ukuphindaphindeka kwe-off-diagonal ngokwemvelo livimbela ukuwa ngoba izici ezihlotshisiwe azikwazi ukufana zonke. Ngokungafani ne-BYOL ayidingi i-asymmetry, i-predictor, noma i-stop-gradient, futhi ngokungafani ne-SimCLR ayidingi amapheya angalungile, nakuba izuza ekushumekeni okuphezulu.
I-Technical Insight
Ukulahlekelwa kunezingxenye ezimbili ezifingqwe phezu kwe-matrix yokuhlobana kwesiphambano C: isamba esingu-(1 - C_ii)^2 amagama angaguquki ku-diagonal, kanye nesamba esinesisindo se-lambda samatemu angu-C_ij^2 wokungaguquki kwe-diagonal. Ngenxa yokuthi i-matrix ijwayele ukudlula inqwaba, indlela iqine ngokwanele kusayizi weqoqo, inzuzo engokoqobo kunezindlela ezihlukile ezilambele amaqoqo amakhulu okuphikisa. Izikali zokusebenza ezinobukhulu bokushumeka, ngakho amaphrojektha avamise ukuba banzi kakhulu.
I-Strategic Impact
Izindleko kanye nesabelomali
Izinqumo zezakhiwo ziqhuba ukusebenza kanye nezindleko zokusebenza iminyaka.
Izinqumo ezicacile
Imfundo yobuchwepheshe isiza amaqembu ukuthi akhethe isitaki esifanele, hhayi nje esisha.
Ukulawulwa kwekhwalithi
Izinketho ezingcono zobunjiniyela zinciphisa izehlakalo ezinokwethenjelwa ekukhiqizeni.
Ikusasa Lamawele E-Barlow kanye Nokwehliswa Kwe-Redundancy
Amawele ka-Barlow asize ukuvusa umndeni wethiyori yolwazi wezindlela zokuzigada, ikakhulukazi i-VICReg, ehlukanisa amatemu ahlukene, okungaguquki, kanye ne-covariance ngokucacile. Lindela izinjongo zokunciphisa ukungasasebenzi kanye nokuhlotshiswa kwesici ukuze uhlale ukwazisa indlela esiqeqesha ngayo kusengaphambili izishumeki ezikhiqiza izici ezihlangene, ezingadideki, futhi unwebe ngalé kwezithombe ziye kuzilungiselelo ze-multimodal kanye nochungechunge lwesikhathi lapho ukumelwa okuhlotshisiwe, okuqinile kusiza amamodeli angezansi afunde kumalebula alinganiselwe.
Ukuqaliswa Komhlaba Wangempela
Ukuqeqesha kusengaphambili izifaki khodi zezithombe ezikhiqiza izici ezihlotshisiwe ziwusizo ekuhlukaniseni umfula ongezansi ngedatha enelebula elinganiselwe.
Ukuqeqeshwa kwezingxenyekazi zekhompuyutha ezimaphakathi lapho amaqoqo amakhulu angalungile engenakwenzeka, njengoba i-Barlow Twins inosayizi we-batch uma kuqhathaniswa ayizwani.
Ikhiqiza ukushumeka okuhlangene, okungadingekile kokuhlanganisa noma ukutholwa okudidayo emfanekisweni wenzwa yezimboni.
Isebenza njengesisekelo esizigadayo ocwaningweni oluqhathanisa namasu okugwema ukuwohloka kuyo yonke i-SimCLR, BYOL, ne-VICReg.
Izingozi & Guardrails
Ukuthuthukisa ibhentshimakhi eyodwa kungafihla ubuthakathaka obubanzi besistimu.
Izindleko zengqalasizinda nezokulungisa zivame ukubukelwa phansi.
Izikhala zokuphepha nokubonakala zingakhula njengoba izinhlelo ziba nzima kakhulu.
Ukuqalisa Umhlahlandlela
Chaza ukubambezeleka, ikhwalithi, nezindleko ezihlosiwe ngaphambi kokuqaliswa.
Ibhentshimakhi ngaphansi komthwalo wangempela nezimo zedatha.
Ukuqapha amathuluzi amaphutha, ukukhukhuleka, nomthelela wabasebenzisi.
Lungiselela izindlela zokuhlehlisa nezigameko ngaphambi kokukala.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Barlow Twins and Redundancy Reduction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
I-Chain-of-Verification Yokunciphisa I-Hallucination
Imibuzo evame ukubuzwa
What is Barlow Twins and Redundancy Reduction?
I-Barlow Twins iyindlela ezigadwayo efunda izethulo ngokwenza i-matrix yokuhlobana phakathi kokubuka okubili okungeziwe eduze kwe-matrix yobunikazi. Igwema ukuwohloka ngomgomo wokunciphisa ukuphindaphindeka esikhundleni sokubi noma izifaki khodi zomfutho.
Iyiphi i-matrix u-Barlow Twins azama ukuyiphusha ebheke kubunikazi?
I-Barlow Twins ishayela i-matrix yokuhlobana phakathi kwezingxenye zokushumeka ezimbili zokubuka ukuya ku-matrix yobunikazi.
Yini ekhuthazwa yi-diagonal ye-matrix yobunikazi obuqondiwe?
Okufakiwe kwe-diagonal eduze noku-1 kusho ukuthi isici ngasinye siphendula ngendlela efanayo kukho kokubili ukubuka okuthuthukisiwe, okuphoqelela ukuguquguquka.
Ngabe okufakiwe kwe-off-diagonal okushayelwa ku-zero kufezani?
I-Zero off-diagonals isho ukuthi izici ezihlukene zihlotshisiwe, zombili ezinciphisa ukuphindaphindeka futhi zimise ukuwa okuncane.
Ubani umgomo we-neuroscience ogqugquzele igama nombono wendlela?
Indlela iqanjwe ngo-H. Barlow kanye nomgomo wakhe wokunciphisa ukuphindaphindeka kwekhodi ye-neural esebenza kahle.
Uma kuqhathaniswa ne-BYOL, yini i-Barlow Twins ENGAyidingi?
I-Barlow Twins isebenzisa amanethiwekhi afanayo angenakho ukubikezela, i-stop-gradient, noma i-EMA target, ethembele ekulahlekelweni kokunciphisa ukuphinda kuncishiswe.