Barlow Twins ak wàññi li ñuy liggéey
Barlow Twins xeetu xam-xam la bu nit ñi di saytu seen bopp, di jàng misaal yi ci defar matrix korrelaasioŋ croix diggante ñaari gis-gis yuñ yokk yu jege matrix dàntite bi.
Résumé
It avoids collapse through a redundancy-reduction principle rather than negatives or momentum encoders.
Plongeur bu xóot
Facebook AI moo ko taxawal ci 2021 te ñu tuddee ko neuroscientist H. Barlow ci njàngalem wàññi redundancy, Barlow Twins dafay dundal ñaari gis-gis yu jaarul yoon ci benn nataal jaaraleko ci reso yu nuróo ngir defar ñaari mbir yuñ samp. Dafay xayma matrix korrelaasioŋ croix bi am ci digganté komposant yi ci ñaari vecteur yiñ samp, ñu natt ko ci lote bi yépp. Mébet bi mooy push matrix bi ci dàntite bi: dugg diagonaal yi dañu wara nekk 1 (man-man bu nekk du soppiku ci yokkute bi) ak dugg yi nekk ci biti diagonal dañu wara nekk 0 (man-man yu wuute yi decorrelé, wàññi redondance). Term bi ci diagonaal dafay tax baña soppiku; terme biy wàññi redondance off-diagonal dafay tere dagg ndax màndarga yiñ decorrelé mënu ñu nekk ñoom ñépp. BYOL wuute na ak BYOL ndax soxlawul asymétrie, predictor, wala stop-gradient, te wuute na ak SimCLR ndax soxlawul ñaari negatif, waaye dafay am njariñ ci ay jëmmal yu kawe.
Gis-gis xarala
Perte bi amna ñaari pàcc yuñ boole ci kaw matrix korrelaasioŋ croisé C: benn (1-C_ii)^2 terme yu dul soppiku ci diagonaal bi, boole ci benn sum bu C_ij^2 ci bitti diagonaal bi. Ndax matrix bi dafa normalise ci kaw lote bi, njuréef bi dafa dëgër lool ci dayo lote bi, muy lu gëna am solo ci njuréefi kontrast yiy xiif ci lote yu bari yu negatif. Balansu performance ak dimensionnalité yuñ samp, moo tax projectër yi dañuy faral di yaatu lool.
njeextalu pexe
Njëgg ak budget
Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
dogal yu gëna leer
Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
Xool kalite
Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.
Ëlëgu seex yu Barlow ak wàññi li ñuy liggéey
Barlow Twins jàppale nañu ci sos famiy theorie ci xibaar ci anam yu ñuy saytu seen bopp, rawatina VICReg, biy tàqale variance, invariance, ak covariance ci anam wu leer. Xaarandi redondance-wàññi ak man-man-decorrelation mébetu wéy di yëgle ni ñuy njëkka tàggat encoder yi defar man-man yu kompact, yu amul redondance, ak dem lu ëpp nataal yi ci jekkal multimodal ak time-series fu decorrelé, représentation yu dëgër di jàppale model yu wàcci jàng ci etiket yu néew.
Doxal ci àdduna dëgg
Tàggat bu njëkk encodeur nataal yiy joxe man-mani decorrelé yu am njariñ ci xaaj bi ci suuf ak done yu néew yuñ etikete.
Taggat ci hardware bu yemmamaay fu lots negatif yu bari mënul nekk, ndax Barlow Twins amul benn yëg-yëg ci dayo lots.
Defar ay embedding yu kompact te amul redondance ngir gis ab cluster wala ab anomali ci nataali kaptëru usine yi.
Liggéey ci gëstub gëstu buy méngale pexe ngir moytu mabb ci SimCLR, BYOL, ak VICReg.
Risk yi ak balustrade yi
Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.
Roadmap ngir samp gi
Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
Benchmark ci biir sargal ak done yu dëggu.
Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Barlow Twins and Redundancy Reduction quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Chaîne de vérification pour réduction de hallucinations
Laaj yi ñuy faral di laaj
What is Barlow Twins and Redundancy Reduction?
Barlow Twins xeetu xam-xam la bu nit ñi di saytu seen bopp, di jàng misaal yi ci defar matrix korrelaasioŋ croix diggante ñaari gis-gis yuñ yokk yu jege matrix dàntite bi. Dafay moytu daanu jaaraleko ci sàrtu wàññi redondance moo gën encodeur negatif wala momentum.
Ban matrix la Barlow Twins di jéema génne ci dàntite bi?
Barlow Twins mooy dawal matrix biy boole ñaari gis-gis yi ci matrix biy dàntite.
Lan la duggal diagonaal yi ci matrix dàntite biñ bëgga ñaax?
Duggal diagonaal ci wetu 1 dafay tekki ni man-man bu nekk dafay tontu luni mel ci ñaari gis-gis yiñ yokk, di doolel invariance.
Lan la duggug off-diagonal bi ñuy yóbbu ci zero def?
Zero off-diagonal dafay tekki ni màndarga yu wuute yi dañu decorrelé, loolu dafay wàññi redondance ba noppi dakkal daanu bu amul benn njariñ.
Kan moo inspiré tur wi ak xalaatu njuréef mi ci neuroscience bi?
Ñu ngi tuddee anam wi H. Barlow ak njàngaleem ci wàññi redondance ci kodage neuronal bu baax.
Buñu ko méngale ak BYOL, lan la Barlow Twins soxlawul?
Barlow Twins dafay jëfandikoo reso yu nuróo te amul benn predictor, stop-gradient, wala EMA target, lu moy loolu dafay wéeru ci wàññi redondance.