Transformatëri gis-gis
Transformatëri gis-gis (ViTs) dañuy jëfandikoo arsitektir transformatër biy dundal ChatGPT ci nataal yi, ñuy jàppee nataal bi ni ay patch yu toppalante ci barabu griy pixel yi.
Résumé
They proved that you do not need convolutions to achieve state-of-the-art image recognition.
Plongeur bu xóot
Lu weesu ay at, reso neuronal convolutionnel (CNNs) ñoo jiitewoon gis-gis ordinatër bi, di skane ay filtre yu ndaw ci biir nataal bi. Këyitu 2020 bi tuddu 'Nataal Jarul 16x16 Baatu' bu bawoo ci Google dafa xeex lii ci dagg benn nataal ci ay pàcc yu takku, lu ci gëna bari 16x16 pixel, daal di platre bu nekk ci benn vecteur, ba noppi joxe li ciy génn ci benn transformatër buñ miin. Bépp patch dafay nekk 'token', lu melni baat ci biir frase. Modèle bi dafay jëfandikoo self-attention suko defee patch bu nekk mëna lëkkaloo ak beneen patch bu nekk, jàpp lëkkaloo yu sori yu filtre convolutionnel bu ndaw mënul gis ci benn jéego. Japp bi: ViTs yi dañu xiif ci done ndax amu ñu ay xalaat yuñ tabax ci CNNs. Ñu ngi leen tàggat ci ay done yu bari yu melni JFT-300M, ñu méngoo wala ñu raw CNN yi gëna baax, ñu soppali gëstub gis-gis bu bees bi.
Gis-gis xarala
ViT dafay xaaj benn nataal ci ay patch yu dul jaxasoo, mu projecte bu nekk ci benn embedding, ba noppi yokk ci encoding positionnel suko defee model bi xam fi patch bu nekk toog ci nataalu njëkk bi. Dañu ci njëkka bind benn 'class token' buñ mëna jàng; Representation bu mujj bi mooy dawal xaaj bi. Ay couche yuñ dajale ngir bàyyi seen xel ci seen bopp, dañuy may patch bu nekk mu jël leeral yi ci ñeneen ñi, loolu mooy tax ñu am barab buy jot ci couche bi njëkk. Ndax bàyyi xel dafay yokk ci anam wu ñeent ak limu patch yi, nataal yu am resolusioŋ bu kawe dañuy seer lool, moo tax dayo patch bi ak anam yu baax yi ñuy bàyyi xel dañu am solo.
njeextalu pexe
Gaawaay ak yaatuwaay
Visual IA mën na otomatise saytu, gis ak etiketu liggéey ci eskaal.
Tabax tànneef
Ekipu kreatif yi mën nañu defar konsept yu gëna gaaw te duñu def lu bari ci loxo.
Ekip ak def liggéey
Liggéeyukaay yi mën nañu jëfandikoo siñaal nataal wala wideo yu jafewoon lool ci liggéey.
Ëlëgu soppi gis-gis
ViTs ak CNN-transformer hybrids leegi ñooy gëna am doole ci sistemu gis-gis yi, te architecture bi dafay jàppale xeetu multimodal yiy boole nataal ak bind, lu melni CLIP ak assistant lakk-gis-gis bu bees. Xaarandi liggéey bu wéy ngir gëna xéewale bàyyi xel ci resolusioŋ bu kawe ak wideo, boole ci tàggat sa bopp (lu melni modeling nataal bu maske) buy wàññi bëgg-bëggu done yu bari yiñ etikete. Lu ordinatër bi di gëna màgg, dig wi dox diggante 'modèlu làkk' ak 'modèlu gis-gis' dafay wéy di gëna fës, transformatër yi di nekk yaxu ndigg buñ bokk ci anam yi, duñu tàqale ay jëmmal.
Doxal ci àdduna dëgg
Google sistemu xaaj nataal ak seetug rang yi jël yaxu ndigg transformatër ginaaw bi ViT wane ni mën na joŋante ak CNNs
CLIP ak yeneen xeeti nataal-tekst yuy jëfandikoo ViT ngir enkode nataal yi suko defee ñu mëna méngale nataal yi ak mbind yi ci barab buñ bokk
Gëstu nataali pajum jëfandikoo ViTs ngir gis motif yi ci scanner bi yépp, du ci textures yu bërëb bi kese
Dawal sa bopp ak robotik gis-gis stack yu boole ViT-style attention ngir xam seen ci barab bi ñuy xoole yépp
Risk yi ak balustrade yi
Yelleefi nataal ak nangu mën na nekk risku yoon sudee fi ñu bawoo leerul.
Performance model bi mën na wuute ci leeraay bi, demographie bi ak environmaa bi.
Njuumteg positive yi mën nañu dem te kenn duko seetlu fileek xool wuñu buntu wóolu sa bopp.
Roadmap ngir samp gi
Mandargal kritërium nangug njub, woowaat ak njëgu njuumte.
Saytu ak done yu méngoo ak anam yi ñuy liggéeyee dëgg.
Yokk jàngat nit ngir xam fu wóorul dara wala am njeexital yu rëy.
Toppal model drift bi nga baaxal ko ginaaw bi kamera bi wala done yi soppeekoo.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Vision Transformers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
CLIP ak misaali làkku gis-gis
Laaj yi ñuy faral di laaj
What is Vision Transformers?
Transformatëri gis-gis (ViTs) dañuy jëfandikoo arsitektir transformatër biy dundal ChatGPT ci nataal yi, ñuy jàppee nataal bi ni ay patch yu toppalante ci barabu griy pixel yi. Firnde nañu ni soxlawolen ay convolution ngir mëna xàmmee nataal ci anam wu bees.
Naka lay def ba Transformateur Vision di njëkka jëfandikoo nataal biñ dugal?
ViT dafay xaaj nataal bi ay pàcc yu takku (ñuy faral di nekk 16x16 pixel), daal di dugal pàcc bu nekk ni token, ba noppi dugal toppalante bi ci transformatër bi.
Ban mecanisme bu am solo mooy may ViT mu mëna boole pàcci nataal yu sori yi ci seen biir?
Self-attention dafay may patch bu nekk mu jël leeral yi ci beneen patch bu nekk, di joxe gis-gis bu gëna mag ci layer bu njëkk bi.
Lan moo waral Transformatëri Vision yu leer yi di soxla ay done yu bari ngir mëna jàll?
ViTs wuute na ak CNNs ndax duñu xalaat ni barab bi wala tekki bi du soppeeku, kon dañu wara jàng motif yooyu ci done yu bari.
Lan mooy njariñu enkodaasu position ci Transformatëru gis-gis?
Fexe sa bopp nekkul lu jëm ci yoon, kon enkodaasu position yi dañuy defaraat barabu patch bu nekk.
Ban wax moo gëna fësal njeextalu ViT ci xoolum ordinatër?
ViT dafa wane ni transformatër bu sell, am done yu doy ak yaatuwaayam, mën na xëcc wala raw reso convolutionnel yi gëna baax.