Vision Transformers
Vision Transformers (ViTs) inoshandisa dhizaini yetransformer inopa simba ChatGPT kumifananidzo, ichitora mufananidzo senhevedzano yezvigamba pachinzvimbo chegridi yemapikisi.
Pfupiso
They proved that you do not need convolutions to achieve state-of-the-art image recognition.
Kudzika Kwakadzika
Kwemakore, convolutional neural network (CNNs) yaitonga komputa kuona nekutarisa mafirita madiki pamufananidzo. Bepa ra2020 'Mufananidzo Wakakodzera 16x16 Mazwi' kubva Google rakapikisa izvi nekucheka chifananidzo kuita mapechi akagadzika, kazhinji 16x16 pixels, kupepeta rimwe nerimwe kuita vector, uye kudyisa zvakatevedzana kuita mushanduri wakajairwa. Chigamba chimwe nechimwe chinova 'chiratidzo,' sezwi riri mumutsara. Iyo modhi inozoshandisa kuzvidzora kuitira kuti chigamba chega chega chigone kuenderana kune chimwe chigamba, kutora hukama hurefu-refu diki sefa isingaone munhanho imwe. Kubata: ViTs vane nzara-data nekuti vanoshaya yakavakirwa-mukati fungidziro dzeCNNs. Vakadzidziswa pamaseti akakura seJFT-300M, vaifananidza kana kurova maCNN akanakisa, vachigadzirazve tsvakiridzo yechiono chazvino.
Technical Insight
A ViT inotsemura chifananidzo kuita chigamba chisina kupindirana, mutsara mapurojekiti chimwe nechimwe mukumisikidza, uye inowedzera maencodings kuitira kuti modhi izive pakagara chigamba chega chega mumufananidzo wekutanga. Yakakosha inodzidzwa 'class token' inofanogadzirirwa; kumiririra kwayo kwekupedzisira kunofambisa kupatsanurwa. Akaturikidzana ega ega ega matinji rega rega rega riyere ruzivo kubva kune mamwe ese, ichipa yepasi rose inogashira munda kubva kune imwe layer. Nekuti kutarisisa kunoyera quadratically nenhamba yezvigamba, mifananidzo yakakwira-resolution inodhura, ndosaka ukuru hwechigamba uye kutarisisa kwakasiyana kunokosha.
Strategic Impact
Kumhanya uye chiyero
Visual AI inogona kuita otomatiki yekuongorora, yekuona, uye yekumaka mabasa pachiyero.
Vaka sarudzo
Zvikwata zvekugadzira zvinogona prototype pfungwa nekukurumidza nekudzokororwa kwemaoko mashoma.
Team uye workflow
Mashandisirwo anogona kushandisa masaini emifananidzo nemavhidhiyo ayo aimbove akaoma kugadzirisa.
Ramangwana reVision Transformers
ViTs neCNN-transformer mahybrids iye zvino ane masimba anotungamira ekuona masisitimu, uye zvivakwa zvinotsigira mamodhi akawanda anosanganisa mifananidzo nemavara, seCLIP nevabatsiri vemutauro wemazuva ano. Tarisira kuenderera mberi nebasa rekuita kuti kutarisa kudhure kune yakakwirira-resolution uye vhidhiyo, pamwe nekuzvitarisira wega pretraining (senge masked-image modelling) iyo inoderedza yakakura yakanyorwa-data havi. Sezvo komputa inokura, mutsetse uripo pakati pe'mutauro modhi' uye 'muenzanisi wechiratidzo' unoramba usina kujeka, nematransformer anoshanda semusana wakagovaniswa mumhando dzese pane kuparadzanisa dhizaini.
Real-World Implementation
Google's image classification and search ranking systems dzakatora transformer backbones mushure mekunge ViT yaratidza kukwikwidzana neCNNs.
CLIP uye mamwe emifananidzo-mameseji mamodheru anoshandisa ViT encode mifananidzo kuitira kuti mafoto nemifananidzo zvienderane munzvimbo yakagovaniswa.
Tsvagiridzo yekufungidzira yekurapa uchishandisa ViTs kuona mapatani mukati mese scansheni kwete chete maumbirwo enzvimbo
Kuzvityaira uye marobhoti ekuona akaturikidzana anosanganisa ViT-maitiro kutarisisa kwekunzwisisa kwechiitiko munzvimbo yakazara yekuona.
Njodzi & Guardrails
Kodzero dzemifananidzo uye kubvumirwa kunogona kuve njodzi dzepamutemo kana provenance isina kujeka.
Kuita kwemuenzaniso kunogona kusiyanisa kupenya, huwandu hwevanhu, uye nharaunda.
Manyepo enhema anogona kusacherechedzwa kunze kwekunge zvikumbaridzo zvekuvimba zvikatariswa.
Implementation Roadmap
Tsanangura maitiro ekugamuchirwa echokwadi, kurangarira, uye mutengo wekukanganisa.
Edzai nedata rinoenderana nemamiriro chaiwo ekugadzira.
Wedzera ongororo yemunhu kune yakaderera-kusavimbika kana yakakwirira-inokanganisa kufanotaura.
Tevera modhi kudonha uye simbisa mushure mekuchinja kwekamera kana dataset.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Vision Transformers quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
CLIP uye Vision-Mutauro Models
Mibvunzo inowanzo bvunzwa
What is Vision Transformers?
Vision Transformers (ViTs) inoshandisa dhizaini yetransformer inopa simba ChatGPT kumifananidzo, ichitora mufananidzo senhevedzano yezvigamba pachinzvimbo chegridi yemapikisi. Vakaratidza kuti haudi convolutions kuti uwane kucherechedzwa kwemufananidzo.
Iyo Vision Transformer inotanga sei kugadzirisa mufananidzo wekuisa?
A ViT inokamura chifananidzo kuita zvigamba zvakagadziriswa (kazhinji 16x16 pixels), inomisikidza chigamba chimwe nechimwe sechiratidzo, uye inodyisa kutevedzana kuita shanduko.
Ndeipi nzira yakakosha inoita kuti ViT iwirirane zvikamu zviri kure zvemufananidzo kune mumwe nemumwe?
Kuzvitarisa kunoita kuti chigamba chega chega chiyere zvakananga ruzivo kubva kune chimwe chigamba, ichipa maonero epasirese kubva padanho rekutanga.
Sei akajeka Vision Transformers achiwanzoda akakura kwazvo ekudzidzisa madhaseti kuti abudirire?
Kusiyana neCNNs, ViTs haifungi nharaunda kana kusarudzika kweshanduro, saka vanofanirwa kudzidza mapatani aya kubva kuhuwandu hwedata.
Ndechei chinangwa chepositical encodings muVision Transformer?
Kuzvitarisa ndiko kurongeka-agnostic, saka ma encodings anodzoreredza nzvimbo yenzvimbo yechigamba chimwe nechimwe.
Ndechipi chirevo chinonyatso ratidza maitiro eViT pakuona komputa?
ViT yakaratidza kuti shanduri yakachena, ine data rakakwana uye chiyero, inogona kukwikwidza kana kupfuura akanakisa convolutional network.