VQGAN ak téere kode
VQGAN dafay tënk nataal yi ci biir benn griy bu ay token yu wuute yuñ jëlee ci benn téere kode buñ jàng, loolu mooy tax transformatër bi defar nataal ni ko modelu làkk yi di defaree mbind.
Plongeur bu xóot
VQGAN, biñu dugal ci këyitu 2021 bi tuddu 'Transformatëri taming ngir synthese nataal bu am dayo bu kawe,' dafa boole autoencoder buñ xayma ci vecteur (VQVAE) ak tàggat yaram ak xam-xam. Encoder dafay boole nataal ci griy bu ndaw bu am vecteur yu am màndarga; vecteur bu nekk dañu koy jël ci duggal bi gëna jege ci téere kode buñ jàng, lu ci melni, 1024 kode yu wuute, ñu soppi nataal bi ci xeetu token yu mat. Decoder bi dafay tabaxaat nataal bi ci token yooyu, ñu tàggat ko ak discriminateur GAN ak perceptual perceptual suko defee tabaxaat yi di nuru lu ñaw moo gën ñu fësul. Ginaaw nataal yi dañuy nekk ay token yu wuute, transformateur autorégressif mën na leen modele ni làkk, di wax luy waaja am benn-benn. VQGAN dafa siiw lool ci jumtukaayi art yu njëkk yi ñuy soppi bind ci nataal suñu ko boole ak tegtali CLIP.
Gis-gis xarala
Li gëna am solo ci liggéey bi mooy kantite vecteur: génnug encoder bu wéy bi dañu koy wecci ak vecteur codebook bi gëna jege, ak estimatëru gradient bu 'jub' suko defee encoder bi mëna jàng ba leegi doonte seetlu bi mënul wuute. Yokk benn GAN bu sukkandiko ci patch ci kaw autoencoder mooy tax VQGAN jëfandikoo griy token bu gëna ndaw (lu melni 16x16) bu gëna VQVAE ci noonu muy tëye textures yi, def modeling transformateur tractable.
njeextalu pexe
Gaawaay ak yaatuwaay
Visual IA mën na otomatise saytu, gis ak etiketu liggéey ci eskaal.
Tabax tànneef
Ekipu kreatif yi mën nañu defar konsept yu gëna gaaw te duñu def lu bari ci loxo.
Ekip ak def liggéey
Liggéeyukaay yi mën nañu jëfandikoo siñaal nataal wala wideo yu jafewoon lool ci liggéey.
Ëlëgu VQGAN ak Codebook nataal
Reset token diskret bu VQGAN moo nekkoon fundaasioŋ ngir modelu nataal ak wideo yu lalu ci token, daale ko ci MaskGIT ba ci sistem multimodal yuy jaxase token nataal ak mbind ci benn transformatër. Gëstu leegi dafay push ci codebook yu gëna mag, finite-scalar wala lookup-free yu moytu codebook collapse ak ci model yuñ boole fu benn vocabulaire bi di jëm ci nataal, audio, ak làkk, may bépp-ba-bépp jamono.
Doxal ci àdduna dëgg
Kodage ab nataal ci biir benn griy bu 16x16 ci jetons codebook suko defee transformateur bi mëna ko modele ak defaraat ko
boole VQGAN ak njiitu CLIP ngir defar 'VQGAN+CLIP' art IA bu siiw ci 2021
Komprime nataal yi ci ay kod diskret yu kompact ngir dencukaay bu baax wala tàggat yaram buy génne
Liggéey ni tokenisatëru nataal ci biir generatër yu gëna mag yu lalu ci token yu melni MaskGIT ak transformatër yu bari
Risk yi ak balustrade yi
Yelleefi nataal ak nangu mën na nekk risku yoon sudee fi ñu bawoo leerul.
Performance model bi mën na wuute ci leeraay bi, demographie bi ak environmaa bi.
Njuumteg positive yi mën nañu dem te kenn duko seetlu fileek xool wuñu buntu wóolu sa bopp.
Roadmap ngir samp gi
Mandargal kritërium nangug njub, woowaat ak njëgu njuumte.
Saytu ak done yu méngoo ak anam yi ñuy liggéeyee dëgg.
Yokk jàngat nit ngir xam fu wóorul dara wala am njeexital yu rëy.
Toppal model drift bi nga baaxal ko ginaaw bi kamera bi wala done yi soppeekoo.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VQGAN and Codebook Image Synthesis quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
SPADE Nataalu Semantik
Laaj yi ñuy faral di laaj
What is VQGAN and Codebook Image Synthesis?
VQGAN dafay tënk nataal yi ci biir benn griy bu ay token yu wuute yuñ jëlee ci benn téere kode buñ jàng, loolu mooy tax transformatër bi defar nataal ni ko modelu làkk yi di defaree mbind.
Lan la VQGAN di jëfandikoo ngir wane nataal ni ay jeton yu wuute?
VQGAN dafay xayma man-mani encoder yi ci duggal yi gëna jege ci téere kode biñ jàng, mu soppi nataal bi ci xeetu indices kode yu wuute.
Lu tax VQGAN di yokk benn GAN ci kaw benn VQVAE?
Perte adversarial ak perceptuel may VQGAN mu defaraat nataal yu fës ci griy token bu dëgër, bi VQVAE kese bëgga fësul.
Bu benn nataal nekkee ay jeton yu toppalante, ban model mooy defar nataal yu bees?
Ndax nataal yi dañuy nekk ay token yu wuute, transformateur bi mën na leen modele ci anam wu autorégresif, melni defar mbind.
Ban pexe mooy may gradient yi ñu jaar ci jéego kantite bu mënul wuute?
Kantifikaasioŋ vecteur mënul wuute, moo tax VQGAN dafay jëfandikoo estimatëru gradient bu jub ngir tàggat encodeur bi.
Ban art IA bu siiw la VQGAN jàppale ci defar ci 2021?
Teg VQGAN ak njiitu CLIP moo jur art bu ñu séddoo bu baax 'VQGAN+CLIP' bi siiwal defar nataal yu lalu ci bind.