VQGAN na Codebook Ishusho Synthesis
VQGAN ikanda amashusho murusobe rwibimenyetso byihariye byakuwe mu gitabo cyize, kureka transformateur ikabyara amashusho nkuko imiterere yindimi itanga inyandiko.
Kwibira cyane
VQGAN, yatangijwe mu mpapuro 2021 'Taming Transformers for High-Resolution Image Synthesis,' ikomatanya ibinyabiziga byitwa autoencoder (VQVAE) hamwe n'amahugurwa yo guhangana no gushishoza. Kodegisi ishushanya ishusho kuri gride ntoya yibiranga; buri vector ifatirwa kumurongo wegereye muri codebook yize ya, tuvuge, 1024 code yihariye, ihindura ishusho murukurikirane rwibimenyetso byuzuye. Decoder yubaka ishusho kuva kuri ibyo bimenyetso, yatojwe hamwe nivangura rya GAN hamwe nigihombo cyo kwiyumvisha kuburyo kwiyubaka bisa nkibikabije aho kuba ibicucu. Kuberako amashusho ubu arikimenyetso cyerekana ibimenyetso, transformateur autoregressive irashobora kubigereranya nkururimi, guhanura ibimenyetso umwe umwe. VQGAN izwi cyane gukoresha ibikoresho byambere-by-amashusho ibikoresho byubuhanzi iyo bihujwe nubuyobozi bwa CLIP.
Ubushishozi
Igikorwa cyibanze ni ingano ya vector: ibisubizo bya kodegisi ikomeza gusimburwa na codebook yegeranye cyane, hamwe na 'igororotse-inyuze' igereranya rya gradient kugirango kodegisi irashobora kwiga nubwo idashakisha itandukaniro. Ongeraho ibice bishingiye kuri GAN ivangura hejuru ya autoencoder nibyo bituma VQGAN ikoresha gride ntoya cyane (urugero 16x16) kuruta VQVAE mugihe igumisha imiterere, bigatuma moderi ya transformateur ikurikirana.
Ingaruka z'Ingamba
Umuvuduko n'igipimo
AI igaragara irashobora gukora igenzura, gutahura, no gutondekanya imirimo kurwego.
Build choices
Amakipe arema arashobora prototype ibitekerezo byihuse hamwe nintoki nkeya.
Itsinda hamwe nakazi
Ibikorwa birashobora gukoresha amashusho nibimenyetso bya videwo byari bigoye gutunganya.
Kazoza ka VQGAN na Codebook Ishusho Synthesis
VQGAN ya discret-token resept yabaye ishingiro ryamashusho yerekana amashusho na moderi yerekana amashusho, kuva MaskGIT kugeza kuri sisitemu ya multimodal ivanga amashusho nibimenyetso byanditse muri transformateur imwe. Ubushakashatsi ubu busunika kuri kodegisi nini, itagira iherezo cyangwa ishakisha-idafite igitabo cyirinda gusenyuka no kwerekana imiterere ihuriweho hamwe aho amagambo amwe akubiyemo amashusho, amajwi, n'ururimi, bigafasha igisekuru icyo ari cyo cyose.
Gushyira mu bikorwa Isi
Gushyira ifoto muri gride ya 16x16 yerekana ibimenyetso bya codebook kugirango transformateur ibashe kwerekana no kuyisubiramo
Guhuza VQGAN hamwe nubuyobozi bwa CLIP kugirango ukore ibihangano bya 'VQGAN + CLIP' ibihangano bya AI byagaragaye muri 2021
Gucomeka amashusho muburyo bworoshye bwo kubika cyangwa guhugura neza
Gukora nkibishusho bya tokenizer imbere nini nini ya generator nka MaskGIT na transformateur ya multimodal
Ingaruka & Kurinda
Uburenganzira bwishusho hamwe no kwemererwa birashobora guhinduka ibyago byemewe n'amategeko niba ibimenyetso bidasobanutse.
Imikorere yicyitegererezo irashobora gutandukana kumurika, demografiya, nibidukikije.
Ibyiza byibinyoma birashobora kutamenyekana keretse niba ibyiringiro byateganijwe bikurikiranwa.
Igishushanyo mbonera
Sobanura ibipimo byo kwemererwa kugiciro, kwibutsa, nibiciro byamakosa.
Gerageza hamwe namakuru ajyanye nuburyo nyabwo bwo gukora.
Ongeraho isubiramo ryabantu kubwizere buke cyangwa guhanura cyane.
Kurikirana icyitegererezo cya drift hanyuma uhindurwe nyuma ya kamera cyangwa dataset ihinduka.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VQGAN and Codebook Image Synthesis quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
SPADE Semantike Ishusho
Ibibazo bikunze kubazwa
What is VQGAN and Codebook Image Synthesis?
VQGAN ikanda amashusho murusobe rwibimenyetso byihariye byakuwe mu gitabo cyize, kureka transformateur ikabyara amashusho nkuko imiterere yindimi itanga inyandiko.
Niki VQGAN ikoresha mugushushanya ishusho nkibimenyetso byihariye?
VQGAN igereranya ibiranga encoder kubintu byegereye byanditse muri codebook yize, ihindura ishusho muburyo bukurikirana indangagaciro za code.
Kuki VQGAN yongeraho ivangura rya GAN hejuru ya VQVAE?
Igihombo cyo guhangana no gushishoza reka VQGAN yubake amashusho atomoye kuva gride token to grid, VQVAE yonyine ikunda guhuzagurika.
Iyo ishusho imaze gukurikiranya ibimenyetso, niyihe moderi itanga amashusho mashya?
Kuberako amashusho ahinduka ibimenyetso byikurikiranya, transformateur irashobora kubigereranya ubwabyo, kimwe no kubyara inyandiko.
Ni ubuhe buhanga butuma gradients zinyura mu ntambwe itandukanye yo kubara?
Umubare wa Vector ntaho utandukaniye, VQGAN rero ikoresha igereranya-igereranya igereranya kugirango ihugure kodegisi.
Ni ubuhe bwoko bw'ubuhanzi bwa AI buzwi VQGAN yafashije kurema muri 2021?
Guhuza VQGAN hamwe nubuyobozi bwa CLIP byabyaye ibihangano bisangiwe cyane 'VQGAN + CLIP' byamamaye kubyara amashusho.