VQGAN da Tsarin Hoto na Codebook
VQGAN yana matsa hotuna zuwa grid na sahihan alamomi da aka zana daga littafin da aka koya, yana barin taswira ya samar da hotuna kamar yadda ƙirar harshe ke samar da rubutu.
Zurfafa nutsewa
VQGAN, wanda aka gabatar a cikin takarda ta 2021 'Taming Transformers for High Resolution Hoto Synthesis,' ya haɗu da na'urar ƙididdigewa ta atomatik (VQVAE) tare da horarwa na gaba da fahimta. Mai rikodin taswirar hoto zuwa ƙaramin grid na sifa; kowane vector yana tsinkewa zuwa shigarwa mafi kusa a cikin littafin koyo na, a ce, lambobi masu hankali 1024, suna juya hoton zuwa jeri na alamomin lamba. Mai gyara hoto yana sake gina hoton daga waɗannan alamun, an horar da shi tare da GAN mai wariya da hasarar fahimta don haka sake ginawa yayi kama da kaifi maimakon blush. Saboda hotuna a yanzu jerin alamomi ne masu hankali, na'urar taswira ta atomatik na iya yin ƙima da su kamar harshe, yana tsinkayar alamu ɗaya bayan ɗaya. VQGAN sanannen kayan aikin fasaha na farkon rubutu-zuwa hoto lokacin da aka haɗa su tare da jagorar CLIP.
Fahimtar Fasaha
Babban aikin shine ƙididdigewa na vector: ana maye gurbin abubuwan da aka ci gaba da shigar da kayan aikin da mafi kusa da su, tare da ma'aunin 'daidai-ta'' ƙididdige ƙididdigewa don haka mai rikodin zai iya koyo duk da binciken da ba shi da bambanci. Ƙara mai nuna wariyar GAN na tushen faci a saman autoencoder shine abin da zai ba VQGAN damar amfani da grid ƙarami (misali 16x16) fiye da VQVAE yayin da yake kiyaye laushi mai laushi, yana sa mai canza canjin canji.
Dabarun Tasiri
Gudu da sikelin
Kayayyakin AI na iya sarrafa aiki da bincike, ganowa, da ayyuka masu alama a sikelin.
Gina zaɓuɓɓuka
Ƙungiyoyin ƙirƙira za su iya samar da ra'ayoyi cikin sauri tare da ƙarancin bita da hannu.
Ƙungiya da aikin aiki
Ayyuka na iya amfani da siginar hoto da bidiyo waɗanda a baya suke da wahalar aiwatarwa.
Makomar VQGAN da Tsarin Hoto na Codebook
VQGAN's discrete-token girke-girke ya zama tushen tushen alamar hoto da ƙirar bidiyo, daga MaskGIT zuwa tsarin multimodal waɗanda ke haɗa hotuna da alamun rubutu a cikin mai canzawa ɗaya. Bincike yanzu yana matsawa zuwa ga manyan littattafai masu ƙarfi, ƙayyadaddun ƙayyadaddun bayanai ko bincika marasa kyauta waɗanda ke guje wa rugujewar littafin da kuma zuwa ga ƙirar ƙira ɗaya inda ƙamus iri ɗaya ke ɗaukar hotuna, sauti, da harshe, ba da damar kowane-zuwa-kowane tsara.
Aiwatar da Gaskiyar Duniya
Sanya hoto a cikin grid 16x16 na alamomin codebook don haka mai canzawa zai iya ƙira da sabunta shi.
Haɗa VQGAN tare da jagorar CLIP don ƙirƙirar fasahar 'VQGAN+CLIP' AI wacce ta zama hoto ko bidiyo mai zagaya yanar gizo da sauri a cikin 2021
Matsa hotuna cikin ƙananan lambobi masu ma'ana don ingantacciyar ma'ajiya ko horon haɓakawa na ƙasa
Yin hidima azaman tokenizer na hoto a cikin manyan janareta na tushen alamar kamar MaskGIT da masu taswirar multimodal
Hatsari & Tsare-tsare
Haƙƙoƙin hoto da yarda na iya zama haxarin doka idan ba a fayyace ba.
Ayyukan samfuri na iya bambanta a ko'ina cikin haske, ƙididdiga, da mahalli.
Ƙarya tabbataccen ƙila ba za a iya lura da shi ba sai dai idan an kula da ƙofofin amincewa.
Taswirar Hanya
Ƙayyade ma'auni na karɓa don daidaito, tunowa, da farashi na kuskure.
Gwada tare da bayanan da suka dace da ainihin yanayin samarwa.
Ƙara bita na ɗan adam don ƙarancin amincewa ko tsinkaya mai tasiri.
Bi diddigin ƙirar ƙira kuma sake ingantawa bayan canje-canjen kamara ko saitin bayanai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VQGAN and Codebook Image Synthesis quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
SPADE Tsarin Hoton Semantic
Tambayoyin da ake yawan yi
What is VQGAN and Codebook Image Synthesis?
VQGAN yana matsa hotuna zuwa grid na sahihan alamomi da aka zana daga littafin da aka koya, yana barin taswira ya samar da hotuna kamar yadda ƙirar harshe ke samar da rubutu.
Menene VQGAN ke amfani da shi don wakiltar hoto azaman alamu masu hankali?
VQGAN yana ƙididdige fasalulluka masu ɓoyewa zuwa shigarwar mafi kusa a cikin littafin da aka koya, yana juya hoton zuwa jeri na fihirisar lamba.
Me yasa VQGAN ke ƙara GAN mai wariya a saman VQVAE?
Asarar abokan gaba da fahimta sun bar VQGAN ta sake gina ƙwaƙƙwaran hotuna daga ƙaƙƙarfan grid alama, wanda VQVAE ita kaɗai ke ƙoƙarin yin duhu.
Da zarar hoto ya zama jerin alamomi, wane tsari ne ke haifar da sabbin hotuna?
Saboda hotuna sun zama jeri-nauyen alamomi, na'urar taswira na iya yin su ta atomatik, kamar samar da rubutu.
Wace dabara ce ke ba da damar gradients su gudana ta hanyar matakan ƙididdigewa mara bambanci?
Ƙididdigar ƙididdigewa ba ta bambanta ba, don haka VQGAN yana amfani da madaidaicin ƙididdigewa don horar da mai rikodin.
Wane shahararren fasahar fasahar AI VQGAN ya taimaka ƙirƙirar a cikin 2021?
Haɗin VQGAN tare da jagorar CLIP ya samar da fasahar 'VQGAN+CLIP' da aka raba ta ko'ina wacce ta haɓaka tsarar hoto da rubutu ke motsawa.