I-VQGAN kanye ne-Codebook Image Synthesis Guide

I-Deep Dive

I-VQGAN, eyethulwe ephepheni lango-2021 elithi 'Taming Transformers for High-Resolution Image Synthesis,' ihlanganisa i-autoencoder ye-vector-quantized autoencoder (VQVAE) nokuqeqeshwa okuphikisayo nokucabangayo. Isifaki khodi sibonisa isithombe kugridi encane yamavekhtha wesici; ivekhtha ngayinye ihlwithwa ekufakweni okuseduze kwe-codebook efundiwe, yithi, amakhodi ahlukene ayi-1024, kuguqule isithombe sibe ukulandelana kwamathokheni enamba. Isikhiphi sakha kabusha isithombe sisuka kulawo mathokheni, aqeqeshwe ngomcwasi we-GAN nokulahlekelwa kombono ukuze ukwakhiwa kabusha kubukeke kucijile kunokufiphala. Ngenxa yokuthi izithombe manje seziwukulandelana kwamathokheni ahlukene, i-autoregressive transformer ingakwazi ukuyifanisa njengolimi, ibikezele amathokheni ngayinye ngayinye. I-VQGAN inikeze amandla amathuluzi obuciko okuqala ombhalo uye esithombeni uma ebhangqwe nesiqondiso se-CLIP.

I-Technical Insight

Umsebenzi oyinhloko uwukubalwa kwevekhtha: okuphumayo kwesifaki khodi okuqhubekayo kushintshaniswa amavektha e-codebook aseduze, ngesilinganiso segradient 'esiqondile sidlule' ukuze isifaki khodi sisakwazi ukufunda naphezu kokubheka okungahlukani. Ukwengeza isihlukanisi se-GAN esisekelwe ku-patch phezu kwe-autoencoder yikho okuvumela i-VQGAN ukuthi isebenzise igridi yethokheni encane kakhulu (isb.

I-Strategic Impact

Isivinini nesikali

I-Visual AI ingakwazi ukuhlola, ukutholwa, nokumaka imisebenzi esikalini.

Yakha ukukhetha

Amathimba aqanjiwe angakwazi ukulinganisa imiqondo ngokushesha ngezibuyekezo ezimbalwa ezenziwa mathupha.

Ithimba kanye nokusebenza komsebenzi

Imisebenzi ingasebenzisa amasiginali wesithombe nawevidiyo obekunzima ukuwenza ngaphambilini.

Ikusasa le-VQGAN kanye ne-Codebook Image Synthesis

Iresiphi yethokheni ye-VQGAN ehlukene yaba yisisekelo samamodeli ezithombe namavidiyo asuselwa kumathokheni, ukusuka ku-MaskGIT kuya ezinhlelweni ze-multimodal ezixuba amathokheni wesithombe nemibhalo ku-transformer eyodwa. Ucwaningo manje luphushela kuma-codebook amakhulu, anesikali noma angabheki agwema ukuwa kwe-codebook futhi abheke kumamodeli ahlanganisiwe lapho silulumagama esifanayo sihlanganisa izithombe, umsindo, nolimi, okuvumela noma yisiphi isizukulwane.

Ukuqaliswa Komhlaba Wangempela

Ukufaka isithombe ngekhodi kugridi engu-16x16 yamathokheni e-codebook ukuze i-transformer ikwazi ukwenza imodeli futhi iyenze kabusha

Ukumatanisa i-VQGAN nesiqondiso se-CLIP ukuze udale ubuciko be-surreal 'VQGAN+CLIP' AI oba negciwane ngo-2021

Ukucindezela izithombe zibe amakhodi ahlangene ukuze ugcine kahle noma ukuqeqeshwa kokukhiqiza

Isebenza njengethokheni yesithombe ngaphakathi kwamajeneretha amakhulu asuselwa kumathokheni afana ne-MaskGIT nama-multimodal transformers

Izingozi & Guardrails

Amalungelo ezithombe kanye nemvume kungaba ubungozi bezomthetho uma ukuvela kungacacile.

Ukusebenza kwemodeli kungahluka kukho konke ukukhanya, izibalo zabantu, kanye nezindawo.

Okuhle okungelona iqiniso kungase kungabonakali ngaphandle uma izinga lokuzethemba liqashelwa.

Ukuqalisa Umhlahlandlela

Chaza indlela yokwamukela yokunemba, ukukhumbula, nezindleko zamaphutha.

Hlola ngedatha efana nezimo zangempela zokukhiqiza.

Engeza isibuyekezo somuntu ukuze uthole ukuzethemba okuphansi noma izibikezelo zomthelela omkhulu.

Landelela ukukhukhuleka kwemodeli bese uqinisekisa kabusha ngemva kwezinguquko zekhamera noma zesethi yedatha.

Qhubeka Uhlole

I-Computer Vision

Qonda amasistimu ayisisekelo anika amandla i-AI ebonakalayo.

Funda Umhlahlandlela

I-AI Image Generation

Hlola ukugeleza komsebenzi wokudala kanye nokuhwebelana kwamamodeli.

Funda Umhlahlandlela

Free newsletter

Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.

One email each weekday. Unsubscribe in one click. We never sell or share your address.

Test yourself

Take the VQGAN and Codebook Image Synthesis quiz

Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.

Qala imibuzo

Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation

Umhlahlandlela olandelayo

I-SPADE Semantic Image Synthesis

Funda umhlahlandlela olandelayo Phequlula yonke imibuzo

Imibuzo evame ukubuzwa

What is VQGAN and Codebook Image Synthesis?

I-VQGAN icindezela izithombe zibe igridi yamathokheni ahlukene athathwe ebhukwini lekhodi elifundiwe, ivumela isiguquli ukuthi sikhiqize izithombe ngendlela efanayo namamodeli olimi akhiqiza ngayo umbhalo.

Isebenzisa ini i-VQGAN ukumela isithombe njengamathokheni ahlukene?

I-VQGAN ilinganisa izici zesifaki khodi kokufakiwe okuseduze ebhukwini lekhodi elifundiwe, iguqule isithombe sibe ukulandelana kwamakhodi amakhodi ahlukene.

Kungani i-VQGAN yengeza i-GAN yokubandlulula phezu kwe-VQVAE?

Ukulahlekelwa okuphikisayo nokwengqondo kuvumela i-VQGAN ukuthi yakhe kabusha izithombe ezihlanzekile kusukela kugridi yethokheni ehlangene, i-VQVAE iyodwa evame ukuyifiphalisa.

Uma isithombe siwukulandelana kwamathokheni, iyiphi imodeli ekhiqiza izithombe ezintsha?

Ngenxa yokuthi izithombe ziba ukulandelana kwamathokheni ahlukahlukene, i-transformer ingakwazi ukuyimodela ngokuzenzakalelayo, njengokwenza umbhalo.

Iyiphi inqubo evumela ama-gradient ageleze esinyathelweni sokulinganisa esingahlukanisiwe?

Ukwenziwa kwe-Vector quantization akuhlukaniseki, ngakho-ke i-VQGAN isebenzisa isilinganiso esiqondile se-gradient ukuqeqesha isishumeki.

Imuphi umkhuba wobuciko odumile we-AI i-VQGAN esiza ukuyidala ngo-2021?

Ukumatanisa i-VQGAN nesiqondiso se-CLIP kukhiqize ubuciko okwabelwana ngaso kabanzi 'be-VQGAN+CLIP' obandisa ukukhiqizwa kwezithombe eziqhutshwa umbhalo.