Nchịkọta onyonyo VQGAN na Codebook
VQGAN na-akwakọba onyogho n'ime grid nke akara ngosi dị iche nke ewepụtara na koodu codebook amụtara, na-ahapụ onye ntụgharị ka ọ mepụta onyonyo otu ụdị asụsụ si ewepụta ederede.
Ime miri emi
VQGAN, ewebata na 2021 akwụkwọ 'Taming Transformers for High-Resolution Image Synthesis,' na-ejikọta vector-quantized autoencoder (VQVAE) na ọzụzụ mmegide na nghọta. Ihe ntinye koodu na-ese onyinyo n'obere grid nke vectors atụmatụ; A na-etinye vector ọ bụla na ntinye kacha nso n'ime akwụkwọ koodu amụtara, sịnụ, koodu 1024 pụrụ iche, na-atụgharị onyonyo ka ọ bụrụ usoro nke akara integer. Onye nrụpụta na-ewughachi onyonyo site na akara ngosi ndị ahụ, zụrụ ya site na ịkpa oke GAN na nhụsianya n'ihi na nrụgharị ahụ na-adị nkọ karịa ka ọ dị nkọ. N'ihi na onyonyo ugbu a bụ usoro akara ngosi pụrụ iche, ihe ntụgharị autoregressive nwere ike ịṅomi ha dị ka asụsụ, na-ebu amụma n'otu n'otu. VQGAN kwadoro ngwa ọrụ nka ederede gaa na onyonyo mgbe ejikọtara ya na ntuziaka CLIP.
Nghọta nka nka
Ọrụ bụ isi bụ ọnụọgụ vector: a na-eji vectors codebook kacha nso nọchiri mpụta ihe ngbanwe na-aga n'ihu, na-eji ihe nleba anya 'straight-through' gradient estimator nke mere na koodu ahụ ka nwere ike ịmụta n'agbanyeghị nleba anya na-enweghị iche. Ịgbakwunye onye ịkpa ókè GAN dabere na patch n'elu autoencoder bụ ihe na-ahapụ VQGAN iji grid token pere mpe (dịka 16 × 16) karịa VQVAE ka ọ na-edobe textures crisp, na-eme ka ihe ngosi nke ntụgharị.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Visual AI nwere ike megharịa nyocha, nchọpụta na mkpado ọrụ n'ọtụtụ.
Mee nhọrọ
Otu ndị na-emepụta ihe nwere ike imepụta echiche ngwa ngwa site na ngbanwe akwụkwọ ntuziaka ole na ole.
Team na usoro ọrụ
Ọrụ nwere ike iji onyonyo na akara vidiyo siri ike ịhazi.
Ọdịnihu nke VQGAN na Synthesis Image Codebook
Ntụziaka pụrụ iche nke VQGAN ghọrọ ntọala maka onyonyo dabere na token na ụdị vidiyo, site na MaskGIT ruo sistemu multimodal na-agwakọta onyonyo na akara ederede n'otu ngbanwe. Nchọpụta na-akwalite ugbu a gaa n'akwụkwọ codebook buru ibu, nke nwere oke ma ọ bụ nchọta na-enweghị akwụkwọ, nke na-ezere ịdaba akwụkwọ koodu yana gaa n'ụdị ejikọtara ọnụ ebe otu okwu na-agbasa onyonyo, ọdịyo, na asụsụ, na-enyere ọgbọ ọ bụla aka.
Mmejuputa n'ezie n'ụwa
Idonye foto n'ime grid 16x16 nke akara koodu ka onye ntụgharị nwere ike ịdepụta ma megharịa ya.
Ijikọ VQGAN na ntuziaka CLIP iji mepụta nka 'VQGAN+CLIP' AI nke na-efe efe na 2021
Na-akpakọ onyonyo n'ime koodu kọmpat pụrụ iche maka nchekwa dị mma ma ọ bụ ọzụzụ mmepụta ala
Na-eje ozi dị ka ihe onyonyo onyonyo n'ime ndị na-emepụta ihe dabere na token dị ka MaskGIT na ndị na-agbanwe agbanwe.
Ihe ize ndụ & okporo ụzọ nche
Ikike onyonyo na nkwenye nwere ike bụrụ ihe egwu dị n'iwu ma ọ bụrụ na edoghị anya.
Ọrụ nlereanya nwere ike ịdịgasị iche n'ofe ọkụ, igwe mmadụ, na gburugburu.
Enwere ike ghara ịhụ ihe dị mma ma ọ bụrụ na enyochaghị oke ntụkwasị obi.
Map mmejuputa
Kọwaa ụkpụrụ nnabata maka nkenke, icheta, na ụgwọ njehie.
Nwalee na data dabara na ọnọdụ mmepụta n'ezie.
Tinye nyocha mmadụ maka obere obi ike ma ọ bụ amụma mmetụta dị elu.
Sochie ihe nlere anya wee megharịa ka emechara mgbanwe igwefoto ma ọ bụ dataset.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VQGAN and Codebook Image Synthesis quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Nchịkọta onyonyo Semantic SPADE
Ajụjụ a na-ajụkarị
What is VQGAN and Codebook Image Synthesis?
VQGAN na-akwakọba onyogho n'ime grid nke akara ngosi dị iche nke ewepụtara na koodu codebook amụtara, na-ahapụ onye ntụgharị ka ọ mepụta onyonyo otu ụdị asụsụ si ewepụta ederede.
Kedu ihe VQGAN na-eji nọchite anya onyonyo dị ka akara ngosi pụrụ iche?
VQGAN na-atụle njirimara koodu n'ime ndenye kacha nso na akwụkwọ koodu amụtara, na-atụgharị onyonyo ka ọ bụrụ usoro nrịbama koodu pụrụ iche.
Kedu ihe kpatara VQGAN ji agbakwunye onye ịkpa oke GAN n'elu VQVAE?
Mfu mmegide na nghọta na-ahapụ VQGAN wughachi ihe onyonyo kpụ ọkụ n'ọnụ site na grid token kọmpat, nke VQVAE naanị na-agbaji.
Ozugbo onyonyo bụ usoro nke token, kedu ihe nlereanya na-ewepụta onyonyo ọhụrụ?
N'ihi na onyogho na-aghọ usoro akara ngosi pụrụ iche, onye ntụgharị nwere ike ịṅomi ha na-emegharị anya, dị ka ịmepụta ederede.
Kedu usoro na-enye ohere ka gradients na-agafe n'usoro nha nha na-enweghị iche?
Ngụkọta vector abụghị ihe dị iche, yabụ VQGAN na-eji ihe nleba anya gradient kwụ ọtọ iji zụọ ihe ngbanwe.
Kedu ihe ama ama AI nka VQGAN nyere aka mepụta na 2021?
Ijikọ VQGAN na ntụzịaka CLIP rụpụtara nka 'VQGAN+CLIP' nke na-ekesa n'ọtụtụ ebe nke mere ka ọgbọ onyonyo na-akwalite.