VQ-VAE na latent dị iche
VQ-VAE na-echikota onyonyo, ọdịyo, ma ọ bụ vidiyo n'ime obere grid nke koodu pụrụ iche ewepụtara na koodu koodu amụtara, kama ọnụọgụgụ na-aga n'ihu.
Nchịkọta
This discrete bottleneck lets powerful sequence models like Transformers treat media as 'tokens', much like words.
Ime miri emi
VQ-VAE (Vector Quantized Variational Autoencoder), nke van den Oord na ndị ọrụ ibe ya na DeepMind webatara na 2017, bụ onye na-ede koodu autoen nke oghere ya nwere ezi uche. Ihe ngbanwe na-atụgharị onyonyo ka ọ bụrụ grid nke vector na-aga n'ihu; A na-etinyezi vector nke ọ bụla na ntinye kacha nso na akwụkwọ ntinye akwụkwọ mmụta (vector quantization). Ihe ngbanwe ahụ na-ewughachi onyonyo a site na koodu ndị ahụ agbapụtara. N'ihi na latent bụzi mkpụrụokwu nwere oke nke indices, ụdị dị iche nwere ike mụta nkesa ha wee mepụta ọdịnaya ọhụrụ. Ntụziaka nke agba abụọ a na-enye ike DALL-E 1, Jukebox maka egwu, na VQGAN, nke na-agbakwunye mfu nghọta na mgbagha maka nrụzigharị dị nkọ. VQ-VAE-2 chịkọtara ọtụtụ mkpebi iji mepụta onyonyo nwere ntụkwasị obi dị elu.
Nghọta nka nka
Nzọụkwụ quantization (argmin nso-agbataobi nchọta) abụghị ihe dị iche, ya mere VQ-VAE na-eji a ogologo-site estimator: gradients na-e depụtaghachiri ozugbo site decoder ntinye azụ na encoder mmepụta dị ka a ga-asị na quantization bụ njirimara. Ọzụzụ na-ejikọta mfu nrụzigharị, mfu codebook na-adọta ntinye na ntinye koodu, yana mfu ntinye aka na-edobe koodu ntinye aka na koodu ọ họọrọ. Ọdịda na-adịkarị bụ ọdịda codebook, ebe a na-eji naanị koodu ole na ole.
Mmetụta atụmatụ
Ọsọ na ọnụ ọgụgụ
Visual AI nwere ike megharịa nyocha, nchọpụta na mkpado ọrụ n'ọtụtụ.
Mee nhọrọ
Otu ndị na-emepụta ihe nwere ike imepụta echiche ngwa ngwa site na ngbanwe akwụkwọ ntuziaka ole na ole.
Team na usoro ọrụ
Ọrụ nwere ike iji onyonyo na akara vidiyo siri ike ịhazi.
Ọdịnihu nke VQ-VAE na Latent pụrụ iche
Latent dị iche iche bụ isi ihe na-aga n'ihu n'ụdị multimodal jikọtara ọnụ nke na-eme ka onyonyo, ọdịyo na vidiyo bụrụ otu okwu dị ka ederede. Mmelite dị ka mbelata na mbelata scalar quantization, nnukwu codebooks, na nhazi nke ojiji ka mma na-ebelata ndakpọ na ịkwalite ntụkwasị obi. Dị ka ụdị na-achọ ịghọta ma mepụta n'ofe usoro, tokenizers siri ike wuru na echiche VQ-VAE ga-anọgide na-abụ ihe ndabere, na-asọ mpi na ijikọta na nso nso a na-aga n'ihu.
Mmejuputa n'ezie n'ụwa
DALL-E 1 jiri ihe pụrụ iche VQ-VAE tokenizer ka onye ntụgharị nwee ike iwepụta onyonyo dịka usoro nke indices codebook.
VQGAN jikọtara VQ-VAE na nhụsianya na nhụsianya na-efunahụ iji mepụta akara ngosi onyonyo dị elu, nke dị elu maka ọgbọ nka.
OpenAI's Jukebox tinye VQ-VAE na ọdịyo raw, na-atụgharị egwu n'ime koodu ndị pụrụ iche maka imepụta ụdị.
VQ-VAE-2 kpokọtara latents hierarchical discrete iji mepụta ụdị dị iche iche, onyonyo ntụkwasị obi dị elu na-emegide GAN nke oge ya.
Ihe ize ndụ & okporo ụzọ nche
Ikike onyonyo na nkwenye nwere ike bụrụ ihe egwu dị n'iwu ma ọ bụrụ na edoghị anya.
Ọrụ nlereanya nwere ike ịdịgasị iche n'ofe ọkụ, igwe mmadụ, na gburugburu.
Enwere ike ghara ịhụ ihe dị mma ma ọ bụrụ na enyochaghị oke ntụkwasị obi.
Map mmejuputa
Kọwaa ụkpụrụ nnabata maka nkenke, icheta, na ụgwọ njehie.
Nwalee na data dabara na ọnọdụ mmepụta n'ezie.
Tinye nyocha mmadụ maka obere obi ike ma ọ bụ amụma mmetụta dị elu.
Sochie ihe nlere anya wee megharịa ka emechara mgbanwe igwefoto ma ọ bụ dataset.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VQ-VAE and Discrete Latents quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Ngwakọta nzuzo na njikọta onyonyo
Ajụjụ a na-ajụkarị
What is VQ-VAE and Discrete Latents?
VQ-VAE na-echikota onyonyo, ọdịyo, ma ọ bụ vidiyo n'ime obere grid nke koodu pụrụ iche ewepụtara na koodu koodu amụtara, kama ọnụọgụgụ na-aga n'ihu. Nke a pụrụ iche bottlene na-ahapụ ike usoro usoro dị ka Transformers na-emeso mgbasa ozi dị ka 'token', dị nnọọ ka okwu.
Kedu ihe mere oghere oghere VQ-VAE dị iche na ọkọlọtọ VAE?
VQ-VAE na-eji koodu pụrụiche ewepụtara site na akwụkwọ koodu a mụtara site na ọnụọgụ vector na-eji dochie latent na-aga n'ihu.
Kedu ka VQ-VAE si agafe gradients site na usoro ngụpụta na-enweghị iche?
Ihe nnleba anya kwụ ọtọ na-eṅomi gradient ntinye ntinye koodu ozugbo gaa na mmepụta koodu, na-ewere ọnụọgụ dị ka njirimara maka ngafe azụ.
Kedu ihe bụ 'codebook collapse'?
Codebook ọdịda na-eme mgbe ihe nlereanya ahụ dabere na koodu ole na ole, na-emebi ọtụtụ n'ime codebook na-emerụ ụdị dị iche iche.
Kedu ihe kpatara latent pụrụ iche ji baa uru maka imepụta ihe n'ichepụta ihe na Transformers?
Indices dị iche iche na-etolite mkpụrụokwu nwere oke, yabụ ụdị usoro dịka Transformers nwere ike mụta ma lelee nkesa ha dị ka okwu.
Kedu ihe VQGAN gbakwunyere n'elu ntụzịaka VQ-VAE bụ isi?
VQGAN na-abawanye VQ-VAE site na ịkpa oke na nhụsianya, na-ekwenye ekwenye na akara ngosi arụgharịrị zuru oke.