VQ-VAE da Latents masu hankali
VQ-VAE tana matsa hotuna, sauti, ko bidiyo zuwa cikin ƙaramin grid na lambobi masu hankali waɗanda aka zana daga littafin lambar koyo, maimakon lambobi masu ci gaba.
Dubawa
This discrete bottleneck lets powerful sequence models like Transformers treat media as 'tokens', much like words.
Zurfafa nutsewa
VQ-VAE (Vector Quantized Variational Autoencoder), wanda van den Oord da abokan aiki a DeepMind suka gabatar a cikin 2017, wani autoencoder ne wanda keɓaɓɓen sarari yake. Mai rikodin rikodi yana juya hoto zuwa grid na vectors masu ci gaba; kowane vector sai a tsinke shi zuwa mafi kusa shigarsa a cikin wani koyo na codebook na embeddings (vector quantization). Mai yankewa yana sake gina hoton daga waɗannan lambobin da aka ƙididdige su. Saboda latents yanzu ƙayyadaddun ƙamus ne na fihirisa, wani keɓantaccen samfurin zai iya koyan rarraba su kuma ya samar da sabon abun ciki. Wannan girke-girke na mataki biyu yana iko da DALL-E 1, Jukebox don kiɗa, da VQGAN, wanda ke ƙara hasarar fahimta da ƙiyayya don sake ginawa. VQ-VAE-2 ta tattara kudurori da yawa don samar da hotuna masu inganci.
Fahimtar Fasaha
Matakin ƙididdigewa (neman makwabcin kusa-argmin) ba shi da bambanci, don haka VQ-VAE yana amfani da madaidaicin ƙididdigewa: ana kwafi gradients kai tsaye daga shigar da dikodi baya zuwa fitarwar encoder kamar ƙididdigewa shine ainihin. Horon ya haɗu da asarar sake ginawa, asarar littafin rikodin abubuwan da ke jawo abubuwan haɗawa zuwa abubuwan da aka haɗa, da kuma asarar sadaukarwa da ke kiyaye rikodin rikodi zuwa lambobin da aka zaɓa. Rashin gazawar gama gari shine rugujewar littafin, inda ake amfani da ƴan lambobi.
Dabarun Tasiri
Gudu da sikelin
Kayayyakin AI na iya sarrafa aiki da bincike, ganowa, da ayyuka masu alama a sikelin.
Gina zaɓuɓɓuka
Ƙungiyoyin ƙirƙira za su iya samar da ra'ayoyi cikin sauri tare da ƙarancin bita da hannu.
Ƙungiya da aikin aiki
Ayyuka na iya amfani da siginar hoto da bidiyo waɗanda a baya suke da wahalar aiwatarwa.
Makomar VQ-VAE da Latent masu hankali
Latent masu hankali sune tsakiyar turawa zuwa ga ingantattun samfura masu yawa waɗanda ke nuna hotuna, sauti, da bidiyo zuwa cikin ƙamus iri ɗaya da rubutu. Haɓakawa kamar saura da ƙayyadaddun ƙididdige ƙididdigewa, manyan littattafan rubutu, da ingantacciyar daidaita amfani suna rage rugujewa da haɓaka aminci. Kamar yadda samfuran ke da niyya ga duka biyun fahimta da samar da hanyoyi daban-daban, ƙaƙƙarfan tokenizers waɗanda aka gina akan ra'ayoyin VQ-VAE za su kasance wani sinadari mai tushe, ƙara fafatawa da haɗawa tare da ci gaba da hanyoyin watsa labarai na ɓoye.
Aiwatar da Gaskiyar Duniya
DALL-E 1 ya yi amfani da tambarin VQ-VAE mai hankali don haka Transformer zai iya samar da hotuna azaman jerin fihirisar codebook.
VQGAN ya haɗe VQ-VAE tare da hasarar gaba da hasashe don samar da kintsattse, manyan alamun hoto don tsara fasaha.
OpenAI's Jukebox ya yi amfani da VQ-VAE zuwa ga danyen sauti, matsar da kida cikin keɓaɓɓen lambobin don ƙirar ƙira.
VQ-VAE-2 ya tattara manyan latents masu ma'ana don haɗa nau'ikan hotuna masu aminci da yawa masu adawa da GANs na zamanin sa.
Hatsari & Tsare-tsare
Haƙƙoƙin hoto da yarda na iya zama haxarin doka idan ba a fayyace ba.
Ayyukan samfuri na iya bambanta a ko'ina cikin haske, ƙididdiga, da mahalli.
Ƙarya tabbataccen ƙila ba za a iya lura da shi ba sai dai idan an kula da ƙofofin amincewa.
Taswirar Hanya
Ƙayyade ma'auni na karɓa don daidaito, tunowa, da farashi na kuskure.
Gwada tare da bayanan da suka dace da ainihin yanayin samarwa.
Ƙara bita na ɗan adam don ƙarancin amincewa ko tsinkaya mai tasiri.
Bi diddigin ƙirar ƙira kuma sake ingantawa bayan canje-canjen kamara ko saitin bayanai.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the VQ-VAE and Discrete Latents quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Haɗewar Latent da Haɗin Hoto
Tambayoyin da ake yawan yi
What is VQ-VAE and Discrete Latents?
VQ-VAE tana matsa hotuna, sauti, ko bidiyo zuwa cikin ƙaramin grid na lambobi masu hankali waɗanda aka zana daga littafin lambar koyo, maimakon lambobi masu ci gaba. Wannan ƙwaƙƙwaran ƙwanƙwasa yana ƙyale samfuran jeri masu ƙarfi kamar Transformers su ɗauki kafofin watsa labarai azaman 'alamu', kamar kalmomi.
Menene ke sa sararin ɓoye na VQ-VAE ya bambanta da daidaitaccen VAE?
VQ-VAE yana maye gurbin ci gaba da latents tare da keɓaɓɓun lambobi waɗanda aka zana daga littafin koyo ta hanyar ƙididdigewa.
Ta yaya VQ-VAE ke ƙetare gradients ta hanyar ƙididdige matakan da ba na bambanta ba?
Mai ƙididdigewa kai tsaye yana kwafin gradient-input ɗin shigar da ƙara kai tsaye zuwa kayan fitarwa, yana ɗaukar ƙididdigewa azaman ainihi don fasfo na baya.
Menene 'Rushewar Codebook'?
Rushewar littafin Codebook yana faruwa lokacin da ƙirar ta dogara da ƴan lambobi, ɓata mafi yawan littafin da cutar da bambancin.
Me yasa keɓaɓɓen latents ke da amfani don ƙirar ƙirƙira tare da Transformers?
Fihirisa masu hankali suna samar da ƙayyadaddun ƙamus, don haka nau'ikan jeri irin su Transformers za su iya koyo da samfurin rarraba su kamar kalmomi.
Menene VQGAN ya ƙara a saman ainihin girke-girke na VQ-VAE?
VQGAN yana haɓaka VQ-VAE tare da mai nuna wariya da hasarar fahimta, yana ba da fa'ida, ƙarin cikakkun bayanai da aka sake ginawa.