Mgbasa spectrogram Riffusion
Riffusion bụ mbanye anataghị ikike nke na-ewepụta egwu site n'imeso ụda dị ka eserese: ọ na-emegharị ihe oyiyi Stable Diffusion ka ọ na-ese spectrograms, wee gbanwee onyonyo ndị ahụ ka ọ bụrụ ọdịyo.
Nchịkọta
It matters because it shows a tool built for one medium (images) can produce another (music) with almost no new architecture.
Ime miri emi
Riffusion, nke Seth Forsgren na Hayk Martiros weputara na ngwụcha 2022, malitere dị ka ọrụ ntụrụndụ. Isi aghụghọ: spectrogram bụ onyonyo 2D ebe axis kwụ ọtọ bụ oge, axis kwụ ọtọ bụ ugboro, na nchapụta pixel bụ oke. Ebe ọ bụ Stable Diffusion ewepụtala onyonyo site na mkpali ederede, ndị okike deziri ya nke ọma na puku kwuru puku ihe atụ ederede spectrogram jikọtara ọnụ. Jiri 'funky jazz bass' kpalite ya ma ọ na-ekwupụta mkpọtụ enweghị usoro n'ime spectrogram nke ụda ahụ. Iji mee ọdịyo enwere ike ịkpọ, Riffusion na-eme spectrogram ahụ site na Griffin-Lim algọridim nke na-ewughachi ozi oge na-efu efu. N'ihi na mgbasa ozi nwere ike imekọrịta nke ọma n'etiti mkpali, Riffusion nwekwara ike gbanwee otu ụdị ka ọ bụrụ nke ọzọ karịa vidiyo na-aga n'ihu, na-agbagharị n'enweghị nsogbu.
Nghọta nka nka
Riffusion na-ejighachi pipeline mgbasa ozi latent anaghị agbanwe agbanwe: U-Net na-ewepụ mkpọtụ Gaussian n'otu ntabi anya na onyonyo CLIP etinyere. Naanị ọrụ a kapịrị ọnụ ngalaba bụ ihe nnọchi anya spectrogram (mel-scale, log power) na nrụgharị oge Griffin-Lim nke na-atụgharị spectrogram dị ukwuu e buru n'amụma laghachi azụ n'ụdị ifegharị. A na-atụfu usoro n'oge itinye koodu, yabụ ntule ntule nke Griffin-Lim bụ isi mmalite nke njirimara 'mmiri'.
Mmetụta atụmatụ
Nweta na iru
Ọ na-eme ka nnweta ya dịkwuo mma site na ndegharị, ịkọ akụkọ, na ntụgharị olu.
Ọnụ ego na mmefu ego
Ndị otu mgbasa ozi nwere ike ibubata ọdịyo a na-egbu maramara ngwa ngwa site na iji obere mmefu ego.
Ọsọ na ọnụ ọgụgụ
Sistemụ na-eche ihu ndị ahịa nwere ike hazie mkparịta ụka n'ọtụtụ buru ibu.
Ọdịnihu nke Mgbasa Spectrogram Riffusion
Riffusion gosipụtara àkwà mmiri spectrogram-dị ka ihe oyiyi na-arụ ọrụ, na echiche ahụ na-adị ugbu a n'ime sistemụ ọdịyo ka ukwuu na ụlọ ọrụ Riffusion ghọrọ. Na-atụ anya ngwaọrụ ga-eme n'ọdịnihu iji dochie Griffin-Lim na-efu efu na vocoders mmụta akwara maka usoro dị ọcha, yana ijikọ mgbasa ozi spectrogram na codecs audio latent. Ihe nkuzi sara mbara, enwere ike ibugharị ụdị onyonyo ahụ gaa na usoro ọhụrụ, na-aga n'ihu na-emetụta ka ndị na-eme nchọpụta na-ebupụ ndị na-emepụta ihe na vidiyo site na ọkpụkpụ azụ azụrụla mbụ.
Mmejuputa n'ezie n'ụwa
Na-emepụta egwu ndabere dị mkpụmkpụ maka egwuregwu vidiyo indie site na ngwa ngwa ederede dị ka 'tense synthwave chase'
Morphing were were were n'etiti ụdị egwu abụọ, dịka. na-agwakọta 'ụlọ okpomọkụ' n'ime 'lo-fi hip hop' n'ofe otu vidiyo
Na-emepụta akwa egwu ihe nkiri na-enweghị eze maka vidiyo YouTube na pọdkastị na-enweghị ụgwọ ikike
Ntugharị egwu egwu ma ọ bụ echiche rhythmic nke onye egwu na-edegharịghachi nke ọma na ebe a na-eme ọdịyo dijitalụ.
Ihe ize ndụ & okporo ụzọ nche
Iji olu eme ihe na ihe egwu mpụta ga-abawanye mgbe nkwenye na-efu.
Izi ezi nwere ike ịdaba n'ofe ụda olu, olumba ma ọ bụ gburugburu mkpọtụ.
Enwere ike imehie ọdịyo sịntetik dị ka ezigbo okwu na-enweghị akara doro anya.
Map mmejuputa
Nweta nkwenye doro anya maka ijide olu, imechi, na ijigharị.
Nwale ogo n'ofe ndị na-ekwu okwu dị iche iche yana ọnọdụ ndabere.
Kọwaa mgbe mmadụ ga-enyocha ma ọ bụ kwado nsonye.
Deba aha ọdịyo sịntetik ma debe ndekọ ihe ndekọ maka ịza ajụjụ.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Riffusion Spectrogram Diffusion quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Vocoder DiffWave Diffusion
Ajụjụ a na-ajụkarị
What is Riffusion Spectrogram Diffusion?
Riffusion bụ mbanye anataghị ikike nke na-ewepụta egwu site n'imeso ụda dị ka eserese: ọ na-emegharị ihe oyiyi Stable Diffusion ka ọ na-ese spectrograms, wee gbanwee onyonyo ndị ahụ ka ọ bụrụ ọdịyo. Ọ dị mkpa n'ihi na ọ na-egosi ngwá ọrụ e wuru maka otu ọkara (ihe oyiyi) nwere ike ịmepụta ọzọ (egwu) na ọ fọrọ nke nta ka ọ bụrụ ihe ọhụrụ ọ bụla.
Kedu ihe nlereanya AI dị ugbu a ka Riffusion mere nke ọma iji mepụta egwu?
Riffusion nke ọma na-ekiri Stable Diffusion, ụdị mgbasa ozi onyonyo, iji mepụta onyonyo spectrogram nke agbanweziri ka ọ bụrụ ọdịyo.
Kedu ihe spectrogram na-anọchite anya anyụike ya abụọ?
Na spectrogram axis kehoraizin bụ oge, axis kwụ ọtọ bụ ugboro, na nchapụta na-anọchi anya ụda olu.
Kedu ihe kpatara Riffusion ji chọọ algọridim dị ka Griffin-Lim?
Ihe nlere anya na-echekwa oke mana ọ na-atụfu oge, yabụ Griffin-Lim na-atụpụta oge iji wughachi usoro ifegharị egwu.
Kedu ikike mgbasa ozi na-enye Riffusion mgbe ị na-agwakọta mkpali abụọ?
Ụdị mgbasa ozi nwere ike ịbanye na oghere nzuzo, na-ahapụ Riffusion morph na-aga n'ihu site n'otu ụdị egwu gaa na nke ọzọ.
Kedu ka esi emepụta Riffusion na mbụ ma hapụ ya?
Riffusion malitere dị ka ihe omume ntụrụndụ Seth Forsgren na Hayk Martiros, wepụtara n'ihu ọha na ngwụcha 2022.