Riffusion Spectrogram Diffusion
Riffusion itsotsi rakangwara rinogadzira mimhanzi nekubata ruzha semufananidzo: inokwenenzvera iyo Yakagadzika Diffusion mufananidzo modhi kupenda spectrogram, yozoshandura iyo mifananidzo kudzoka kuita odhiyo.
Pfupiso
It matters because it shows a tool built for one medium (images) can produce another (music) with almost no new architecture.
Kudzika Kwakadzika
Riffusion, yakaburitswa mukupera kwa2022 naSeth Forsgren naHayk Martiros, yakatanga sechirongwa chekuzvivaraidza. Iwo musimboti hunyengeri: spectrogram mufananidzo we 2D uko horizontal axis inguva, vertical axis is frequency, uye pixel kupenya kune ruzha. Sezvo Stable Diffusion yatogadzira mifananidzo kubva kumashoko ekukurudzira, vagadziri vakaigadzirisa zvakanaka pazviuru zvepeyina spectrogram-zvinyorwa mienzaniso. Ikurudzire ne 'funky jazz bass' uye inoburitsa ruzha rusina kurongeka muratidziro yeruzha irworwo. Kugadzira odhiyo inotamba, Riffusion inomhanyisa iyo spectrogram kuburikidza neGriffin-Lim algorithm inovakazve iyo isipo ruzivo rwechikamu. Nekuti kupararira kunogona kupindirana zvakanaka pakati pekukurudzira, Riffusion inogona zvakare morph imwe dhizaini kuita imwe pamusoro peinoenderera clip clip, ichitenderera isina musono.
Technical Insight
Riffusion inoshandisazve pombi yekuparadzira yakavanda isina kuchinjika: U-Net inodzokorodza inobvisa ruzha rweGaussian kubva pamufananidzo uripo wakamisikidzwa pane CLIP yekumisikidza mavara. Iyo chete domain-chaiyo basa ndeye spectrogram inomiririra (mel-scale, log simba) uye Griffin-Lim chikamu chekuvakazve chinoshandura chakafanotaurwa magnitude spectrogram kudzoka kuita waveform. Phase inoraswa panguva yekukodha, saka Griffin-Lim's iterative fungidziro ndiyo huru sosi yehunhu 'yemvura' zvigadzirwa.
Strategic Impact
Svika uye svika
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza.
Mutengo uye bhajeti
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki.
Kumhanya uye chiyero
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru.
Ramangwana reRiffusion Spectrogram Diffusion
Riffusion yakaratidza kuti spectrogram-se-image bhiriji inoshanda, uye iyo pfungwa ikozvino inogara mukati mehombe maodhiyo masisitimu uye kambani Riffusion yakava. Tarisira maturusi emangwana ekutsiva akarasika Griffin-Lim neakadzidza neural vocoder yechikamu chekuchenesa, uye kubatanidza spectrogram diffusion nelatent audio codecs. Chidzidzo chakafara, icho mamodheru emifananidzo anogona kudzoserwa kumaitiro matsva, anoenderera mberi nekupesvedzera kuti vaongorori bootstrap odhiyo uye vhidhiyo jenareta kubva kune iripo yakadzidziswa musana.
Real-World Implementation
Kugadzira mapfupi mafupi ekumashure emitambo yevhidhiyo yeindie kubva pamashoko anokurumidza senge 'tense synthwave chase'.
Morphing zvakanaka pakati pezvitaera zviviri zvekuimba, semuenzaniso. kusanganisa 'tropical house' kuita 'lo-fi hip hop' pachikamu chimwe chete
Kugadzira mibhedha yemimhanzi yeYouTube mavhidhiyo uye mapodcasts pasina mari yerezinesi.
Prototyping melodic kana rhythmic mazano ayo muimbi anobva anyora zvakare zvakanaka mudhijitari audio workstation.
Njodzi & Guardrails
Kushandisa izwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo.
Kururama kunogona kudonha mumitauro, mataurirwo, kana nharaunda dzine ruzha.
Synthetic audio inogona kukanganisa kutaura kwechokwadi isina mavara akajeka.
Implementation Roadmap
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Riffusion Spectrogram Diffusion quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
DiffWave Diffusion Vocoder
Mibvunzo inowanzo bvunzwa
What is Riffusion Spectrogram Diffusion?
Riffusion itsotsi rakangwara rinogadzira mimhanzi nekubata ruzha semufananidzo: inokwenenzvera iyo Yakagadzika Diffusion mufananidzo modhi kupenda spectrogram, yozoshandura iyo mifananidzo kudzoka kuita odhiyo. Izvo zvine basa nekuti inoratidza chishandiso chakavakirwa imwe svikiro (mifananidzo) inogona kuburitsa imwe (mimhanzi) ine inenge isina zvivakwa zvitsva.
Ndeipi iripo AI modhi yakaita Riffusion yakanaka-tune kugadzira mimhanzi?
Riffusion yakanyatso-tuned Stable Diffusion, mufananidzo wekuparadzira modhi, kugadzira mifananidzo yespectrogram inozoshandurwa kuita odhiyo.
Ko spectrogram inomirirei pamademo ayo maviri?
Mune spectrogram axis yakachinjika inguva, vertical axis i frequency, uye kupenya kunomiririra ruzha.
Nei Riffusion ichida algorithm senge Griffin-Lim?
Iyo spectrogram inochengetedza ukuru asi inorasa chikamu, saka Griffin-Lim anofungidzira zvakare chikamu chekuvakazve inotamba waveform.
Ndeupi hunyanzvi hunopa Riffusion kana uchibatanidza zvirevo zviviri?
Diffusion modhi inogona kupindirana munzvimbo yakanyarara, ichirega Riffusion morph ichienderera kubva kune imwe mimhanzi kuenda kune imwe.
Riffusion yakatanga kusikwa nekuburitswa sei?
Riffusion yakatanga sechirongwa chekuzvivaraidza naSeth Forsgren naHayk Martiros, chakaburitswa pachena mukupera kwa2022.