Riffusion Spectrogram Diffusion
Riffusion hack ne mai wayo wanda ke haifar da kida ta hanyar ɗaukar sauti azaman hoto: yana daidaita ƙirar hoton Stable Diffusion don fenti spectrograms, sannan ya canza waɗannan hotunan zuwa sauti.
Dubawa
It matters because it shows a tool built for one medium (images) can produce another (music) with almost no new architecture.
Zurfafa nutsewa
Riffusion, wanda Seth Forsgren da Hayk Martiros suka fitar a ƙarshen 2022, an fara ne azaman aikin sha'awa. Babban abin zamba: spectrogram hoto ne na 2D inda axis a kwance lokaci ne, axis na tsaye shine mita, kuma hasken pixel shine ƙara. Tun da Stable Diffusion ya riga ya samar da hotuna daga faɗakarwar rubutu, masu ƙirƙira sun daidaita shi akan dubban misalan rubutun-spectrogram guda biyu. Ƙirƙiri shi da 'funky jazz bass' kuma yana musanta hayaniyar bazuwar cikin siffar wannan sautin. Don yin sauti mai iya kunnawa, Riffusion yana gudanar da sikirin ta hanyar Griffin-Lim algorithm wanda ke sake gina bayanan lokaci da ya ɓace. Saboda yadawa na iya shiga tsakani a hankali tsakanin faɗakarwa, Riffusion kuma na iya jujjuya salo ɗaya zuwa wani akan ci gaba da faifan bidiyo, yana zazzagewa ba tare da wata matsala ba.
Fahimtar Fasaha
Riffusion yana sake yin amfani da bututun watsawa mai ɓoye ba canzawa: U-Net yana kawar da hayaniyar Gaussian a hankali daga hoton ɓoye wanda aka sharadi akan rubutun CLIP. Iyakar takamaiman aikin yanki shine wakilcin spectrogram (mel-sikelin, ikon log) da sake gina lokaci na Griffin-Lim wanda ke juyar da sikelin girman girman da aka annabta a baya zuwa tsarin igiyar ruwa. An watsar da mataki yayin sanyawa, don haka kimantawar Griffin-Lim shine babban tushen halayen kayan tarihi na 'ruwa'.
Dabarun Tasiri
Shiga ku isa
Yana inganta samun dama ta hanyar rubutu, ba da labari, da mu'amalar murya.
Kudin da kasafin kuɗi
Ƙungiyoyin kafofin watsa labaru na iya jigilar sauti mai gogewa cikin sauri tare da ƙaramin kasafin kuɗi.
Gudu da sikelin
Tsarin fuskantar abokin ciniki na iya aiwatar da hulɗar magana a mafi girman ma'auni.
Makomar Riffusion Spectrogram Diffusion
Riffusion ya tabbatar da aikin gada-kamar hoto, kuma wannan ra'ayin yana rayuwa a cikin manyan tsarin sauti kuma kamfanin Riffusion ya zama. Yi tsammanin kayan aikin nan gaba don maye gurbin Griffin-Lim mai asara tare da koyan vocoders na jijiyoyi don tsaftataccen lokaci, da kuma haɗa watsawar bakan gizo tare da latent codecs audio. Babban darasi, cewa za a iya karkatar da samfuran hoto zuwa sabbin hanyoyin, yana ci gaba da yin tasiri kan yadda masu bincike ke ɗaure sauti da janareta na bidiyo daga ƙasusuwan da aka riga aka horar da su.
Aiwatar da Gaskiyar Duniya
Ƙirƙirar gajerun waƙoƙin bayan fage don wasannin bidiyo na indie daga saƙon rubutu kamar 'tense synthwave chase'
Zazzagewa cikin kwanciyar hankali tsakanin salon kiɗa biyu, misali. hadawa 'gidan wurare masu zafi' zuwa 'lo-fi hip hop' a fadin faifan hoto guda daya
Samar da gadaje na kiɗan yanayi mara sarauta don bidiyon YouTube da kwasfan fayiloli ba tare da kuɗin lasisi ba
Samar da ra'ayoyin waƙoƙi ko rhythmic wanda mawaƙi zai sake yin rikodi yadda ya kamata a cikin aikin sauti na dijital.
Hatsari & Tsare-tsare
Rashin amfani da murya da haɗarin kwaikwaya yana ƙaruwa lokacin da aka rasa izini.
Daidaituwa na iya faɗuwa cikin lafuzza, yaruka, ko mahalli masu hayaniya.
Ana iya kuskuren sauti na roba don ingantacciyar magana ba tare da bayyananniyar lakabi ba.
Taswirar Hanya
Sami tabbataccen izini don ɗaukar murya, cloning, da sake amfani.
Gwajin ingantattun masu magana daban-daban da yanayin baya.
Ƙayyade lokacin da dole ne ɗan adam ya duba ko ya amince da abubuwan da aka fitar.
Yi lakabin sauti na roba da kuma adana bayanan da aka tabbatar don yin lissafi.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Riffusion Spectrogram Diffusion quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
DiffWave Diffusion Vocoder
Tambayoyin da ake yawan yi
What is Riffusion Spectrogram Diffusion?
Riffusion hack ne mai wayo wanda ke haifar da kida ta hanyar ɗaukar sauti azaman hoto: yana daidaita ƙirar hoton Stable Diffusion don fenti spectrograms, sannan ya canza waɗannan hotunan zuwa sauti. Yana da mahimmanci saboda yana nuna kayan aikin da aka gina don matsakaici ɗaya (hotunan) na iya samar da wani (music) ba tare da kusan sabon gine-gine ba.
Wane samfurin AI na yanzu Riffusion ya daidaita don samar da kiɗa?
Riffusion mai kyau-saukar Stable Diffusion, samfurin watsa hoto, don samar da hotunan bakan da aka canza zuwa sauti.
Menene spectrogram ke wakilta akan gaturansa guda biyu?
A cikin spectrogram a kwance axis lokaci ne, axis na tsaye shine mita, kuma haske yana wakiltar ƙara.
Me yasa Riffusion yana buƙatar algorithm kamar Griffin-Lim?
spectrogram ɗin yana adana girma amma yana watsar da lokaci, don haka Griffin-Lim akai-akai yana ƙididdige lokaci don sake gina yanayin motsin motsi.
Menene iyawar watsawa ke ba Riffusion yayin haɗuwa da tsokaci biyu?
Samfuran watsa shirye-shiryen na iya yin musanya a cikin sararin samaniya, barin Riffusion morph ya ci gaba daga salon kiɗan zuwa wani.
Ta yaya aka fara ƙirƙirar Riffusion kuma aka sake shi?
Riffusion ya fara azaman aikin sha'awa ta Seth Forsgren da Hayk Martiros, waɗanda aka saki a bainar jama'a a ƙarshen 2022.