Riffusion Spectrogram Diffusion
Riffusion ni hack yubwenge itanga umuziki ufata amajwi nkifoto: ihuza neza icyitegererezo cyerekana ishusho ya Stable Diffusion kugirango ishushanye amashusho, hanyuma ihindure ayo mashusho mumajwi.
Incamake
It matters because it shows a tool built for one medium (images) can produce another (music) with almost no new architecture.
Kwibira cyane
Riffusion, yasohowe mu mpera za 2022 na Seth Forsgren na Hayk Martiros, yatangiye nk'umushinga wo kwishimisha. Amayeri yibanze: spekrogramu ni ishusho ya 2D aho umurongo utambitse ni igihe, umurongo uhagaritse ni inshuro, na pigiseli yaka ni ijwi. Kubera ko Diffusion Stable isanzwe itanga amashusho uhereye kumyandiko isaba, abayiremye bayitunganije neza kubihumbi n'ibihumbi byombi byerekanwe na ecran ya progaramu. Bimenyeshe hamwe na 'funky jazz bass' kandi byerekana urusaku rudasanzwe muri ecran ya majwi. Kugirango ukore amajwi acuranga, Riffusion ikoresha spekrogramu ikoresheje algorithm ya Griffin-Lim yubaka amakuru yabuze. Kuberako ikwirakwizwa rishobora guhuza neza hagati yibisobanuro, Riffusion irashobora kandi guhinduranya uburyo bumwe mubindi hejuru ya clip ikomeza, ikazunguruka.
Ubushishozi
Riffusion yongeye gukoresha umuyoboro wogukwirakwiza udahindutse: U-Net ikuraho byimazeyo urusaku rwa Gaussiya kumashusho yihishe iteganijwe kumyandiko ya CLIP. Igikorwa cyonyine cyihariye ni icyerekezo cyerekana (mel-scale, log power) hamwe na Griffin-Lim icyiciro cyo kwiyubaka gihindura ubunini bwahanuwe busubira inyuma. Icyiciro cyajugunywe mugihe cya kodegisi, bityo igereranya rya Griffin-Lim nisoko nyamukuru yibintu biranga 'amazi'.
Ingaruka z'Ingamba
Kugera no kugera
Itezimbere kugerwaho binyuze mu kwandukura, kuvuga, no guhuza amajwi.
Igiciro na bije
Amatsinda yibitangazamakuru arashobora kohereza amajwi yihuse hamwe na bije nto.
Umuvuduko n'igipimo
Sisitemu ireba abakiriya irashobora gutunganya imikoranire ivugwa murwego runini.
Kazoza ka Riffusion Spectrogram Diffusion
Riffusion yerekanye spekrogramu-nk-ishusho ikiraro gikora, kandi icyo gitekerezo ubu kiba imbere muri sisitemu nini zamajwi hanyuma isosiyete Riffusion iba. Tegereza ibikoresho bizaza kugirango usimbuze Griffin-Lim yatakaye hamwe na majwi yize yijwi ryicyiciro cyogusukura, no guhuza ikwirakwizwa rya spekrogramu hamwe na codecs yihishe. Isomo ryagutse, iyo moderi yerekana amashusho irashobora kwerekezwa muburyo bushya, ikomeje guhindura uburyo abashakashatsi batangiza amajwi n'amashusho bituruka kumugongo wateganijwe mbere.
Gushyira mu bikorwa Isi
Gukora ibice bigufi byerekana inzira yimikino ya indie uhereye kumyandiko yihuta nka 'tense synthwave chase'
Morphing neza hagati yuburyo bubiri bwumuziki, kuvanga 'inzu yubushyuhe' muri 'lo-fi hip hop' hejuru ya clip imwe
Gukora ibitanda byumuziki udafite ubwami bwibitanda bya videwo na podcaste utishyuye
Prototyping melodic cyangwa rhythmic ibitekerezo umucuranzi noneho yongera kwandika neza mubikorwa byamajwi ya digitale
Ingaruka & Kurinda
Gukoresha nabi amajwi no kwigira ibyago byiyongera mugihe uruhushya rubuze.
Ukuri kurashobora kugabanuka hejuru yimvugo, imvugo, cyangwa urusaku rwibidukikije.
Amajwi yubukorikori arashobora kwibeshya kumvugo yukuri nta kirango gisobanutse.
Igishushanyo mbonera
Shaka uruhushya rusobanutse rwo gufata amajwi, gukoroniza, no gukoresha.
Ikizamini cyiza mubiganiro bitandukanye hamwe nuburyo bwimbere.
Sobanura igihe umuntu agomba gusuzuma cyangwa kwemeza ibisubizo.
Andika amajwi yubukorikori kandi ugumane inyandiko zerekana kubazwa.
Komeza Ubushakashatsi
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Riffusion Spectrogram Diffusion quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ubuyobozi bukurikira
Vocoder ya DiffWave
Ibibazo bikunze kubazwa
What is Riffusion Spectrogram Diffusion?
Riffusion ni hack yubwenge itanga umuziki ufata amajwi nkifoto: ihuza neza icyitegererezo cyerekana ishusho ya Stable Diffusion kugirango ishushanye amashusho, hanyuma ihindure ayo mashusho mumajwi. Nibyingenzi kuko yerekana igikoresho cyubatswe muburyo bumwe (amashusho) gishobora kubyara ikindi (umuziki) hafi yububiko bushya.
Ni ubuhe bwoko bwa AI buriho Riffusion yakoresheje neza kugirango atange umuziki?
Riffusion itunganijwe neza Stable Diffusion, moderi yo gukwirakwiza amashusho, kugirango itange amashusho ya spekrogramu hanyuma ihindurwe amajwi.
Niki spekrogramu igereranya kumashoka yayo abiri?
Muri spekrogramu itambitse itambitse ni igihe, umurongo uhagaritse ni inshuro, kandi umucyo ugereranya amajwi.
Kuki Riffusion ikeneye algorithm nka Griffin-Lim?
Spekrogramu ibika ubunini ariko ikajugunya icyiciro, Griffin-Lim rero igereranya icyiciro cyo kongera kubaka imiyoboro ikinirwa.
Ni ubuhe bushobozi gukwirakwiza Riffusion mugihe uhuza ibyifuzo bibiri?
Moderi ya Diffusion irashobora guhuza umwanya wihishe, ikareka Riffusion morph ikomeza kuva muburyo bwa muzika ikajya mubindi.
Nigute Riffusion yaremewe kandi ikarekurwa?
Riffusion yatangiye ari umushinga wo kwishimisha na Seth Forsgren na Hayk Martiros, washyizwe ahagaragara kumugaragaro mu mpera za 2022.