I-Riffusion Spectrogram Diffusion
I-Riffusion iwukugebenga okuhlakaniphile okukhiqiza umculo ngokuphatha umsindo njengesithombe: ishuna kahle imodeli yesithombe eStable Diffusion ukuze ipende ama-spectrogram, bese iguqula lezo zithombe zibuyele emsindweni.
Uhlolojikelele
It matters because it shows a tool built for one medium (images) can produce another (music) with almost no new architecture.
I-Deep Dive
I-Riffusion, ekhishwe ngasekupheleni kuka-2022 nguSeth Forsgren noHayk Martiros, yaqala njengephrojekthi yokuzilibazisa. Iqhinga eliwumgogodla: i-spectrogram yisithombe se-2D lapho i-eksisi evundlile kuyisikhathi, i-eksisi eqondile iyimvamisa, nokukhanya kwe-pixel kungumsindo. Njengoba i-Stable Diffusion isivele ikhiqiza izithombe kusuka ekwazisweni kombhalo, abadali bayilungise kahle ezinkulungwaneni zezibonelo zombhalo we-spectrogram obhanqiwe. Itshele 'nge-funky jazz bass' futhi ikhiphe umsindo ongahleliwe ibe i-spectrogram yalowo msindo. Ukuze yenze umsindo odlalekayo, i-Riffusion isebenzisa i-spectrogram nge-algorithm ye-Griffin-Lim eyakha kabusha ulwazi lwesigaba olungekho. Ngenxa yokuthi ukusabalalisa kungahumusheka kahle phakathi kokwaziswa, i-Riffusion ingaphinda iguqule isitayela esisodwa sibe kwesinye phezu kwesiqeshana esiqhubekayo, ivuleke ngaphandle komthungo.
I-Technical Insight
I-Riffusion iphinda isebenzise ipayipi lokusabalalisa elicashile lingashintshiwe: i-U-Net isusa ngokuphindaphindiwe umsindo we-Gaussian esithombeni esicashile esifakwe ekushumekeni kombhalo we-CLIP. Okuwukuphela komsebenzi oqondene nesizinda ukumelela kwe-spectrogram (isikali se-mel, amandla elogi) kanye nokwakhiwa kabusha kwesigaba se-Griffin-Lim esiphendulela i-spectrogram yobukhulu obubikezelwe ibe i-waveform. Isigaba siyalahlwa ngesikhathi sombhalo wekhodi, ngakho-ke ukulinganisa okuphindaphindayo kuka-Griffin-Lim kuwumthombo oyinhloko wesici sezinto zobuciko 'ezinamanzi'.
I-Strategic Impact
Finyelela futhi ufinyelele
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi.
Izindleko kanye nesabelomali
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane.
Isivinini nesikali
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu.
Ikusasa Le-Riffusion Spectrogram Diffusion
I-Riffusion ifakazele ukuthi ibhuloho le-spectrogram-as-image liyasebenza, futhi lowo mbono manje uhlala ngaphakathi kwamasistimu omsindo amakhulu futhi inkampani i-Riffusion yaba. Lindela amathuluzi esikhathi esizayo wokushintsha u-Griffin-Lim olahlekile ufake amavokhoda e-neural afundiwe esigabeni esihlanzekile, kanye nokuhlanganisa ukuhlukaniswa kwe-spectrogram namakhodekhi omsindo acashile. Isifundo esibanzi, sokuthi amamodeli ezithombe angaqondiswa kabusha ezindleleni ezintsha, siyaqhubeka nokuba nomthelela ekutheni abacwaningi baqala kanjani amajeneretha omsindo nevidiyo avela kumgogodla oqeqeshwe kusengaphambili.
Ukuqaliswa Komhlaba Wangempela
Ikhiqiza amathrekhi angemuva amafushane alandelanayo emidlalo yevidiyo ye-indie kusukela kumlayezo wombhalo ofana ne-'tense synthwave chase'
I-Morphing kahle phakathi kwezitayela zomculo ezimbili, isb. ukuhlanganisa 'indlu eshisayo' ibe 'lo-fi hip hop' esiqeshaneni esisodwa
Ukukhiqiza imibhede yomculo we-ambient yamavidiyo e-YouTube nama-podcast ngaphandle kwezindleko zamalayisense
I-Prototyping melodic imibono noma enesigqi umculi bese ayiqopha kabusha kahle endaweni yokusebenza yomsindo wedijithali
Izingozi & Guardrails
Ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyanda uma imvume ingekho.
Ukunemba kungase kwehle kuzo zonke izinhlobo zokuphimisela, izilimi zesigodi, noma izindawo ezinomsindo.
Umsindo wokwenziwa ungenziwa iphutha njengenkulumo eyiqiniso ngaphandle kokulebula okucacile.
Ukuqalisa Umhlahlandlela
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Riffusion Spectrogram Diffusion quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
I-DiffWave Diffusion Vocoder
Imibuzo evame ukubuzwa
What is Riffusion Spectrogram Diffusion?
I-Riffusion iwukugebenga okuhlakaniphile okukhiqiza umculo ngokuphatha umsindo njengesithombe: ishuna kahle imodeli yesithombe eStable Diffusion ukuze ipende ama-spectrogram, bese iguqula lezo zithombe zibuyele emsindweni. Kubalulekile ngoba kukhombisa ukuthi ithuluzi elakhelwe imidiya eyodwa (izithombe) lingakhiqiza elinye (umculo) cishe elingenazo izakhiwo ezintsha.
Iyiphi imodeli ye-AI ekhona u-Riffusion ayicule kahle ukuze akhiqize umculo?
I-Riffusion ecushwe kahle i-Stable Diffusion, imodeli yokusabalalisa isithombe, ukuze kukhiqizwe izithombe ze-spectrogram ezibuye ziguqulelwe kumsindo.
Imelelani i-spectrogram ezimbazweni zayo ezimbili?
Ku-spectrogram i-eksisi evundlile yisikhathi, i-eksisi eqondile imvamisa, futhi ukukhanya kumelela umsindo.
Kungani i-Riffusion idinga i-algorithm efana ne-Griffin-Lim?
I-spectrogram igcina ubukhulu kodwa ilahla isigaba, ngakho u-Griffin-Lim ulinganisela ngokuphindaphindiwe isigaba ukuze akhe kabusha i-waveform edlalekayo.
Iliphi ikhono ukusabalalisa okulinikeza i-Riffusion lapho kuhlanganiswa imiyalo emibili?
Amamodeli okusabalalisa angashintshana endaweni ecashile, avumele i-Riffusion morph ngokuqhubekayo isuka kusitayela somculo esisodwa iye kwesinye.
I-Riffusion yadalwa futhi yakhululwa kanjani ekuqaleni?
I-Riffusion yaqala njengephrojekthi yokuzilibazisa kaSeth Forsgren noHayk Martiros, ekhishwe esidlangalaleni ngasekupheleni kuka-2022.