Spektrogram diffusion
Riffusion hack bu am xel la buy defar music ci jëfandikoo son bi ni nataal: dafay defar modelu nataalu Stable Diffusion ngir defar ay spectrogram, ba noppi mu soppi nataal yooyu ñu nekk audio.
Résumé
It matters because it shows a tool built for one medium (images) can produce another (music) with almost no new architecture.
Plongeur bu xóot
Riffusion, Seth Forsgren ak Hayk Martiros ñoo ko genne ci 2022, mingi tàmbali nekk projet bu ñuy bëgg. Li gëna am solo mooy: spectrogram mooy nataalu 2D bi axis horizontal bi di jot, axis vertical bi di fréquence, pixel bi di leeralu kàddu gi. Bi Stable Diffusion tàmbalee defar ay nataal ci ay mbind, defarkat yi dañu ko gëna defar ci ay junni misaali spectrogram-tekst yuñ boole. Soo ko laajee 'bass jazz bu xoromu' mu dindi bruit bu bari ci spectrogram bu son boobu. Ngir defar audio buñ mëna jouer, Riffusion dafay doxal spectrogram bi jaaraleko ci algorithm Griffin-Lim buy defaraat leerali fase yi ñàkk. Ndax diffusion mën na interpoler bu baax ci diggante laaj yi, Riffusion mën na itam soppi benn stil ci beneen ci kaw clip bu wéy, di loop bu baax.
Gis-gis xarala
Riffusion dafay jëfandikoowaat pipeline diffusion bu nëbbu bi te du soppiku: U-Net dafay dindi bruit Gaussian ci nataal bu nëbbu biñ tëral ci benn bind CLIP. Benn liggéey biñ def ci domen bi mooy wane spectrogram bi (mel-scale, doole log) ak tabaxaat fasu Griffin-Lim biy soppi spectrogram magnitude biñ séentuwoon ci jëmmu vague. Faas bi dañu koy sànni ci enkodaas bi, kon xayma iteratif bu Griffin-Lim mooy gëna am solo ci màndarga 'ndox' yi.
njeextalu pexe
Dugg ak yegg
Dafay gëna yombal jëfandikoo gi jaaraleko ci transkripsioŋ, nettali ak interfaasu baat.
Njëgg ak budget
Ekipu mejaa yi mën nañu yónnee audio bu leer ci anam wu gëna gaaw te seen xaalis gëna néew.
Gaawaay ak yaatuwaay
Sistem yiy jàkkarloo ak kiliyaan bi mën nañu def waxtaan ci anam wu gëna yaatu.
Ëlëgu spectrogram diffusion
Riffusion dafa firndeel ni pont spectrogram-ni-nataal dafay dox, te xalaat boobu mingi wéy di dundu ci biir sistem audio yu gëna mag, liggéeyukaay bi Riffusion nekk. Xaarandil jumtukaayi ëlëg ngir wecci Griffin-Lim bu ñàkk ak vocoder neuronal yu jàng ngir fase bu gëna set, ak boole diffusion spectrogram ak codec audio yu nëbbu. Njàngale mu gëna yaatu, mooy xeetu nataal yi mën nañu leen yóbbu ci anam yu bees, di wéy di am njeexital ci ni gëstukat yi di bootstrap ay generatëru audio ak wideo ci yaxu ndigg yiñ tàggat bu njëkk.
Doxal ci àdduna dëgg
Defar ay piyees yu gàtt ci ginaaw ngir jeu video indie ci ab bind bu melni 'topp synthwave bu metti'
Morphing bu jaar yoon ci diggante ñaari xeeti music, lu melni jaxase 'këru toropikaal' ak 'lo-fi hip hop' ci benn clip
Defar ay lalu music ambiant te doo fay royalty ngir wideo YouTube ak podcast te doo fay lisence
Prototyping xalaat melodic wala ritm yu musicien bi enregistrewaat ci anam wu jaar yoon ci benn station audio numérique
Risk yi ak balustrade yi
Jëfandikoo baat ci anam wu jaarul yoon ak niru ak nit dafay gëna yokk sudee nanguwul.
Jaar-jaar mën na wàññeeku ci aksan yi, dialect yi wala barab yu bari xumbaay.
Audio synthetik mën nañu ko jaawale ak wax ju dëggu sudee amul etiket bu leer.
Roadmap ngir samp gi
Wutal ndigal bu leer ngir jàpp baat bi, klone ko ak jëfandikoowaat ko.
Saytu kalite ci kàddukat yu bari ak anam yu bari ci ginaaw.
Mandargal kañ la nit wara xoolaat wala nangu ay génne.
Etiketu audio synthetik te nga denc dokimaa ci fimu bawoo ngir mëna lim.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Riffusion Spectrogram Diffusion quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Vocodeur de diffusion d'onde diffusion
Laaj yi ñuy faral di laaj
What is Riffusion Spectrogram Diffusion?
Riffusion hack bu am xel la buy defar music ci jëfandikoo son bi ni nataal: dafay defar modelu nataalu Stable Diffusion ngir defar ay spectrogram, ba noppi mu soppi nataal yooyu ñu nekk audio. Dafa am solo ndax dafay wane ni jumtukaay buñ tabax ngir benn mejaa (nataal) mën na defar beneen (musik) te daanaka amul benn architecture bu bees.
Ban xeetu IA la Riffusion defar ngir defar music?
Riffusion dafa defar diffusion bu dëgër, muy xeetu diffusion nataal, ngir génne nataali spectrogram yuñ soppi def ko audio.
Lan la spectrogram di màndargaal ci ñaari axe yi?
Ci spectrogram bi, axe horizontal bi mooy waxtu, axe vertikaal bi mooy fréquence bi, leeraay bi mooy kàddu yu kawe yi.
Lu tax Riffusion soxla algorithm bu melni Griffin-Lim?
Spectrogram bi dafay denc magnitude waaye dafay sànni faas bi, moo tax Griffin-Lim dafay xayma faas bi ngir tabaxaat forme vague buñ mëna jouer.
Ban kàttan la diffusion mëna jox Riffusion sooy jaxase ñaari laaj?
Modèle diffusion yi mën nañu interpoler ci espace bu nëbbu, bàyyi Riffusion mu wéy di soppeeku joge ci benn stil musical dem ci beneen.
naka lañu njëkka sos Riffusion ba noppi génne ko?
Riffusion mingi tàmbali nekk projet bu Seth Forsgren ak Hayk Martiros def, ñu génne ko ci 2022.