SoundStorm defar audio paralel
SoundStorm Google xeetu defar audio la buy defar kàddu ak son ci paralel, du benn token benn yoon, loolu mooy tax synthese audio bu baax gëna gaaw.
Résumé
It matters because it cuts generation latency for long clips from minutes to seconds without sacrificing fidelity.
Plongeur bu xóot
SoundStorm, bi Google dugal ci 2023, dafay defar ay token akustik yu wuute ci kodek bu tuddu SoundStream. Modèle yu njëkk yu melni AudioLM dañu defar token yooyu ci anam wu autoregressivement, di wax luy waaja am ci token bu nekk ci seeni toppalante, te loolu dafa yeex ci audio yu gudd. Lu moy loolu SoundStorm dafay jëfandikoo ab jego bu dul dellu ginaaw, bu sukkandiko ci mask buñu leble ci xeetu defar nataal yu melni MaskGIT. Dafay tàmbali ci ay token yu maskeer, ba noppi di leen feesal ci ay jéego yu néew ci dekodaas, di wax luy waaja am ci ay token yu bari benn yoon ci paralel. Dañu ko tëral ci ay token semantik (ci xeetu AudioLM wala SPEAR-TTS), mën na defar 30 segond ci waxtaan bu natureel ci lu tollu ci genn-wàllu segond ci kaw TPU, lu tollu ci 100 yoon lu gëna gaaw ci liiñ yu autoregressive yi, fekk ñu méngoo ak seen kalite ak seen njub.
Gis-gis xarala
SoundStorm dafay wane xeetu vecteur yu des (RVQ) yu bawoo ci SoundStream. Bu ñuy tàggat, dañuy maske ay token yu bari te model bi jàng ni ñu leen di seetlu. Ci inference dafay def decodage parallèle bu sukkandiko ci wóolu: ci iteration bu nekk dafay wax luy waaja am ci token yiñ maske yépp, denc ñi gëna wóolu, ba noppi maskewaat ñeneen ñi. Dafay njëkka dekode niveau RVQ yu dëgër, ginaaw ga yu gëna rafet, dem ba ci audio bu mat ci jéego yu néew lool ci generation token-by-token.
njeextalu pexe
Dugg ak yegg
Dafay gëna yombal jëfandikoo gi jaaraleko ci transkripsioŋ, nettali ak interfaasu baat.
Njëgg ak budget
Ekipu mejaa yi mën nañu yónnee audio bu leer ci anam wu gëna gaaw te seen xaalis gëna néew.
Gaawaay ak yaatuwaay
Sistem yiy jàkkarloo ak kiliyaan bi mën nañu def waxtaan ci anam wu gëna yaatu.
Ëlëgu SoundStorm defar audio paralel
Dekodage bu sukkandiko ci masku paralel mingi nekk jumtukaay buñ miin ngir audio bu gaaw te ñu mëna saytu. Xaarandil ni dina mëna dooleel agent yiy waxtaan ci jamono dëgg, synthese baat ci saasi, ak podcast wala audiobook yu gudd, fu latency def model autoregressif yu amul njariñ. Soo ko boole ak kondisioŋu semantik bu gëna am doole ak watermarking dina gëna rafetal waxtaan wi ak mëna toppu. Benn xalaat biy setal iteratif bi mën na boole ak jegewaale diffusion yi, di dindi dig wi dox diggante codec-token ak generatëru audio yuy wéy.
Doxal ci àdduna dëgg
Defar ay waxtaan yu 30 sikon ngir assistant baat yu IA ci diir bu néew benn sikon
Dajale waxtaan yu bari ak kàdduy waxkat yuy méngoo ngir prototyping
Dooleel bind-ci-kaddu bu yeex ci ab ndawu weccoo xalaat fu xeetu otoregresif yi di yeex
Defar ay audio yu gudd te nettali ci lu gaaw ci feesal ay token akustik ci paralel
Risk yi ak balustrade yi
Jëfandikoo baat ci anam wu jaarul yoon ak niru ak nit dafay gëna yokk sudee nanguwul.
Jaar-jaar mën na wàññeeku ci aksan yi, dialect yi wala barab yu bari xumbaay.
Audio synthetik mën nañu ko jaawale ak wax ju dëggu sudee amul etiket bu leer.
Roadmap ngir samp gi
Wutal ndigal bu leer ngir jàpp baat bi, klone ko ak jëfandikoowaat ko.
Saytu kalite ci kàddukat yu bari ak anam yu bari ci ginaaw.
Mandargal kañ la nit wara xoolaat wala nangu ay génne.
Etiketu audio synthetik te nga denc dokimaa ci fimu bawoo ngir mëna lim.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SoundStorm Parallel Audio Generation quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Diffusion audio bu nëbbu bu dëgër
Laaj yi ñuy faral di laaj
What is SoundStorm Parallel Audio Generation?
SoundStorm Google xeetu defar audio la buy defar kàddu ak son ci paralel, du benn token benn yoon, loolu mooy tax synthese audio bu baax gëna gaaw. Dafa am solo ndax dafay dagg latency generation ci clips yu gudd ci simili dem ba ci seconde te du yàq fidélité.
Lan mooy jéego yu am solo yi tax SoundStorm gëna gaaw AudioLM?
SoundStorm dafay wecci otoregresif bu yeex, defar token-ci-token ak dekodage paralel bu dul otoregressif, di wax luy waaja am ci token yu bari benn yoon.
Ban pexe ci defar nataal la SoundStorm mëna jëfandikoo?
SoundStorm leble pexem dekodaas bu sukkandiko ci wóolu sa bopp, mask-ak-recharge bi MaskGIT siiwal ci synthese nataal.
Ban xeetu fësal la SoundStorm di defar?
SoundStorm dafay wax luy waaja am ci ay token akustik yu codec SoundStream defar, ñu dekode ko ci forme onde.
Lu tollu ci ban diir la SoundStorm di jël ngir defar 30 seconde ci audio ci TPU?
SoundStorm mën na defar 30 seconde audio ci lu tollu ci 0.5 seconde, lu tollu ci 100x gëna gaaw ligne de base autorégressif.
Nan lay def ba SoundStorm xam ban token lañu wara tëye ci dekodage bi?
Ci iteration bu nekk dafay tëye ay prediction token yu gëna wóolu ba noppi maskewaat yi wóorul ci paas bi ci topp.