Kodeku Neural
SoundStream codec audio neuronal la buy tënk kàddu ak music ci bitrate yu woyof lool te baña soppi kalite bi.
Résumé
It matters because it beats traditional codecs like Opus at the same bitrate and powers modern generative audio models.
Plongeur bu xóot
Google moo ko dugal ci 2021, SoundStream codec neural la bu mat sëkk, ñu tabaxee ko ci ñatti mbir yuñ boole: encodeur convolutionnel buy soppi forme onde bu ñor ci benn vecteur bu dëgër, vecteur residuel buy soppi vecteur yi (RVQ) defaraat jëmmu vague bi. Dañu ko tàggat ak ñàkka tabaxaat ak discriminateur adversarial bu nuru GAN, kon génne gi dafay nuru lu naturel te baña jege ci lim. Benn ci màndarga yi gëna fës mooy 'scalable' wala tàggat-dropout quantizer: benn model mën na doxal ci bitrate yu tollu ci 3 ba 18 kbps ci jëfandikoo lu gëna bari wala lu néew ay couche quantizer ci inference, te amul benn tàggataat. Ci 3 kbps, dafa gëna am doole Opus ci 12 kbps ci test déglu, jëfandikoo kàddu, music, ak audio général ci benn model bu mëna daw ci jamono dëgg ci CPU bu xarañ.
Gis-gis xarala
Formu vague bi dafay jaar ci ay convolution yu am doole yuy wàññi bu baax, di defar benn embedding ci kadre bu nekk (lu melni 75 kadre/segond). RVQ dafay kode bépp embedding ni stack indices codebook. Bitrate mingi méngoo ak limu kantisatër yi ak bit yi ci téere kode bu nekk. Quantizer dropout dafay dagg stack RVQ bi ci diiru tàggat yaram, di forse codebooks yu njëkk yi ñu yóbbaale leeral yi gëna am solo suko defee codec bi di degrade ci anam wu woyof ci njëg yu gëna néew.
njeextalu pexe
Dugg ak yegg
Dafay gëna yombal jëfandikoo gi jaaraleko ci transkripsioŋ, nettali ak interfaasu baat.
Njëgg ak budget
Ekipu mejaa yi mën nañu yónnee audio bu leer ci anam wu gëna gaaw te seen xaalis gëna néew.
Gaawaay ak yaatuwaay
Sistem yiy jàkkarloo ak kiliyaan bi mën nañu def waxtaan ci anam wu gëna yaatu.
Ëlëgu Kodeku Neural SoundStream
SoundStream moo taxawal gaaraas bi codec yu melni EnCodec ak DAC setal ci ginaaw, ba noppi ay token yu wuute nekk nañu substrat ngir sistem yuy defar lu melni AudioLM ak MusicLM. Xaarandil ay mbokk yuy puus ngir yegg ci bitrate yu gëna ndaw, ay jeton yuñ defar ci anam wu semantik yuy nekk ñaari yoon ay dugal ci generatëru audio yu nuroo ak làkk, ak gëna dëgër ci aparey ngir woote ci saasi, aparey yiy dégg, ak streaming fu bandwidth ak latency tëju bu baax.
Doxal ci àdduna dëgg
Komprime woote baat ci ~3 kbps ci noonu mu gëna leer codec yu yàgg yi ci bitrate yu gëna kawe
Defar ay jeton audio yu wuute yuy dundal xeetu AudioLM ak MusicLM
Diffusion audio ci temps réel ci aparey mobile yu am encodage ak decodage ci CPU
Denc wala joxe music ak son ambiant ci anam wu jaar yoon ci benn model buy jëflante ak bépp xeetu ëmbiit
Risk yi ak balustrade yi
Jëfandikoo baat ci anam wu jaarul yoon ak niru ak nit dafay gëna yokk sudee nanguwul.
Jaar-jaar mën na wàññeeku ci aksan yi, dialect yi wala barab yu bari xumbaay.
Audio synthetik mën nañu ko jaawale ak wax ju dëggu sudee amul etiket bu leer.
Roadmap ngir samp gi
Wutal ndigal bu leer ngir jàpp baat bi, klone ko ak jëfandikoowaat ko.
Saytu kalite ci kàddukat yu bari ak anam yu bari ci ginaaw.
Mandargal kañ la nit wara xoolaat wala nangu ay génne.
Etiketu audio synthetik te nga denc dokimaa ci fimu bawoo ngir mëna lim.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SoundStream Neural Codec quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Kodeku audio neuronal
Laaj yi ñuy faral di laaj
What is SoundStream Neural Codec?
SoundStream codec audio neuronal la buy tënk kàddu ak music ci bitrate yu woyof lool te baña soppi kalite bi. Dafa am solo ndax dafa raw codec yu yàgg yi melni Opus ci benn bitrate bi te dafay dooleel model audio yu bees yi.
Yan ñatti mbir ñoo bokk ci arsitektir SoundStream?
SoundStream ñu ngi ko tabax ci encodeur convolutionnel, benn vecteur residuel buy xàjjale embedding yi, ak decodeur convolutionnel, ñoom ñépp ñu tàggat leen.
Ban pexe mooy may benn xeetu SoundStream mu mëna jëfandikoo ay bitrate yu bari te kenn duko tàggataat?
Ci diiru tàggat yaram, SoundStream dafay daaneel ay couche quantizer ci anam wu mujjee, suko defee nga mëna jëfandikoo niveau RVQ yu bari wala yu néew ngir jot bitrate yu wuute ci benn model.
Ci test yiñ def ci déglu AIU, SoundStream ci 3 kbps moo gëna am doole ci codec ci 12 kbps?
SoundStream ci 3 kbps rekk méngoo wala raw codec Opus bi ñuy jëfandikoo bu bari di dawal ci 12 kbps ci jàngat yiñ def ci kalite.
Ban xeetu siñaal tàggat yaram la SoundStream di jëfandikoo ngir génne son bu natureel?
Lu weesu ñàkka tabaxaat, SoundStream dafay jëfandikoo ay diskriminatër adversarial (GAN) suko defee tabaxaat yi di sone ci anam wu dëggu, duñu jege ci lim.
naka la bitrate SoundStream di méngoo ak ay kantisatëram?
Bitrate dafay méngoo ak limu couche RVQ yiy dox (taux de cadre yoon bits ci codebook bu nekk), kon yokk quantizers dafay yokk bitrate ak kalite.