I-SoundStream Neural Codec
SoundStream is Google's end-to-end neural audio codec that compresses speech and music to extremely low bitrates while preserving quality.
Uhlolojikelele
It matters because it beats traditional codecs like Opus at the same bitrate and powers modern generative audio models.
I-Deep Dive
Yethulwe yi-Google ngo-2021, i-SoundStream iyikhodekhi ye-neural ephelele eyakhiwe ngezingcezu ezintathu eziqeqeshwe ndawonye: isifaki khodi esiguqulayo esishintsha i-waveform eluhlaza ibe ukulandelana okuhlangene kwama-vector, i-residual vector quantizer (RVQ) ehlukanisa lawo ma-vector aguqulayo futhi akha kabusha i-wavector. Iqeqeshwe ngakho kokubili ukulahlekelwa kokwakha kabusha kanye nokucwasa okuphikisayo kwesitayela se-GAN, ngakho okukhiphayo kuzwakala kungokwemvelo kunokuba nje kuvaleke ngokwezinombolo. Isici esivelele 'siyasikala' noma ukuqeqeshwa kokuyeka i-quantizer-dropout: imodeli eyodwa ingasebenza kuwo wonke ama-bitrate ukusuka cishe ku-3 ukuya ku-18 kbps ngokusebenzisa nje izendlalelo zequantizer eziningi noma ezimbalwa ekuqondeni, ngaphandle kokuqeqeshwa kabusha. Ngo-3 kbps kubikwa ukuthi idlula i-Opus ku-12 kbps ezivivinyweni zokulalela, ukuphatha inkulumo, umculo, nomsindo ojwayelekile kumodeli eyodwa engasebenza ngesikhathi sangempela ku-smartphone CPU.
I-Technical Insight
I-waveform idlula kuma-convolutions anemigqa ehlisa isampula kakhulu, ikhiqiza ukushumeka okukodwa kuhlaka ngalunye (isb. amafreyimu angu-75/isekhondi). I-RVQ bese ibhala ngekhodi ukushumeka ngakunye njengenqwaba yezinkomba ze-codebook. I-Bitrate ilingana nezinga lozimele izikhathi lapho inani lezikhawuli zezikhathi ezisebenzayo amabhithi ebhukwana ngalinye lekhodi. Ukuyeka ukuphuma kwe-Quantizer kunciphisa ngokungahleliwe isitaki se-RVQ phakathi nokuqeqeshwa, kuphoqe ama-codebook angaphambili ukuthi aphathe ulwazi olubaluleke kakhulu ukuze i-codec yonakale kahle ngamanani aphansi.
I-Strategic Impact
Finyelela futhi ufinyelele
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi.
Izindleko kanye nesabelomali
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane.
Isivinini nesikali
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu.
Ikusasa le-SoundStream Neural Codec
I-SoundStream isungule isifanekiso okwathi kamuva ama-codec afana ne-EnCodec ne-DAC yacwengisiswa, futhi amathokheni ayo ahlukene aba isisekelo sezinhlelo ezikhiqizayo ezifana ne-AudioLM ne-MusicLM. Lindela inzalo ephushela kuma-bitrate aphansi ngisho nangaphansi, amathokheni amiswe ngokwezibalo aphindwe kabili njengokufakwa kubakhiqizi bomsindo besitayela solimi, kanye nokusetshenziswa okuqinile kudivayisi kumakholi abukhoma, izinsiza zokuzwa, nokusakaza lapho umkhawulokudonsa nokubambezeleka kubambezeleke kakhulu.
Ukuqaliswa Komhlaba Wangempela
Ukucindezela izingcingo zezwi zibe ~3 kbps kuyilapho kuzwakala kucace kunamakhodekhi efa kuma-bitrate aphezulu
Ikhiqiza amathokheni omsindo ahlukene afunza Google amamodeli akhiqizayo we-AudioLM kanye ne-MusicLM
Ukusakaza-bukhoma komsindo womkhawulokudonsa ophansi wesikhathi sangempela kumadivayisi eselula anombhalo wekhodi oku-CPU nokukhetha
Ukugcina noma ukudlulisa umculo nomsindo we-ambient ngendlela efanele ngemodeli eyodwa ephatha zonke izinhlobo zokuqukethwe
Izingozi & Guardrails
Ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyanda uma imvume ingekho.
Ukunemba kungase kwehle kuzo zonke izinhlobo zokuphimisela, izilimi zesigodi, noma izindawo ezinomsindo.
Umsindo wokwenziwa ungenziwa iphutha njengenkulumo eyiqiniso ngaphandle kokulebula okucacile.
Ukuqalisa Umhlahlandlela
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SoundStream Neural Codec quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
I-Neural Audio Codecs
Imibuzo evame ukubuzwa
What is SoundStream Neural Codec?
SoundStream is Google's end-to-end neural audio codec that compresses speech and music to extremely low bitrates while preserving quality. It matters because it beats traditional codecs like Opus at the same bitrate and powers modern generative audio models.
Yiziphi izingxenye ezintathu ezakha ukwakheka kwe-SoundStream?
I-SoundStream yakhiwe ngesishumeki se-convolutional, i-quantizer ye-vector eyinsalela ehlukanisa ukushumeka, kanye ne-convolutional decoder, konke okuqeqeshiwe kuya ekugcineni.
Iyiphi inqubo evumela imodeli eyodwa ye-SoundStream ukuthi isebenze ama-bitrate amaningi ahlukene ngaphandle kokuqeqeshwa kabusha?
Ngesikhathi sokuqeqeshwa, i-SoundStream yehlisa ngokungahleliwe izendlalelo ze-quantizer ukuze ngokuqonda kwakho usebenzise amazinga e-RVQ amaningi noma ambalwa ukuze ushaye ama-bitrate ahlukene kumodeli eyodwa.
Ezivivinyweni zokulalela ze-Google, i-SoundStream engu-3 kbps kubikwe ukuthi idlula iyiphi i-codec ngo-12 kbps?
I-SoundStream ngo-3 kbps kuphela ifaniswe noma yehlula i-Opus codec esetshenziswa kabanzi esebenza ngo-12 kbps ekuhlolweni kwekhwalithi okuzimele.
Iluphi uhlobo lwesiginali yokuqeqeshwa eyengeziwe esetshenziswa yi-SoundStream ukwenza okukhiphayo kuzwakale kwendalo?
Ngaphandle kokulahlekelwa kokwakha kabusha, i-SoundStream isebenzisa ababandlululi be-adversarial (GAN) ukuze ukwakha kabusha kuzwakale kungokoqobo kunokuba nje kuvaleke ngokwezinombolo.
I-bitrate ye-SoundStream ihlobana kanjani nezilinganisi zayo?
Izilinganiso ze-Bitrate ezinenani lezendlalelo ze-RVQ ezisebenzayo (izikhathi zesilinganiso sozimele amabhithi ibhukwana ngalinye), ngakho ukwengeza ama-quantizer kuphakamisa i-bitrate nekhwalithi.