SoundStream Neural Codec
SoundStream bụ Google's end-to-end neural audio codec nke na-akpakọ okwu na egwu na bitrate dị oke ala ma na-echekwa ịdịmma.
Nchịkọta
It matters because it beats traditional codecs like Opus at the same bitrate and powers modern generative audio models.
Ime miri emi
Nke __AIU_PROTECTED_11_ webatara na 2021, SoundStream bụ codec neural zuru oke nke e wuru site na iberibe atọ a zụrụ ọnụ: ihe ngbanwe mgbanwe nke na-atụgharị raw waveform ka ọ bụrụ usoro kọmpat nke vectors, residual vector quantizer (RVQ) nke na-agbagha vectors ndị ahụ, yana ihe na-agbanwe agbanwe. A zụrụ ya site na mfu nrụzigharị yana onye ịkpa oke n'ụdị GAN, yabụ mmepụta na-ada ụda eke kama ịbụ nso nsoro ọnụ. Ngosipụta pụtara bụ 'scalable' ma ọ bụ ọzụzụ quantizer-dropout: otu ụdị nwere ike ịrụ ọrụ n'ofe bitrates site na ihe dị ka 3 ruo 18 kbps naanị site na iji ọkwa quantizer karịa ma ọ bụ dị nta na ntinye, na-enweghị nlọghachi azụ. Na 3 kbps, a na-akọ na ọ karịrị Opus na 12 kbps n'ime ule ige ntị, ijikwa okwu, egwu, na ọdịyo izugbe n'otu ụdị nke nwere ike ịgba ọsọ ozugbo na CPU smartphone.
Nghọta nka nka
Ụdị ebili mmiri ahụ na-agafe mgbanwe ndị gbawara agbawa na-agbadata nke ukwuu, na-ewepụta otu ntinye n'otu etiti (dịka freelụ 75/sekọnd). RVQ wee tinye koodu ntinye ọ bụla dị ka njupụta nke indices codebook. Bitrate ha nhata nha nha n'etiti ọnụọgụ nke oge quantizer na-arụ ọrụ n'otu akwụkwọ koodu. Quantizer dropout randomly truncates the RVQ stack during training, forcing earlier codebooks to carry the most important information so the codec degrades gracefully at lower rates.
Mmetụta atụmatụ
Nweta na iru
Ọ na-eme ka nnweta ya dịkwuo mma site na ndegharị, ịkọ akụkọ, na ntụgharị olu.
Ọnụ ego na mmefu ego
Ndị otu mgbasa ozi nwere ike ibubata ọdịyo a na-egbu maramara ngwa ngwa site na iji obere mmefu ego.
Ọsọ na ọnụ ọgụgụ
Sistemụ na-eche ihu ndị ahịa nwere ike hazie mkparịta ụka n'ọtụtụ buru ibu.
Ọdịnihu nke Codec Neural SoundStream
SoundStream guzobe ndebiri nke mechara codecs dị ka EnCodec na DAC nụchara anụcha, na akara ngosi ya nwere ezi uche ghọrọ mkpụrụ maka sistemụ nrụpụta dị ka AudioLM na MusicLM. Na-atụ anya ka ụmụ na-aga n'ihu na ọbụna obere bitrates, akara usoro ahaziri ahazi nke okpukpu abụọ dị ka ntinye na ndị na-emepụta ọdịyo ụdị asụsụ, yana ntinye ngwa ngwa maka oku ndụ, ihe enyemaka ịnụ ihe, na nkwanye ugwu ebe bandwit na latency na-akpachi anya.
Mmejuputa n'ezie n'ụwa
Na-akpakọ oku olu na ~ 3 kbps ka ọ na-ada nke ọma karịa codecs nketa na bitrates dị elu
Na-emepụta ihe ngosi ọdịyo pụrụ iche na-enye Google's AudioLM na ụdị mmepụta MusicLM
Mgbasa ụda ọdịyo dị obere obere oge na ngwaọrụ mkpanaka nwere ntinye na ngbanwe on-CPU
Na-echekwa ma ọ bụ na-ebusa egwu na ụda ihe nke ọma n'otu ụdị nke na-ejikwa ụdị ọdịnaya niile
Ihe ize ndụ & okporo ụzọ nche
Iji olu eme ihe na ihe egwu mpụta ga-abawanye mgbe nkwenye na-efu.
Izi ezi nwere ike ịdaba n'ofe ụda olu, olumba ma ọ bụ gburugburu mkpọtụ.
Enwere ike imehie ọdịyo sịntetik dị ka ezigbo okwu na-enweghị akara doro anya.
Map mmejuputa
Nweta nkwenye doro anya maka ijide olu, imechi, na ijigharị.
Nwale ogo n'ofe ndị na-ekwu okwu dị iche iche yana ọnọdụ ndabere.
Kọwaa mgbe mmadụ ga-enyocha ma ọ bụ kwado nsonye.
Deba aha ọdịyo sịntetik ma debe ndekọ ihe ndekọ maka ịza ajụjụ.
Nọgide na-eme nchọpụta
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SoundStream Neural Codec quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Ntuziaka na-esote
Codec ọdịyo akwara
Ajụjụ a na-ajụkarị
What is SoundStream Neural Codec?
SoundStream is Google's end-to-end neural audio codec that compresses speech and music to extremely low bitrates while preserving quality. Ọ dị mkpa n'ihi na ọ na-akụ codecs ọdịnala dịka Opus n'otu bitrate ma na-akwado ụdị ọdịyo ọgbara ọhụrụ.
Kedu ihe atọ mejupụtara ihe owuwu SoundStream?
Ewubere SoundStream site na ngbanwe convolutional, vector quantizer residous vector quantizer nke na-ewepụ ihe mgbanye, yana ihe ngbanwe mgbanwe, ngwụcha ruo ngwụcha a zụrụ azụ.
Kedu usoro na-ahapụ otu ụdị SoundStream ijere ọtụtụ bitrates dị iche iche na-enweghị ọzụzụ ọzọ?
N'oge ọzụzụ, SoundStream na-adaba ọkwa quantizer na-enweghị usoro nke mere na n'uche ị nwere ike iji ọkwa RVQ karịa ma ọ bụ ole na ole iji kụọ bitrates dị iche iche site na otu ụdị.
N'ime ule nge ntị nke Google, a kọrọ na SoundStream na 3 kbps ga-aka nke ọma nke codec na 12 kbps?
SoundStream na naanị 3 kbps dakọtara ma ọ bụ tie Opus codec a na-ejikarị na-agba ọsọ na 12 kbps na nleba anya nke onwe.
Kedu ụdị mgbama ọzụzụ agbakwunyere ka SoundStream na-eji mee ka mmepụta dị mma?
Ewezuga mfu nrụgharị, SoundStream na-eji ndị mmegide (GAN) ịkpa oke ka nrụgharị ahụ na-ada n'ụzọ ezi uche dị na ya kama ịbụ naanị nso nso.
Kedu ka bitrate SoundStream siri metụta ọnụọgụgụ ya?
Ọnụ ọgụgụ nke ọnụọgụ ọnụọgụ ọnụọgụ nke ọkwa RVQ na-arụ ọrụ (obere oge ọnụego oge n'otu akwụkwọ koodu), yabụ ịgbakwunye quantizer na-ebuli bitrate na ịdị mma.