SoundStream Neural Codec
SoundStream shine Google's na ƙarshen-zuwa-ƙarshen audio codec na jijiya wanda ke matse magana da kiɗa zuwa ƙananan bitrates yayin kiyaye inganci.
Dubawa
It matters because it beats traditional codecs like Opus at the same bitrate and powers modern generative audio models.
Zurfafa nutsewa
An gabatar da shi ta Google a cikin 2021, SoundStream cikakken tsarin neural codec ne wanda aka gina shi daga guntu guda uku waɗanda aka horar dasu tare: maɓalli mai jujjuyawar juzu'i wanda ke juyar da ɗanyen igiyar ruwa zuwa ƙaramin jeri na vectors, ragowar vector quantizer (RVQ) wanda ke ɓata waɗancan vectors, kuma mai jujjuya juyi. An horar da shi tare da asarar sake ginawa da kuma mai nuna wariya irin na GAN, don haka fitowar ta yi sauti na halitta maimakon kawai ta kusa. Siffar da ta tsaya tsayin daka ita ce 'mai iya daidaitawa' ko horarwar juzu'i: ƙirar ƙira ɗaya na iya aiki a tsakanin bitrates daga kusan 3 zuwa 18 kbps kawai ta amfani da ƙarin ko ƙananan yadudduka na ƙididdigewa, ba tare da sake horarwa ba. A 3 kbps an ba da rahoton cewa ya fi Opus a 12 kbps a cikin gwajin sauraron sauraro, sarrafa magana, kiɗa, da sauti na gabaɗaya a cikin tsari ɗaya wanda zai iya aiki a ainihin lokacin akan CPU na wayar hannu.
Fahimtar Fasaha
Siffar igiyar igiyar ruwa ta ratsa ta cikin juzu'i masu rarrabuwar kawuna waɗanda suka yi ƙasa da misali sosai, suna samar da abin haɗawa ɗaya a kowane firam (misali firam 75/daƙiƙa). RVQ sannan ya sanya kowane sakawa azaman tarin fihirisar codebook. Bitrate yayi daidai da adadin firam yawan adadin lokutan ƙididdigewa mai aiki a kowane littafin lamba. Quantizer dropout ba da gangan ya yanke jigon RVQ yayin horo, yana tilasta wa littattafan farko don ɗaukar mahimman bayanai don haka codec ɗin ya ƙasƙanta da kyau a ƙananan farashi.
Dabarun Tasiri
Shiga ku isa
Yana inganta samun dama ta hanyar rubutu, ba da labari, da mu'amalar murya.
Kudin da kasafin kuɗi
Ƙungiyoyin kafofin watsa labaru na iya jigilar sauti mai gogewa cikin sauri tare da ƙaramin kasafin kuɗi.
Gudu da sikelin
Tsarin fuskantar abokin ciniki na iya aiwatar da hulɗar magana a mafi girman ma'auni.
Makomar SoundStream Neural Codec
SoundStream ya kafa samfuri wanda daga baya codecs kamar EnCodec da DAC suka daidaita, kuma alamun sa masu hankali sun zama tushen tsarin haɓakawa kamar AudioLM da MusicLM. Yi tsammanin zuriyar da za ta tura zuwa ko da ƙananan bitrates, ƙayyadaddun alamomin ma'ana waɗanda ke ninka a matsayin abubuwan shigar da nau'ikan nau'ikan sauti na harshe, da ƙaƙƙarfan tura kayan aiki don kiran kai-tsaye, kayan ji, da yawo inda bandwidth da latency ke takurawa sosai.
Aiwatar da Gaskiyar Duniya
Matsa kiran murya zuwa ~3 kbps yayin da yake ƙara bayyanawa fiye da na'urori na gado a mafi girman bitrates
Samar da sahihan alamun odiyo waɗanda ke ciyar da Google's AudioLM da samfuran ƙirƙira na MusicLM
Sauraron sauti mai ƙarancin bandwidth na ainihin lokaci akan na'urorin hannu tare da rufaffen kan-CPU da yankewa
Ajiye ko watsa kiɗa da sautin yanayi da kyau a cikin ƙirar ƙira ɗaya wanda ke sarrafa kowane nau'in abun ciki
Hatsari & Tsare-tsare
Rashin amfani da murya da haɗarin kwaikwaya yana ƙaruwa lokacin da aka rasa izini.
Daidaituwa na iya faɗuwa cikin lafuzza, yaruka, ko mahalli masu hayaniya.
Ana iya kuskuren sauti na roba don ingantacciyar magana ba tare da bayyananniyar lakabi ba.
Taswirar Hanya
Sami tabbataccen izini don ɗaukar murya, cloning, da sake amfani.
Gwajin ingantattun masu magana daban-daban da yanayin baya.
Ƙayyade lokacin da dole ne ɗan adam ya duba ko ya amince da abubuwan da aka fitar.
Yi lakabin sauti na roba da kuma adana bayanan da aka tabbatar don yin lissafi.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the SoundStream Neural Codec quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Neural Audio Codecs
Tambayoyin da ake yawan yi
What is SoundStream Neural Codec?
SoundStream shine Google's na ƙarshen-zuwa-ƙarshen audio codec na jijiya wanda ke matse magana da kiɗa zuwa ƙananan bitrates yayin kiyaye inganci. Yana da mahimmanci saboda yana bugun codecs na gargajiya kamar Opus a daidai wannan bitrate kuma yana ba da ikon ƙirar sauti na zamani.
Wadanne abubuwa uku ne suka hada da gine-ginen SoundStream?
An gina SoundStream daga na'urar rikodin juyin juya hali, ragowar vector quantizer wanda ke ɓata abubuwan da aka saka, da mai canza juyi, duk horon ƙarshe zuwa ƙarshe.
Wace dabara ce ke ƙyale samfurin SoundStream guda ɗaya yayi hidimar bitrates daban-daban ba tare da sake horarwa ba?
A lokacin horo, SoundStream ba da gangan ba ya sauke yadudduka na ƙididdigewa ta yadda za ku iya amfani da matakan RVQ da yawa ko kaɗan don buga bitrates daban-daban daga ƙirar ɗaya.
A cikin gwaje-gwajen sauraron Google, SoundStream a 3 kbps an ruwaito ya fi wani codec a 12 kbps?
SoundStream a kawai 3 kbps yayi daidai ko doke Opus codec da aka yi amfani da shi da yawa yana gudana a 12 kbps a cikin ƙimar ƙimar ƙima.
Wane irin ƙarin siginar horo ne SoundStream ke amfani da shi don yin sautin fitarwa na halitta?
Bayan hasarar sake ginawa, SoundStream yana amfani da masu nuna wariya (GAN) don haka sake ginawa ya yi sauti mai ma'ana maimakon lambobi kawai.
Ta yaya SoundStream's bitrate yake da alaƙa da ƙididdigansa?
Ma'auni na Bitrate tare da adadin raƙuman RVQ masu aiki (lokacin ƙididdige adadin lokutan ragi a kowane littafin code), don haka ƙara ƙididdiga yana ɗaga bitrate da inganci.