Mimi Streaming Audio Codec
Mimi codec ne na jijiya mai jiwuwa wanda ke danne magana cikin ƙaramin rafi na alamomi masu ƙima a cikin ainihin lokaci, don haka ƙirar AI na iya saurara da magana da ƙarancin jinkiri.
Dubawa
It is the audio backbone behind Kyutai's Moshi voice model.
Zurfafa nutsewa
Mimi, wanda gidan binciken Faransa na Kyutai ya saki a cikin 2024, codec ne na jijiya wanda ke juya sautin 24 kHz zuwa rafi na alamomi masu hankali a kusan 1.1 kbps kuma alamun 12.5 kawai a cikin daƙiƙa. Yana amfani da maɓalli-dikodi tare da ragowar vector quantization (RVQ), rarrabuwar alamomi zuwa matakin farko na 'semantic' wanda aka karkasa daga ƙirar magana mai kulawa da kai (WavLM) da matakan 'acoustic' da yawa waɗanda ke ɗaukar rubutun murya. Mahimmanci yana da cikakken yawo da sanadi: yana fitar da alamu yayin da sauti ya zo maimakon jiran cikakken shirin, tare da kusan 80 ms na latency. Wannan yana ƙyale samfurin harshe ya ɗauki magana kamar alamomin rubutu, yana ba Moshi damar yin magana cikin cikakken duplex yayin da yake kiyaye sautin da aka sake ginawa da fahimta da na halitta.
Fahimtar Fasaha
Dabarar Mimi shine tsarin raba-RVQ. An horar da littafin code na farko tare da hasarar ɓarna don dacewa da abubuwan da aka haɗa daga WavLM, tilasta shi ɗaukar ma'anar sauti, yayin da littafan rikodin sauti na layi ɗaya suna sake gina dalla-dalla. Mai Transformer yana aiki a cikin ƙugiya, kuma asarar gaba (GAN) akan na'urar tana haɓaka ingancin fitarwa. Juyin yanayi yana kiyaye komai yana gudana, don haka latency yana kusa da 80 ms.
Dabarun Tasiri
Shiga ku isa
Yana inganta samun dama ta hanyar rubutu, ba da labari, da mu'amalar murya.
Kudin da kasafin kuɗi
Ƙungiyoyin kafofin watsa labaru na iya jigilar sauti mai gogewa cikin sauri tare da ƙaramin kasafin kuɗi.
Gudu da sikelin
Tsarin fuskantar abokin ciniki na iya aiwatar da hulɗar magana a mafi girman ma'auni.
Makomar Mimi Streaming Audio Codec
Yi tsammanin codecs kamar Mimi su zama daidaitaccen mu'amala tsakanin sauti da samfuran harshe, tura mataimakan murya na ainihi zuwa lokutan amsawar ms-100. Bincike yana tuƙi farashin alamar har ma da ƙasa yayin da ake adana ainihin lasifikar, motsin rai, da kiɗa. Saboda Mimi da Moshi na Bude Kyutai, da alama zai iya samar da tsarin magana-zuwa-magana da yawa, mataimakan na'urori, da kayan aikin sadarwar murya mara ƙarfi.
Aiwatar da Gaskiyar Duniya
Ƙaddamar da Kyutai's Moshi cikakken mataimakin murya mai duplex domin ya iya sauraro da magana lokaci guda
Yawo alamomin magana cikin ƙirar harshe don fassarar magana-zuwa-lokaci na gaske
Kiran murya mai ƙarancin-ƙasa (~ 1.1 kbps) don matalauta ko cunkoso yanayin cibiyar sadarwa
Alamar sauti don magana mai ƙirƙira da bututun rubutu-zuwa-magana waɗanda ke yin dalili akan sauti kamar rubutu
Hatsari & Tsare-tsare
Rashin amfani da murya da haɗarin kwaikwaya yana ƙaruwa lokacin da aka rasa izini.
Daidaituwa na iya faɗuwa cikin lafuzza, yaruka, ko mahalli masu hayaniya.
Ana iya kuskuren sauti na roba don ingantacciyar magana ba tare da bayyananniyar lakabi ba.
Taswirar Hanya
Sami tabbataccen izini don ɗaukar murya, cloning, da sake amfani.
Gwajin ingantattun masu magana daban-daban da yanayin baya.
Ƙayyade lokacin da dole ne ɗan adam ya duba ko ya amince da abubuwan da aka fitar.
Yi lakabin sauti na roba da kuma adana bayanan da aka tabbatar don yin lissafi.
Ci gaba da Bincike
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mimi Streaming Audio Codec quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Jagora na gaba
Neural Audio Codecs
Tambayoyin da ake yawan yi
What is Mimi Streaming Audio Codec?
Mimi codec ne na jijiya mai jiwuwa wanda ke danne magana cikin ƙaramin rafi na alamomi masu ƙima a cikin ainihin lokaci, don haka ƙirar AI na iya saurara da magana da ƙarancin jinkiri. Kashin bayan sauti ne a bayan samfurin muryar Moshi na Kyutai.
Wane lab ne ya saki Mimi da samfurin muryar Moshi?
An sake Mimi da Moshi a cikin 2024 ta gidan binciken Faransa Kyutai, wanda ya buɗe duka biyun.
Kusan alamomi nawa Mimi ke fitarwa don magana?
Mimi tana matsar sautin 24 kHz zuwa kusan alamomin 12.5 a sakan daya a kusan 1.1 kbps.
Menene littafin code na farko ('semantic') a cikin kamawar Mimi?
An horar da matakin RVQ na farko ta hanyar distillation daga WavLM don ɗaukar abun ciki na ma'ana/semantic, yayin da matakan daga baya suna ƙara dalla-dalla.
Me yasa aka siffanta Mimi a matsayin 'streaming' ko 'dalili'?
Mimi tana aiwatar da sauti ta hanyar dalili kuma tana fitar da alamu a hankali, tana ba da jinkirin 80 ms wanda ya dace da zance kai tsaye.
Wace dabarar ƙididdigewa Mimi ke amfani da ita don ɓoye sauti?
Mimi tana amfani da ragowar vector ƙididdigewa, tana tattara littattafai masu yawa don haka kowanne ya ƙara dalla-dalla ga na baya.