I-Mimi Streaming Audio Codec
I-Mimi iyikhodekhi yomsindo we-neural ecindezela inkulumo ibe umfudlana wamathokheni ahlukahlukene ngesikhathi sangempela, ukuze amamodeli e-AI akwazi ukulalela futhi akhulume ngokubambezeleka okuphansi kakhulu.
Uhlolojikelele
It is the audio backbone behind Kyutai's Moshi voice model.
I-Deep Dive
I-Mimi, ekhishwe ilebhu yaseFrance i-Kyutai ngo-2024, iyikhodekhi ye-neural eguqula umsindo we-24 kHz ube ukusakazwa kwamathokheni ahlukahlukene cishe ku-1.1 kbps kanye namathokheni angu-12.5 kuphela ngomzuzwana. Isebenzisa i-encoder-decoder ene-residual vector quantization (RVQ), ihlukanise amathokheni abe ileveli yokuqala 'ye-semantic' exutshwe kusuka kumodeli yenkulumo ezigadile (WavLM) kanye namaleveli 'acoustic' ambalwa athwebula ukuthungwa kwezwi. Okubalulekile ukuthi isakaza ngokugcwele futhi iyimbangela: ikhipha amathokheni njengoba umsindo ufika kunokulinda isiqeshana esigcwele, esine-latency engaba ngu-80 ms. Lokhu kuvumela imodeli yolimi iphathe inkulumo njengamathokheni ombhalo, okwenza i-Moshi ikwazi ukuxoxa nge-duplex egcwele kuyilapho igcina umsindo owakhiwe kabusha uqondakala futhi ungokwemvelo.
I-Technical Insight
Iqhinga lika-Mimi wuhlelo lwe-split-RVQ. I-codebook yokuqala iqeqeshelwe ukulahlekelwa kwe-distillation ukuze ifane nokushumeka okuvela ku-WavLM, okuphoqelela ukuthi iphathe 'incazelo' yefonetiki, kuyilapho ama-codebook ahambisanayo akha kabusha imininingwane ye-waveform. I-Transformer isebenza ngaphakathi kwebhodlela, futhi ukulahlekelwa kwe-adversarial (GAN) kusikhiphi silola ikhwalithi yokuphumayo. I-causal convolutions igcina yonke into isakaza, ngakho-ke ukubambezeleka kuhlala eduze kuka-80 ms.
I-Strategic Impact
Finyelela futhi ufinyelele
Ithuthukisa ukufinyeleleka ngokuloba, ukulandisa, nezixhumi ezibonakalayo zezwi.
Izindleko kanye nesabelomali
Amaqembu emidiya angathumela umsindo opholishiwe ngokushesha ngamabhajethi amancane.
Isivinini nesikali
Amasistimu abhekene nekhasimende angacubungula ukusebenzelana okukhulunyiwe ngesilinganiso esikhulu.
Ikusasa le-Mimi Streaming Audio Codec
Lindela amakhodekhi afana no-Mimi ukuthi abe isixhumi esibonakalayo esijwayelekile phakathi kwamamodeli omsindo nezilimi ezinkulu, asunduze abasizi bezwi besikhathi sangempela ezikhathini zokuphendula ezingaphansi kwe-100 ms. Ucwaningo lwehlisa amanani ethokheni ngenkathi kugcinwa ubunikazi besikhulumi, imizwa, nomculo. Ngenxa yokuthi i-Kyutai inemithombo evulekile engu-Mimi kanye ne-Moshi, kungenzeka ithole amasistimu amaningi avulekile enkulumo-kuya-inkulumo, abasizi abakudivayisi, namathuluzi okuxhumana ezwi aphansi kakhulu.
Ukuqaliswa Komhlaba Wangempela
Inika amandla umsizi wezwi we-Moshi we-Kyutai ogcwele-duplex ukuze alalele futhi akhulume kanyekanye
Ukusakaza amathokheni enkulumo kumodeli yolimi yokuhumusha ngesikhathi sangempela inkulumo-kuya-enkulumweni
Amakholi ezwi e-Ultra-low-bitrate (~1.1 kbps) ngezimo zenethiwekhi ezimbi noma eziminyene
Ukwenza ithokheni yomsindo wenkulumo ekhiqizayo namapayipi okuguqula umbhalo-ube-inkulumo acabanga ngomsindo njengombhalo
Izingozi & Guardrails
Ukusetshenziswa kabi kwezwi kanye nezingozi zokuzenza ongeyena ziyanda uma imvume ingekho.
Ukunemba kungase kwehle kuzo zonke izinhlobo zokuphimisela, izilimi zesigodi, noma izindawo ezinomsindo.
Umsindo wokwenziwa ungenziwa iphutha njengenkulumo eyiqiniso ngaphandle kokulebula okucacile.
Ukuqalisa Umhlahlandlela
Thola imvume esobala yokuthwebula izwi, ukuhlanganisa, nokusebenzisa kabusha.
Ikhwalithi yokuhlola kuzo zonke izipikha nezimo zangemuva.
Chaza ukuthi kunini lapho umuntu kufanele abuyekeze noma agunyaze okuphumayo.
Lebula umsindo wokwenziwa futhi ugcine amarekhodi atholakalayo ukuze aziphendulele.
Qhubeka Uhlole
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mimi Streaming Audio Codec quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Umhlahlandlela olandelayo
I-Neural Audio Codecs
Imibuzo evame ukubuzwa
What is Mimi Streaming Audio Codec?
I-Mimi iyikhodekhi yomsindo we-neural ecindezela inkulumo ibe umfudlana wamathokheni ahlukahlukene ngesikhathi sangempela, ukuze amamodeli e-AI akwazi ukulalela futhi akhulume ngokubambezeleka okuphansi kakhulu. Iwumgogodla womsindo ngemuva kwemodeli yezwi ka-Kyutai ye-Moshi.
Iyiphi ilebhu ekhiphe uMimi kanye nemodeli yezwi yaseMoshi?
UMimi noMoshi bakhululwa ngo-2024 ngelebhu yocwaningo yaseFrance iKyutai, eyavula bobabili.
Cishe mangaki amathokheni ngomzuzwana u-Mimi akhiphayo ukuze akhulume?
U-Mimi ucindezela umsindo ongu-24 kHz wehle ufike cishe kumathokheni angu-12.5 ngomzuzwana cishe ku-1.1 kbps.
Ithatha ini i-codebook yokuqala ('semantic') ku-Mimi?
Ileveli yokuqala ye-RVQ iqeqeshwa nge-distillation kusuka ku-WavLM ukuze iphathe okuqukethwe kwe-semantic/phonetic, kuyilapho amaleveli akamuva engeza imininingwane ye-acoustic.
Kungani uMimi echazwa ngokuthi 'usakaza' noma 'umbangela'?
I-Mimi icubungula umsindo ngesizathu futhi ikhipha amathokheni ngokwandayo, inikeze cishe ukubambezeleka okungu-80 ms okulungele ingxoxo ebukhoma.
Iyiphi indlela yokulinganisa i-Mimi eyisebenzisayo ukufaka ikhodi yomsindo?
I-Mimi isebenzisa ukulinganisa kwe-vector eyinsalela, inqwabelanisa ama-codebook amaningi ukuze ngayinye yengeze imininingwane emincane kunangaphambili.