Ini Kutenderera Audio Codec
Mimi ndeye neural odhiyo codec inomanikidza kutaura kuita diki diki diki tokeni munguva chaiyo, saka maAI mamodheru anogona kuteerera uye kutaura neakadzika kwazvo latency.
Pfupiso
It is the audio backbone behind Kyutai's Moshi voice model.
Kudzika Kwakadzika
Mimi, akaburitswa neFrench lab Kyutai muna 2024, ndeye neural codec inoshandura 24 kHz odhiyo kuita rukova rwe discrete tokens paanenge 1.1 kbps uye chete 12.5 tokens pasekondi. Inoshandisa encoder-decoder ine yasara vector quantization (RVQ), kupatsanura tokeni kuita 'semantic' yekutanga nhanho yakanyungudutswa kubva kune inozvitarisisa yekutaura modhi (WavLM) pamwe akati wandei 'acoustic' mazinga anobata manyorerwo ezwi. Kunyanya iyo iri kutenderera yakazara uye inokonzeresa: inoburitsa tokeni sezvo odhiyo ichisvika pane kumirira yakazara clip, ine angangoita 80 ms ye latency. Izvi zvinoita kuti modhi yemutauro ibate kutaura senge tokens, zvichiita kuti Moshi akwanise kutaura mune duplex yakazara uku achichengeta odhiyo yakagadziridzwa inonzwisisika uye yakasikwa.
Technical Insight
Hunyengeri hwaMimi ndeyekupatsanura-RVQ chirongwa. Iro bhuku rekutanga rekodhi rakadzidziswa nekurasikirwa kwe distillation kuenzanisa embeddings kubva kuWavLM, ichimanikidza kuti itakure phonetic 'zvinoreva,' nepo parallel acoustic codebooks inovakazve waveform ruzivo. A Transformer inoshanda mukati mebhodhoro, uye adversarial (GAN) kurasikirwa pane decoder inorodza kunaka kwekubuda. Causal convolutions inochengeta zvese zvichitenderera, saka latency inogara padyo ne80 ms.
Strategic Impact
Svika uye svika
Inonatsiridza kusvikika kuburikidza nekunyora, kurondedzera, uye mazwi ekubatanidza.
Mutengo uye bhajeti
Zvikwata zveMedia zvinogona kutumira odhiyo yakakwenenzverwa nekukurumidza nemabhajeti madiki.
Kumhanya uye chiyero
Masisitimu anotarisana nevatengi anogona kugadzirisa kutaurirana kwekutaura pamwero mukuru.
Ramangwana reMimi Kutenderera Audio Codec
Tarisira macodecs akaita saMimi kuti ave akajairika interface pakati peodhiyo nemitauro mikuru, kusunda vabatsiri vezwi chaiyo-nguva kusvika sub-100 ms mhinduro nguva. Tsvagiridzo iri kutyaira mareti etokeni zvakatodzika uchichengetedza kuzivikanwa kwemutauri, manzwiro, uye mimhanzi. Nekuti Kyutai yakavhurika-sourced Mimi naMoshi, ingangodyara akawanda akavhurika ekutaura-kune-kutaura masisitimu, pane-mudziyo vabatsiri, uye ekupedzisira-yakaderera-bandwidth izwi rekutaurirana maturusi.
Real-World Implementation
Kusimbisa Kyutai's Moshi yakazara-duplex izwi mubatsiri kuti ikwanise kuteerera uye kutaura panguva imwe chete
Kuburitsa ma tokeni ekutaura mumhando yemutauro yeshanduro yenguva chaiyo yekutaura-kukutaura
Mazwi emafoni epamusoro-pasi-bitrate (~ 1.1 kbps) yemamiriro etiweki asina kunaka kana akazara
Kuratidza odhiyo yekutaura kunogadzira uye mameseji-ku-kutaura-mapaipi anokonzeresa pamusoro pekurira semavara
Njodzi & Guardrails
Kushandisa izwi zvisizvo uye njodzi dzekuedzesera dzinowedzera kana chibvumirano chisipo.
Kururama kunogona kudonha mumitauro, mataurirwo, kana nharaunda dzine ruzha.
Synthetic audio inogona kukanganisa kutaura kwechokwadi isina mavara akajeka.
Implementation Roadmap
Wana mvumo yakajeka yekutora inzwi, kugadzira, uye kushandisa zvakare.
Yedza mhando pavatauri vakasiyana uye mamiriro ekumashure.
Tsanangura apo munhu anofanira kuongorora kana kubvumidza zvabuda.
Label synthetic odhiyo uye chengetedza marekodhi ekuzvidavirira.
Ramba Uchiongorora
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mimi Streaming Audio Codec quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gaidhi rinotevera
Neural Audio Codecs
Mibvunzo inowanzo bvunzwa
What is Mimi Streaming Audio Codec?
Mimi ndeye neural odhiyo codec inomanikidza kutaura kuita diki diki diki tokeni munguva chaiyo, saka maAI mamodheru anogona kuteerera uye kutaura neakadzika kwazvo latency. Ndiyo inonzwika musana kuseri kweKyutai's Moshi voice modhi.
Ndeipi lab yakaburitsa Mimi neMoshi voice modhi?
Mimi naMoshi vakaburitswa muna 2024 neFrench research lab Kyutai, iyo yakavhura-sourced vese.
Zvingangove zvingani zviratidzo pasekondi izvo Mimi anoburitsa pakutaura?
Ini anomanikidza 24 kHz odhiyo pasi kusvika kune angangoita 12.5 tokens pasekondi pane angangoita 1.1 kbps.
Ko rekutanga ('semantic') codebook muMimi rinotora chii?
Yekutanga RVQ nhanho inodzidziswa kuburikidza ne distillation kubva kuWavLM kutakura semantic / phonetic zvemukati, nepo gare gare mazinga anowedzera acoustic ruzivo.
Sei Mimi achitsanangurwa se'kuyerera' kana 'kukonzeresa'?
Mimi inogadzira odhiyo inokonzeresa uye inoburitsa tokens zvakanyanya, ichipa anenge 80 ms latency akakodzera kutaura kwepamoyo.
Ndeipi nzira yekuwedzera iyo Mimi anoshandisa encode odhiyo?
Mimi anoshandisa yakasara vector quantization, kurongedza akawanda macodebooks kuitira kuti imwe neimwe iwedzere yakakwenenzverwa kune yakapfuura.