Kodeku Audio Mimi
Mimi codec audio neural la buy tënk wax ci ay token yu ndaw ci jamono dëgg, suko defee model IA yi mëna déglu ak wax ci latency bu tuuti lool.
Résumé
It is the audio backbone behind Kyutai's Moshi voice model.
Plongeur bu xóot
Mimi, bi lab français Kyutai genne ci 2024, codec neuronal la buy soppi audio 24 kHz ci ay token yu wuute ci lu tollu ci 1.1 kbps ak 12.5 token ci seconde. Dafay jëfandikoo enkodeer-dekodeer bu am kantite vecteur residuel (RVQ), di xaaj jeton yi ci niveau 'semantik' bu njëkk buñ distiye ci modelu wax buñuy saytu boppam (WavLM) boole ci niveau 'akustik' yu bari yuy jàpp texture baat bi. Li gëna am solo mooy streaming bu mat sëkk ak sabab: dafay génne ay token yu audio bi yegsee, du xaar clip bu mat sëkk, ak lu tollu ci 80 ms ci latency. Loolu dafay tax xeetu làkk bi di jëfandikoo kàddu yi ni ay token bind, loolu mooy tax Moshi mëna waxtaan ci duplex bu mat sëkk, boole ci tëye audio biñ defaraat mu leer te natureel.
Gis-gis xarala
Kafe Mimi mooy xaaj RVQ. Codebook bu njëkk bi dañu ko tàggat ak ñàkka distillation ngir méngoo ak embeddings yu bawoo ci WavLM, di ko forse mu yóbbaale 'tekki' fonetik, ci noonu la codebook akustik paralel di defaraat detaay yi ci forme onde. Transformateur bi dafay dox ci biir bottleneck bi, ba noppi ab perte adversarial (GAN) ci decodeur bi dafay gëna ñaw kalite biy génn. Convolusioŋ yiy waral lépp di daw, moo tax latency bi dafay des ci 80 ms.
njeextalu pexe
Dugg ak yegg
Dafay gëna yombal jëfandikoo gi jaaraleko ci transkripsioŋ, nettali ak interfaasu baat.
Njëgg ak budget
Ekipu mejaa yi mën nañu yónnee audio bu leer ci anam wu gëna gaaw te seen xaalis gëna néew.
Gaawaay ak yaatuwaay
Sistem yiy jàkkarloo ak kiliyaan bi mën nañu def waxtaan ci anam wu gëna yaatu.
Ëlëgu Codec Audio Mimi
Xaarandil codec yu melni Mimi ñu nekk interface buñ miin diggante model audio ak làkk yu yaatu, di puus assistant baat ci jamono dëgg ci diiru tontu bu néew 100 ms. Gëstu dafay wàññi tolluwaayu token yi, boole ci baña yàq dàntite waxkat bi, yëg-yëg bi ak music bi. Ndax Kyutai ubbeeku na Mimi ak Moshi, mën nañu ni dina jur sistem yu bari yu ubbeeku ci wax-ci-wax, ay assistant ci aparey yi, ak jumtukaayi jokkoo baat yu am bandwidth bu woyof.
Doxal ci àdduna dëgg
Dafay dooleel assistant baat bu Moshi bu Kyutai suko defee mu mëna déglu ak wax benn yoon
Streaming tokens wax ci modelu làkk ngir tekki kàddu-ci-kaddu ci jamono dëgg
Wootu baat yu am bitrate bu woyof (~1.1 kbps) ngir anam yu reso bu baaxul wala bu xawa jaxasoo
Tokenizing audio ngir wax ak pipeline bind-ci-kaddu buy xalaat ci kaw son bu melni bind
Risk yi ak balustrade yi
Jëfandikoo baat ci anam wu jaarul yoon ak niru ak nit dafay gëna yokk sudee nanguwul.
Jaar-jaar mën na wàññeeku ci aksan yi, dialect yi wala barab yu bari xumbaay.
Audio synthetik mën nañu ko jaawale ak wax ju dëggu sudee amul etiket bu leer.
Roadmap ngir samp gi
Wutal ndigal bu leer ngir jàpp baat bi, klone ko ak jëfandikoowaat ko.
Saytu kalite ci kàddukat yu bari ak anam yu bari ci ginaaw.
Mandargal kañ la nit wara xoolaat wala nangu ay génne.
Etiketu audio synthetik te nga denc dokimaa ci fimu bawoo ngir mëna lim.
Weyal di banneexu
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Mimi Streaming Audio Codec quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Gis bi ci topp
Kodeku audio neuronal
Laaj yi ñuy faral di laaj
What is Mimi Streaming Audio Codec?
Mimi codec audio neural la buy tënk wax ci ay token yu ndaw ci jamono dëgg, suko defee model IA yi mëna déglu ak wax ci latency bu tuuti lool. Mooy yax gi nekk ci ginaaw modelu baat Moshi bu Kyutai.
Ban labo moo genne Mimi ak modelu baat Moshi?
Mimi ak Moshi ñu ngi leen génne ci 2024 ci labo gëstu bu français bi tuddu Kyutai, ñoom ñaar ñépp ñu ngi leen génne.
Lu tollu ci ñaata token la Mimi di def ci seconde ngir wax?
Mimi dafay kompresse audio bu 24 kHz ba ci lu tollu ci 12.5 token ci seconde ci lu tollu ci 1.1 kbps.
Lan mooy téere kode bi njëkk ('semantik') ci Mimi?
Niveau RVQ bu njëkk bi dañu koy tàggat ci distilaasioŋ bu bawoo ci WavLM ngir yóbbaale ëmbiit semantik/fonetik, ci noonu la niveau yu ci topp yi di yokk ay detay akustik.
Lu tax ñuy wax Mimi ni 'streaming' wala 'causal'?
Mimi dafay doxal audio bi ci anam wu jaar yoon, ba noppi di genne ay token yuy yokk, di joxe lu tollu ci 80 ms latency bu méngoo ak waxtaan ci saasi.
Ban pexem kantite la Mimi di jëfandikoo ngir enkode audio bi?
Mimi dafay jëfandikoo kantite vecteur residuel, dajale téere kode yu bari suko defee ku nekk ci ñoom yokk ay detay yu gëna ndaw ci bi njëkk.