Moshi Full-Duplex Speech
Moshi is an open-source, real-time voice AI from Kyutai that talks and listens at the same time — full-duplex — instead of taking strict turns.
Overview
That removes the awkward lag and rigid turn-taking of traditional voice assistants.
Deep Dive
Moshi, released by the French lab Kyutai in 2024, is a speech-to-speech foundation model built for natural, low-latency conversation. Unlike pipeline assistants that chain speech-to-text, then a language model, then text-to-speech, Moshi handles audio directly and continuously. Its key idea is full duplex: it models two audio streams simultaneously — the user's and its own — so it can listen while speaking, handle interruptions, backchannel with 'mhm,' and overlap naturally like humans do. It reaches latency around 160-200 milliseconds, far below typical assistant lag. Under the hood it pairs a 7B-parameter text-and-audio language model (Helium) with Mimi, a neural audio codec that compresses speech into discrete tokens the model can generate. Kyutai released the weights and code openly.
Technical Insight
Moshi's trick is its Mimi codec, which turns continuous audio into a low-bitrate stream of discrete tokens at 12.5 Hz, including a distilled semantic token. The language model predicts its own speech tokens and the user's in parallel time-aligned streams, so generation never has to stop to 'listen.' An 'Inner Monologue' method predicts text before audio, improving the linguistic quality and coherence of what Moshi actually says.
Strategic Impact
Access and reach
It improves accessibility through transcription, narration, and voice interfaces.
Cost and budget
Media teams can ship polished audio faster with smaller budgets.
Speed and scale
Customer-facing systems can process spoken interactions at larger scale.
The Future of Moshi Full-Duplex Speech
Full-duplex modeling is becoming the template for natural voice AI, influencing systems across the industry. Expect smaller, on-device versions, multilingual support, lower latency, and integration into agents, customer service, and accessibility tools. Because Moshi is open, researchers can probe and improve it freely. Challenges remain around factual reliability, safety in overlapping speech, and emotional nuance, but the shift from rigid turn-taking to fluid, interruptible conversation is likely permanent.
Real-World Implementation
A hands-free voice companion you can interrupt mid-sentence, with replies in under 200 milliseconds.
Open research baseline for studying real-time, full-duplex spoken dialogue without proprietary black boxes.
Accessibility assistants that converse fluidly with users who need quick, natural back-and-forth.
Prototyping interruptible customer-service voice bots that backchannel and react while the caller is still talking.
Risks & Guardrails
Voice misuse and impersonation risks increase when consent is missing.
Accuracy can drop across accents, dialects, or noisy environments.
Synthetic audio can be mistaken for authentic speech without clear labeling.
Implementation Roadmap
Obtain explicit consent for voice capture, cloning, and reuse.
Test quality across diverse speakers and background conditions.
Define when a human must review or approve outputs.
Label synthetic audio and keep provenance records for accountability.
Keep Exploring
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Moshi Full-Duplex Speech quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Next guide
Text Normalization for Speech
Frequently asked questions
What is Moshi Full-Duplex Speech?
Moshi is an open-source, real-time voice AI from Kyutai that talks and listens at the same time — full-duplex — instead of taking strict turns. That removes the awkward lag and rigid turn-taking of traditional voice assistants.
What does 'full-duplex' mean in the context of Moshi?
Full-duplex means the model handles incoming and outgoing audio simultaneously, so it can listen while speaking and handle interruptions naturally.
Which organization released Moshi?
Moshi was developed and open-sourced by Kyutai, a French AI research lab, in 2024.
What is Mimi in the Moshi system?
Mimi is the neural audio codec that compresses continuous speech into a low-bitrate stream of discrete tokens the model can generate.
How does Moshi differ from a traditional voice-assistant pipeline?
Traditional assistants chain speech-to-text, a language model, and text-to-speech; Moshi is a single speech-to-speech model handling audio continuously.
Roughly what conversational latency does Moshi target?
Moshi achieves latency around 160-200 ms, far lower than typical voice assistants, enabling natural overlap and interruption.