Kyutai Moshi

✓ Editorially verified

Open-source, full-duplex speech-to-speech foundation model with sub-200ms latency

Free· Free and open source. Models under CC-BY 4.0, code under MIT (Python) / Apache 2.0 (Rust). Self-hosted only — you pay your own compute (24GB+ GPU for PyTorch, or Apple Silicon via MLX).AudioMoshi (7B-class speech-text foundation model) + Mimi neural audio codec, in-house by Kyutai

Visit website →

Best for

Researchers, voice-AI engineers and indie teams who need a self-hosted, low-latency spoken-dialogue model with open weights and no per-minute API costs.

Skip if

Product teams that want a managed voice API, multilingual coverage out of the box, or GPT-4o-level knowledge and reasoning wrapped in speech.

Moshi is a speech-text foundation model released by the French non-profit AI lab Kyutai for real-time, full-duplex spoken dialogue. Unlike traditional voice assistants that pipeline ASR to an LLM to TTS, Moshi natively processes two continuous audio streams — the user's microphone and its own synthesized speech — while simultaneously predicting text tokens as an 'inner monologue' that grounds what it says. The result is a conversational voice model that can be interrupted, back-channel ('mm-hmm'), talk over the user, and respond with roughly 200ms of end-to-end latency on an L4 GPU, well below the human turn-taking threshold. The stack includes Mimi, a neural audio codec that compresses 24kHz speech to a 1.1kbps stream at 12.5Hz with 80ms frame latency and outperforms competing streaming codecs. Two fine-tuned voice variants ship: Moshiko (male synthetic voice) and Moshika (female). Weights are available in PyTorch (bf16, int8), MLX (int4/int8/bf16 for on-device Mac and iPhone inference), and a Rust/Candle backend intended for production deployments. Typical workflows are running the browser UI with built-in echo cancellation for demos and research, embedding the Rust server behind a WebSocket for latency-sensitive voice apps, or using MLX to prototype on-device voice interactions on Apple Silicon. Because everything is open weights under permissive licenses, Moshi is best suited to researchers exploring duplex speech modeling, engineers building low-latency voice interfaces without sending audio to a third-party API, and teams that need to fine-tune or study the internals of a full speech dialogue system.

Editor's take

Moshi is the most interesting open release in real-time voice AI — genuinely full-duplex, sub-200ms, and permissively licensed. It won't replace GPT-4o Voice for consumer polish, but if you care about owning your voice stack or studying how duplex speech models actually work, nothing else in the open ecosystem comes close.

— The AI Tool Bible editorial team

Pros

✅ Truly full-duplex — handles interruptions, overlap and back-channels rather than rigid turn-taking
✅ Sub-200ms practical latency on a single L4 GPU, well below third-party voice APIs
✅ Fully open weights (CC-BY 4.0) plus MIT/Apache code — self-host with no per-minute billing
✅ Ships with Mimi, a streaming neural audio codec that beats SpeechTokenizer and SemantiCodec
✅ Multiple inference backends: PyTorch for research, Rust/Candle for production, MLX for on-device Mac/iPhone
✅ Inner-monologue text prediction gives you a transcript alongside the audio stream for free

Cons

⚠️ English-only voices at launch — no multilingual support out of the box
⚠️ Knowledge and reasoning quality trail top text LLMs; it's a 7B-class model, not GPT-4o Voice
⚠️ Requires a 24GB+ GPU for the reference PyTorch build; on-device is only viable via MLX on Apple Silicon
⚠️ No hosted API or SaaS tier — you own the ops, scaling and safety filtering
⚠️ Only two fixed synthetic voices (Moshiko/Moshika); no voice cloning or speaker conditioning in the release

Use cases

Real-time voice assistant prototypesResearch on full-duplex spoken dialogueOn-device voice interaction on Apple Silicon via MLXLow-latency conversational agents behind WebSocketNeural audio codec experimentation with MimiSelf-hosted voice interface for privacy-sensitive appsSpeech tokenizer for downstream audio LLM trainingInterruptible in-car or wearable voice UX