Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS

Every voice interaction has a latency budget.

By the time a user hears your application respond, you've already spent precious milliseconds capturing audio, transcribing speech, running an LLM, retrieving context, and generating a response. Text-to-speech (TTS) is the final step — and the one users notice most. If speech generation is slow, the whole experience feels slow.

The more of that pipeline you can run and tune yourself, the more of the latency budget you get back.

Voice AI is moving fast. Integrated speech models offer simplicity — one API call, audio in, audio out — but they trade the ability to fine-tune each component for your domain, swap in better models as they ship, enforce data residency, and understand exactly where latency is coming from. For more control, a cascaded architecture — purpose-built ASR, TTS, and LLM components running together — keeps each layer independently tunable and deployable on infrastructure you own.

NVIDIA Magpie Multilingual TTS is built for that. With open weights, production-ready NVIDIA NIM, and support for 12 languages, you can deploy multilingual speech inside your own infrastructure, optimize latency for your workload, and customize the model for your domain — end to end, in your own environment.

The latest release expands multilingual coverage with Modern Standard Arabic, Korean, and Brazilian Portuguese, while improving quality across many existing languages through updated training data and model improvements.

Whether you're building customer support agents, healthcare assistants, enterprise copilots, translation systems, or conversational AI applications, Magpie provides an open foundation for production voice AI.

Voice AI Is Becoming Multilingual by Default

Today's voice applications don't serve a single language.

Global customer support, enterprise assistants, healthcare documentation, retail automation, and translation workflows increasingly require natural conversations across multiple languages — all while maintaining low latency.

Supporting more languages is only part of the challenge. Developers also need the ability to:

- Deploy where their data lives

- Meet enterprise privacy requirements

- Customize pronunciation and voices

- Predict latency under production workloads

- Scale on their own infrastructure

Open models change what's possible on every one of these.

One Open Model, Twelve Languages

Magpie TTS Multilingual is a 364M-parameter open-weights model supporting:

English · Spanish · French · German · Italian · Vietnamese · Mandarin · Hindi · Japanese · Modern Standard Arabic (new) · Korean (new) · Brazilian Portuguese (new)

Each language includes male and female speaker voices through a shared multilingual speaker representation.

This release also improves multilingual flexibility with expanded code-switching support for Hindi and Japanese, enabled through IPA grapheme-to-phoneme processing and custom pronunciation dictionaries — making it easier to accurately pronounce names, technical terminology, and mixed-language content.

Instead of maintaining separate TTS models for different regions, developers can build multilingual applications on a single open foundation.

The Latency Your Users Actually Notice