Bench · Text to speech · October 2, 2026
Which voice should Holly speak with?
Holly talks back on your own Mac. So we gave 14 voice models the same four lines, in English, Chinese, and both mixed, and kept every clip.
Our two picks
Two kinds of voice model, and the one we'd reach for in each. Your ears get the final say.
Built-in voices
Kokoro
What Holly speaks with today. Tiny and very fast, with a set of fixed voices in English and Mandarin. It switches to a Mandarin voice for Chinese, so mixed sentences change voice.
- real time, in Python on the GPU
- memory at most, in audio.cpp
- 82M
- parameters
Voice cloning
Qwen3-TTS
Copies a voice from a few seconds of audio, and keeps that one voice through English, Chinese and both mixed. Whisper heard back of what the 1.7B said, as much as any model we can share that also sounds clean. The 0.6B trades a little of that for speed.
- real time for 1.7B, for 0.6B
- memory at most, 1.7B
- Apache
- 2.0 license, free to ship
The same family also comes with preset speakers (CustomVoice, tested here) that can take tone instructions like "very happy" (not tested yet), and a version that designs a new voice from a written description (VoiceDesign, not tested yet).
Listen and compare
Pick a line and a voice. Models that copy voices heard the same short clip first, so the difference you hear is the model. Suggested order puts our picks first, then the clearest (how much of each line Whisper heard back), then the ones that sounded off when we listened. You can also sort by speed, memory or size. The ones we can't share audio for are always last.
Speed
Seconds of speech made per second of work, typical across all eight clips. Above 1× it can keep up with a conversation. Heard back: how much of the four lines Whisper wrote down correctly from the clearest run.
Python means mlx-audio on the Mac's GPU, except Kokoro (hexgrad's own kokoro package, PyTorch on the GPU, given the same phonemes as audio.cpp; Holly runs Kokoro with onnx on the CPU today) and S1-mini (PyTorch). audio.cpp is a native engine in C++ on the GPU. Peak memory is the most the run used. Load is the time to start and load the model in audio.cpp.
What we learned
- The native engine isn't always faster. Qwen3-TTS 1.7B went from in Python to in audio.cpp, using less memory. But with the exact same Kokoro model and the same phonemes in both, Python on the GPU made speech at and audio.cpp at . Chatterbox came out about the same in both.
- An accent follows the voice it copies. Copy an English voice and the Chinese comes out with an English accent. Copy a Mandarin voice and the English words get a Chinese one. That's why every model here says every line in both voices.
- English-only models fall apart on Chinese. Pocket TTS made a few seconds of sound for the Chinese line. Whisper heard "Tui?".
- The biggest model isn't the answer on a laptop. Fish Audio S2 Pro needed up to of memory and made speech at about a third of real time, so a reply would wait. Its license is also non-commercial.
- When audio.cpp took longer, it had made more sound. A few clips took longer in audio.cpp, but each time it had made a longer clip, mostly English-only models babbling through Chinese words that Python skipped (Pocket made 37 seconds of sound for a line Python voiced in 8). Per second of speech, most of those gaps disappear: Pocket was within about 10% either way. Chatterbox is the real exception, up to a quarter slower in audio.cpp: it builds speech step by step on the GPU, the same work in both engines, so C++ saves almost nothing and small differences show.
- Kokoro's trouble with mixed lines was the older model, not the engine. Kokoro v1.0 reads English words inside Chinese with its Chinese reader, in Python and in audio.cpp alike, so "Claude" comes out "lao de". Kokoro v1.1-zh, made for mixed lines, says every line right in both. That's the one compared here, and the one Holly uses for Mandarin.
- Small details break things. In audio.cpp, Chatterbox didn't take Chinese at all, and its command line quietly dropped some settings: we patched it so Kokoro could be given the same phonemes as Python.
How we tested
- The Mac. , with other apps open.
- The lines. Four short paragraphs: English, Chinese, mostly English with Chinese words, and mostly Chinese with English words. Each is shown above its players.
- The voices. Every model was asked for an English voice and a Mandarin voice. Models that copy voices heard a short clip of Kokoro's Heart or zf_001 voice first. Models with fixed voices used their closest ones, named on each clip.
- Speed is the time to make each clip, once, after the model was loaded. Each model ran on its own.
- Whisper heard is what Whisper large v3 wrote down from each clip, unedited. It shows skipped or repeated words. It is not a quality score, and it often spells Chinese words in English letters.
- Settings. Each model's defaults, with a fixed seed where it takes one. The precision of every run is shown next to its engine: most ran as 8-bit model files, so their 16-bit versions are still to test. Kokoro ran at 32-bit in Python and bf16 in audio.cpp; Pocket TTS in Python and Fish Audio S1-mini at bf16.
What this test can't tell you
- Which one sounds best. That's for your ears, so every clip we're allowed to share is here.
- It's one Mac and one run per clip. Times move a little between runs, and a Mac with more memory will run the big models better.
- The copied voices start from synthetic ones. A real person's recording might clone differently.
Thanks
Every voice here is someone's open work. The models we tested, with their makers and licenses:
- Kokoro 82M v1.1-zh and v1.0 by hexgrad, Apache-2.0
- Qwen3-TTS 1.7B, 0.6B and 1.7B CustomVoice by the Qwen team at Alibaba, Apache-2.0
- VoxCPM2 by OpenBMB, Apache-2.0
- VibeVoice 1.5B by Microsoft, MIT
- Chatterbox and Chatterbox Turbo by Resemble AI, MIT
- Pocket TTS by Kyutai, CC-BY-4.0
- Supertonic 3 by Supertone, OpenRAIL
- OmniVoice by k2-fsa, CC-BY-NC 4.0
- Fish Audio S2 Pro and S1-mini by Fish Audio, non-commercial licenses
- Apple's system voices, built into macOS
And the tools that ran them: audio.cpp by ShugoAI (Apache-2.0), mlx-audio (MIT), hexgrad's kokoro (Apache-2.0), and Whisper large v3 by OpenAI (MIT), which listened back to every clip.