r/LocalLLaMA

Qwen3-TTS voice cloning is now in mainline llama.cpp — the old demo finally became real support

2026-08-05 · 07:47 UTC ·u/BTA_Labs ·1 min read

Brief

Qwen3-TTS voice cloning was merged into llama.cpp on 2026-08-04, enabling local use of Qwen3-TTS-12Hz-1.7B-Base GGUF (WAV/MP3 speaker references, ~3s cloning claim, multi-language output) via the llama-tts binary. The poster notes it currently targets only the Base model, the /tts server is a draft, there are breaking changes, and no independent CPU/Metal/CUDA/ROCm benchmarks or voice-similarity comparisons versus specialized ports have appeared yet.

Why it matters

On 2026-08-04, a new implementation of Qwen3-TTS was merged into mainline llama.cpp (PR #26254), adding Qwen3-TTS-12Hz-1.7B-Base in GGUF with speaker MP3/WAV reference support and language support for English, Chinese, German, Italian, Spanish, French, Portuguese, Russian, Japanese and Korean; audio is generated via the llama-tts binary (example: `llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF -p "Hello..." --tts-lang en --tts-speaker-file speaker.mp3 --output out.wav`).

Key details

  • Significant limitations and open questions: the merged code targets only the 1.7B Base (not CustomVoice/VoiceDesign), uses llama-tts (the /tts server endpoint is still a draft PR #26603), includes a breaking change to the llama-tts binary, and lacks independent comparisons vs specialized ports (qwen3-tts.cpp, audio.cpp). The OP requests cross-platform benchmarks (CPU, Apple Metal, CUDA, ROCm) measuring real-time factor, peak RAM/VRAM, voice similarity, long-form stability and time to first audio; commenters expect specialized C++ ports may still be faster while llama.cpp improves portability and integration.
Source evidence

People may remember the Qwen3-TTS llama.cpp demo from a few months ago. That PR said it probably wouldn’t be merged because llama.cpp was missing some of the graph and API pieces it needed.
A new implementation was merged into master yesterday.
What works now:
- Qwen3-TTS-12Hz-1.7B-Base in GGUF
- WAV or MP3 files as the speaker reference
- English, Chinese, German, Italian, Spanish, French, Portuguese, Russian, Japanese and Korean
- Audio generation through the llama-tts binary
Example:
llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF \
-p "Hello, this is running locally." \
--tts-lang en \
--tts-speaker-file speaker.mp3 \
--output out.wav
Qwen describes the Base model as capable of cloning a voice from around three seconds of reference audio. I haven’t seen an independent test yet showing whether the llama.cpp version matches the original PyTorch implementation in voice similarity or stability.
The interesting part is not that Qwen3-TTS can run locally. Dedicated C++ implementations already existed. It is that voice cloning is now part of mainline llama.cpp, which should make it much easier to add local speech output to projects already built around that runtime.
There are still some important limitations:
- The merged implementation currently uses llama-tts
- The /tts server endpoint is still a draft PR
- It only targets the 1.7B Base model, not CustomVoice or VoiceDesign
- There are no proper comparisons yet against qwen3-tts.cpp or audio.cpp
- The update includes a breaking change to the existing llama-tts binary
The comparison I’d like to see is one identical three-second reference clip and one identical paragraph tested across CPU, Metal, CUDA and ROCm, with:
- Real-time factor
- Peak RAM and VRAM
- Voice similarity
- Long-form stability
- Time until the first audio
The specialized ports may still win on speed, while llama.cpp may win on portability and integration.
Has anyone updated and tested it yet? M-series Mac and CPU-only results would be especially useful.
Source:
https://github.com/ggml-org/llama.cpp/pull/26254
Draft server endpoint:
https://github.com/ggml-org/llama.cpp/pull/26603

Link: https://i.redd.it/kxag5u5ehihh1.png

Subreddit: r/LocalLLaMA