What is it?
What it is
A 0.6B-parameter multilingual text-to-speech model enabling zero-shot voice cloning with support for 11 languages, leveraging a DualAR architecture for efficient speech generation.
Why it exists
To provide an efficient, multilingual TTS solution with zero-shot voice cloning capabilities, reducing the need for extensive training data while maintaining high-quality speech synthesis.
Who should use it
Machine learning engineersAI researchersSpeech synthesis developersApplications requiring zero-shot voice cloningDevelopers needing multilingual TTS support
Who should avoid it
Users without a CUDA-capable GPUUsers requiring extensive dialect support (limited in preview)Users needing high-fidelity audio for long, noisy reference clipsUsers seeking production-ready stability for all languages
How it works
A quick walkthrough in plain English
How Audio8_TTS works
Step 1 of 3
You interact with it
Open Audio8_TTS, send a request, or connect it to your stack.
Features
Advantages
- Small model size (0.6B parameters) with competitive performance against larger models
- Efficient CPU deployment (ONNX INT4 version uses ~1 GiB memory on Apple M2)
- Multilingual support in preview release
- Zero-shot voice cloning capability
- Flexible deployment options (PyTorch, ONNX, SGLang Omni)
- Permissive Apache 2.0 license
- Strong performance in benchmarks (Seed-TTS: best English WER; competitive Chinese CER; CV3 multilingual: competitive error rates)
Disadvantages
- Preview release has limited language coverage (only 11 languages recommended)
- Performance may degrade with very long, noisy, or inaccurate reference audio
- Potential for misuse in voice cloning (impersonation, misinformation) requires responsible use
- Evaluation comparisons may not be strictly matched due to different normalizers and evaluators
- Chinese dialect support is planned for future releases (not in preview)
Installation
native
python3 -m venv.venv source.venv/bin/activate pip install -r requirements.txt
FAQ
How do I install Audio8_TTS and run inference with a reference voice?
Create a virtual environment, install dependencies, download the checkpoint, and run the inference script: ```bash python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt # Download checkpoint to model/audio8_tts_0_6B_preview/ python audio8_tts_infer.py \ --text "Welcome to audio8_tts." \ --reference-audio examples/reference.wav \ --reference-text "Transcript of the reference recording." \ --output outputs/clone.wav ```
What is the format for batch inference and how do I run it?
Create a JSONL manifest where each line is a JSON object. Text, reference audio, and reference text are optional but must be provided together. Example: ```json {"id":"sample_001","text":"Target text","reference_audio":"audio/ref.wav","reference_text":"Reference transcript"} {"id":"sample_002","text":"Text without a reference voice"} ``` Run: ```bash python audio8_tts_infer.py \ --input-jsonl data/prompts.jsonl \ --output-dir outputs/batch \ --batch-size 2 ``` The script writes `manifest.jsonl` and `failures.jsonl` and skips existing WAV files unless `--overwrite` is used.
How can I deploy Audio8_TTS on a CPU using ONNX Runtime?
Download the INT4 ONNX model from Hugging Face and follow the guide in `onnx_runtime/README.md`. Steps: 1. Install dependencies: `pip install -r requirements-onnx.txt`. 2. Download the ONNX model: `hf download Audio8/Audio8-TTS-Preview-0.6B-ONNX-INT4 --local-dir onnx_model`. 3. Run the CLI inference or start the local web/HTTP service as described in the ONNX Runtime README. The CPU runtime uses FP16 activations and a lightweight codec, keeping memory usage around 1 GiB on an Apple M2.
What languages are supported by the Audio8_TTS Preview checkpoint?
The preview is optimized for 11 languages: - Cantonese (粤语) - Chinese (中文) - Dutch (荷兰语) - English (英语) - French (法语) - German (德语) - Italian (意大利语) - Japanese (日语) - Korean (韩语) - Polish (波兰语) - Spanish (西班牙语)
Featured in Videos
YouTube tutorials and walkthroughs for Audio8_TTS
Alternatives
Similar projects ranked by category, topics, and text overlap.
