Instructions to use hypaai/Hypa-Orpheus-3b-TTS-VC with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hypaai/Hypa-Orpheus-3b-TTS-VC with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="hypaai/Hypa-Orpheus-3b-TTS-VC")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("hypaai/Hypa-Orpheus-3b-TTS-VC")
model = AutoModelForCausalLM.from_pretrained("hypaai/Hypa-Orpheus-3b-TTS-VC", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use hypaai/Hypa-Orpheus-3b-TTS-VC with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hypaai/Hypa-Orpheus-3b-TTS-VC"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hypaai/Hypa-Orpheus-3b-TTS-VC",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/hypaai/Hypa-Orpheus-3b-TTS-VC

SGLang

How to use hypaai/Hypa-Orpheus-3b-TTS-VC with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hypaai/Hypa-Orpheus-3b-TTS-VC" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hypaai/Hypa-Orpheus-3b-TTS-VC",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hypaai/Hypa-Orpheus-3b-TTS-VC" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hypaai/Hypa-Orpheus-3b-TTS-VC",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use hypaai/Hypa-Orpheus-3b-TTS-VC with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for hypaai/Hypa-Orpheus-3b-TTS-VC to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for hypaai/Hypa-Orpheus-3b-TTS-VC to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for hypaai/Hypa-Orpheus-3b-TTS-VC to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="hypaai/Hypa-Orpheus-3b-TTS-VC",
    max_seq_length=2048,
)

Docker Model Runner
How to use hypaai/Hypa-Orpheus-3b-TTS-VC with Docker Model Runner:
```
docker model run hf.co/hypaai/Hypa-Orpheus-3b-TTS-VC
```

An open multilingual text-to-speech and zero-shot voice-cloning model for underrepresented languages, built on Orpheus 3B and trained on Hypa-Voices.

Model Description

Hypa-Orpheus 3B (hypaai/Hypa-Orpheus-3b-TTS-VC) is a LoRA-merged 16-bit bfloat16 checkpoint produced by Hypa Intelligence. It is Step 3a of our Hypa-Whispering-Llama roadmap: an independent multilingual voicebox that turns text (and optional reference speech) into 24 kHz waveform output via discrete SNAC audio tokens.

Built by fine-tuning Orpheus 3B on the full Hypa-Voices corpus, the model unifies four speech capabilities as a single next-token prediction problem:

Capability	Description
Multilingual TTS	Generate speech from text in a chosen language and named speaker voice.
Translation-conditioned TTS	Synthesize spoken audio from semantically equivalent text in another language.
Voice cloning (VC)	Reproduce a reference speaker's voice on new text from a single clip.
Cross-lingual voice cloning	Preserve speaker identity while generating speech in a different language.

Collection: hypaai/hypa-orpheus

This release documents the two primary artifacts in the collection:

Repository	Format	Best for
`hypaai/Hypa-Orpheus-3b-TTS-VC` (this repo)	Merged 16-bit weights	Inference, vLLM serving, production deployment
`hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs`	LoRA adapters + TensorBoard	Adapter stacking, continued fine-tuning, training metrics

Property	Value
Base model	`unsloth/orpheus-3b-0.1-ft`
Method	4-bit QLoRA (r=512, α=512) via Unsloth, merged to bf16
Trainable parameters	778M / 4.08B (19.07%)
Training data	3.73M constructed examples (2.98M TTS + 756K VC) from Hypa-Voices
Corpus scale	8,442 hours, 22 languages
Shipped checkpoint	Step 44,334 (validation-best)
Sequence length (train)	2,048 tokens (~25 s of speech at ~82 tokens/sec)
Audio codec	SNAC @ 24 kHz
Compute	1× NVIDIA A100 80 GB PCIe (RunPod)
License	Apache 2.0

Hypa-Orpheus powers pronunciation in Hypa Dictionary and spoken output in Hypa Translate, supports experimentation in Hypa Labs, and is being integrated into Hypa Keyboard read-aloud. It also serves as the cascaded speech fallback in our longer-term plan to fuse hearing and speech inside Hypa-Whispering-Llama:

User Speech → Hypa-Whisper → Hypa-Llama → Response Text → Hypa-Orpheus → Response Speech

Quick Start

Load the merged model (Transformers)

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "hypaai/Hypa-Orpheus-3b-TTS-VC"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

Orpheus generation requires SNAC token decoding and training-identical control-token prompts. For a complete TTS or voice-cloning pipeline (prompt assembly, SNAC encode/decode, frame deduplication, and WAV output), see the reference handler.py in this repository and the Orpheus TTS project.

vLLM serving

This repository includes a production vLLM inference handler (handler.py) with backward-compatible API support for legacy Hypa Orpheus clients. Serve with:

vllm serve hypaai/Hypa-Orpheus-3b-TTS-VC --dtype bfloat16 --max-model-len 4096

Use bfloat16 at inference to match training numerics.

Load LoRA adapters instead

To continue fine-tuning or stack adapters, use the companion repository:

from peft import PeftModel
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/orpheus-3b-0.1-ft",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)
model = PeftModel.from_pretrained(model, "hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs")

See hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs for TensorBoard logs and adapter details.

Prompt Format

Hypa-Orpheus treats speech as causal language modeling over interleaved SNAC tokens. Training and inference must share the same prompt structure.

Text prompts

transcribe (same-language TTS): {speaker}: {text}
translate (translation-conditioned TTS): {speaker} - {Language}: {text}

In translate rows from Hypa-Voices, speech is in src_lang and text is in tgt_lang. The language tag in the prompt refers to the text side (tgt_lang).

Control tokens

Orpheus wraps content in role and speech markers. Two rules are fixed at train and serve time:

<BOS> (<|begin_of_text|>) appears once, at the start of the first human turn.
Every text span ends with <EOT> (<|eot_id|>), including the boundary between two texts in one human turn.

Vanilla TTS sequence:

[start_of_human] <BOS> {speaker}: {text} <EOT> [end_of_human]
[start_of_ai] [start_of_speech] <SNAC codes> [end_of_speech] [end_of_ai]

Voice cloning (continue-speaking layout):

[start_of_human] <BOS> {ref_text} <EOT> {target_text} <EOT> [end_of_human]
[start_of_ai] [start_of_speech] <ref SNAC codes> ...continues into target codes... [end_of_speech] [end_of_ai]

SNAC codes use Orpheus's 7-token-per-frame interleave (one coarse, two mid, four fine codes per frame), each offset into a distinct 4,096-token vocabulary slot above the base Llama vocabulary.

Languages Covered

Hypa-Orpheus was trained on 22 languages spanning AfroVoices studio recordings, African-accented English and French, and adapted public speech (including Common Voice-derived English and Pidgin):

Language	Language	Language	Language
Annang	Arabic	Ebira	Efik
Eggon	English	Fongbe	French
Hausa	Ibibio	Idoma	Igala
Igbo	Nupe	Pidgin	Portuguese
Spanish	Swahili	Tiv	Twi
Urhobo	Yoruba

Quality varies by language. Core AfroVoices languages received the densest supervision; tail languages (Fongbe, Twi, Urhobo, Ebira, and others) have thinner cross-lingual connectivity and should be validated by native listeners before production use.

Training Data

Training data comes from Hypa-Voices, our multilingual audio-text corpus:

Split	Records	Role
TTS source rows	2,979,208	Same-language transcription and cross-lingual translation pairs
VC constructed rows	755,760	Bidirectional same-speaker voice-cloning pairs
Total training rows	3,734,968	Mixed TTS + VC with modality-weighted interleaving

Every row pairs speech and text as transcribe (same language) or translate (cross-lingual). Named speaker IDs enable controllable, voice-aware synthesis rather than anonymous audio.

Offline SNAC precompute

The full corpus was normalized to fast-decoding FLAC, then SNAC-encoded offline into integer codes_list shards. Training reads only precomputed tokens, keeping the GPU compute-bound rather than blocked on audio decode or codec forward passes. A curated 8,800-record public subset is released in both FLAC (Hypa-Voices) and SNAC token form (Hypa-Voices-snac).

For corpus statistics (language tiers, speaker concentration, cross-lingual routes, and sequence-length filtering), see:

Hypa-Voices: analysis of the leading multilingual low-resource dataset

Training Procedure

Hyperparameter	Value
LoRA rank (r)	512
LoRA alpha (α)	512
LoRA dropout	0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Quantization	4-bit NF4 base, bf16 compute
Optimizer	AdamW 8-bit
Learning rate	5e-5
LR schedule	cosine, 3% warmup
Weight decay	0.01
Max grad norm	1.0
Per-device batch size	16
Gradient accumulation	6
Effective batch size	96
Sequence length	2,048
Prompt masking	Response-only (loss from `[start_of_speech]` onward)
Packing	Disabled
Checkpoint/eval interval	2,217 steps
Optimizer-step budget	44,334
Precision	bfloat16
Gradient checkpointing	Unsloth variant
Hardware	1× NVIDIA A100 80 GB PCIe (RunPod)
Random seed	3407

The step budget targets roughly one effective pass through the TTS distribution under weighted interleaving (~70% TTS / ~30% VC per drawn batch).

Training used Unsloth with 4-bit QLoRA. Labels use response-only masking: reference SNAC codes in voice-cloning prompts provide context but are not supervised targets.

Evaluation and Checkpoint Selection

Interactive training logs:

TensorBoard on Hugging Face

Metric	Value	Step
Initial training loss	4.743	1
Best training loss	3.549	26,604
Final training loss	3.560	42,123
Initial validation loss	3.870	2,217
Best validation loss (shipped)	3.513	44,334

Validation loss improved at every evaluation interval (19 intervals total). Training loss reached its minimum earlier (step 26,604); because checkpoint selection follows held-out performance, the released weights use the validation-best final step 44,334.

Loss and approximate perplexity (exp(loss)) measure uncertainty over the supervised target SNAC stream, not text-only SFT perplexity and not speech naturalness directly. Informal listening checks across held-out speakers and languages agreed with the scalar choice; formal WER/CER, speaker-similarity, and MOS benchmarks are planned for a future evaluation release.

For downstream use, we recommend the merged 16-bit weights in this repository (step 44,334).

Intended Use

Direct use cases:

Multilingual text-to-speech with named speaker control
Translation-conditioned speech synthesis (text in tgt_lang, speech reflecting src_lang semantics)
Zero-shot voice cloning from a short reference clip
Cross-lingual voice cloning when the reference and target languages differ
Cascaded speech output paired with Hypa-Llama or other text models

Downstream use:

Continued fine-tuning from the LoRA adapters in the companion repository
Adapter stacking for domain- or language-specific heads
Research on codec-token speech modeling and mixed-task TTS/VC training

Out-of-Scope and Limitations

Not a general chat model. Despite the Llama backbone, this checkpoint is trained for speech-token generation, not open-ended dialogue.
Prompt sensitivity. Missing <BOS>, duplicate BOS tokens, or omitted internal <EOT> markers put inference off-distribution and can corrupt audio.
Context ceiling. Training used 2,048 tokens (~25 s of speech). Longer outputs may degrade; voice-cloning rows with two clips hit the limit first.
Quality varies by language and speaker. Tail languages and speakers with few training hours may produce less stable output.
No published perceptual benchmark yet. Declining validation loss and informal listening support a stable fine-tune, but intelligibility, naturalness, and cloning fidelity require formal evaluation before strong product claims.
SNAC dependency. Inference requires the SNAC codec (hubertsiuzdak/snac_24khz) for encode/decode around the language model.

Bias, Risks, and Limitations

This model inherits biases from its base Orpheus weights and from Hypa-Voices, which is weighted toward AfroVoices studio material, English/French pivot translation routes, and adapted public speech. Generated speech should be reviewed for cultural appropriateness in customer-facing applications, especially for underrepresented languages with thinner training coverage.

Do not use this model unsupervised for applications affecting people's rights, health, finances, or wellbeing. Like all generative models, it can produce confident-sounding but incorrect speech.

Released Artifacts

Merged 16-bit model (this repo): hypaai/Hypa-Orpheus-3b-TTS-VC
LoRA adapters + TensorBoard: hypaai/Hypa-Orpheus-3b-TTS-VC-LoRAs
Base model: unsloth/orpheus-3b-0.1-ft
Training dataset (public subset): Hypa-Voices collection
Dataset analysis: Hypa-Voices statistical breakdown
Upstream projects: Orpheus TTS, SNAC, Unsloth
Related text model: Hypa-Llama collection

For notebooks, serving code, or contributions, contact chris@hypaintelligence.com or open an issue on Hugging Face.

Citation

If you use Hypa-Orpheus 3B or any of the related work, please cite:

@misc{hypaorpheus2026,
  title        = {Hypa-Orpheus 3B: Multilingual Text-to-Speech and Voice Cloning for Underrepresented Languages},
  author       = {{Hypa Intelligence and AfroVoices}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/hypaai/Hypa-Orpheus-3b-TTS-VC}},
  note         = {Apache 2.0 License. Collection: \url{https://huggingface.co/collections/hypaai/hypa-orpheus}}
}

If you use the Hypa-Voices training corpus, please also cite:

@misc{hypavoices2026,
  title={Hypa-Voices: Multilingual Low-Resource Audio-Text Dataset},
  author={Hypa Intelligence and AfroVoices},
  year={2026},
  howpublished={Hugging Face collection},
  url={https://huggingface.co/collections/hypaai/hypa-voices}
}

And the SNAC codec:

@inproceedings{siuzdak2024snac,
  title={SNAC: Multi-Scale Neural Audio Codec},
  author={Siuzdak, Hubert and Gr{\"o}tschla, Florian and Lanzend{\"o}rfer, Luca A.},
  booktitle={Audio Imagination: NeurIPS 2024 Workshop AI-Driven Speech, Music, and Sound Generation},
  year={2024}
}

License

Released under the Apache License 2.0. See the LICENSE file in this repository for full details.

Acknowledgments

Canopy Labs, for Orpheus TTS and for treating speech generation as language modeling over discrete audio tokens.
Meta AI / FAIR, for the Llama family and the open-weights ecosystem Orpheus builds on.
Hubert Siuzdak and collaborators, for SNAC and for open-sourcing the codec.
Unsloth, for efficient 4-bit QLoRA and Orpheus integration.
RunPod, for reliable compute during the production fine-tune.
Mozilla and the Common Voice community, for open speech data used in parts of Hypa-Voices.
AfroVoices, for the steadfast curation of high-quality recordings across underrepresented languages.
Hypa Intelligence Research (HaIR), for our commitment to breaking the language barrier.

Contact and Contributions

For questions, issues, or contributions, please open an issue in this repository or contact chris@hypaintelligence.com. Contributions are welcome.

Closing Remarks

By releasing Hypa-Orpheus, we hope to give researchers and developers practical access to multilingual speech synthesis and voice cloning for underrepresented languages, and to make the data path, training recipe, and checkpoint-selection process transparent from the start.

At Hypa Intelligence, we believe that for AI to be truly aligned with humanity, it must understand and represent all of us, not just a select few. The first step toward this goal is solving the challenge of multilingualism by breaking the language barrier.

Hypa Intelligence remains steadfast in its mission to accelerate the advent of AGI and ASI and to ensure their benefits are globally distributed.

AfroVoices, a subsidiary of Hypa AI, is dedicated to amplifying African voices, languages, and cultures in the intelligence age. Focused on bridging the digital representation gap, AfroVoices curates datasets and resources for African languages, promoting inclusivity and cultural appreciation in AI technologies.

Hypa Intelligence • Website • Hugging Face • Updates

Multilingualism is not a feature. It is a prerequisite for AI that represents all of us.