Long-form text-to-speech isn’t just about sounding natural for a sentence — it’s about staying consistent for chapters, managing latency, and avoiding those jarring seams that break immersion. At AI Tech Inspire, we spotted a release that leans into that challenge with a couple of gutsy design choices: character-level tokenization and a timeline-aware chunking scheme. Meet TontaubeV1, an open-weight 2.9B-parameter TTS model built for expressive narration and on-device speed.
Snapshot: the most important facts
- TontaubeV1 is a 2.9B-parameter open-weight TTS model focused on expressive speech, long-form generation/narration, and low-latency local inference.
- Primarily aimed at English and German; trained on 7 languages and ~200k hours of audio; supports zero-shot voice cloning from up to 1 minute of reference audio.
- Built on DualCodec, a multi-codebook discrete audio codec; semantic codebook model initialized from a Qwen 3 1.7B checkpoint.
- Uses character-level tokenization (forced via the Qwen tokenizer) instead of BPE; reported to preserve language understanding while improving TTS robustness.
- Employs a chunking and logical position-ID scheme aligning text and audio on the same timeline (text: 1 char/position; audio: 12.5 fps), with boundary markers and reserved positions to prevent leakage.
- Streaming joins chunks using overlapping DualCodec windows, re-encoding into a VibeVoice acoustic space and decoding with a shared causal state to reduce audible seams.
- Hardware: needs ≥24 GB VRAM (low-VRAM/balanced) or 32 GB (high-throughput). Current VRAM use is heavily influenced by vLLM KV-cache reservations and multi-engine serving.
- Planned: quantized versions for smaller memory footprints and on-device use, plus fine-tuning support.
- Benchmark: in a 400-passage LLM-as-a-judge audiobook test, TontaubeV1’s prosody scored 50.1% against ElevenLabs Flash v2.5 and was preferred over Fish Audio S2 Pro, Gradium, and Cartesia Sonic 3 (with caveats noted; human listening remains the gold standard). Planned submissions to public TTS arenas.
- Resources: Hugging Face model page, HF demo, inference code (GitHub), and a technical report.
Why character-level tokens might be the most interesting choice here
Most LLM-augmented TTS systems inherit a subword/BPE tokenizer from models in the GPT-style family. That is convenient — until long-form speech encounters rare token sequences or quirky punctuation. TontaubeV1 forces the Qwen tokenizer to output individual characters for spoken text. The payoff is a simpler mapping between what’s written and how it sounds: syllables and short character sequences are core to speech, whereas BPE tokens can lump together patterns that the TTS training set barely covers.
The developers report that when Qwen was “forced” into this character-output mode, it still answered questions correctly — a strong hint that language understanding survives the change in representation. The tradeoff is longer token sequences, but for TTS, context length is less punishing than in general LLM reasoning. Chunking mitigates the rest.
Key takeaway: Character-level tokenization reduces out-of-distribution token combos and makes pronunciation control more predictable — handy when text includes names, codes, or punctuation-heavy passages.
For engineers who’ve wrestled with odd prosody around symbols (like “v2.5—alpha/RC1”), this could be a practical advantage. If you’ve ever had to rewrite text to “steer” a TTS engine, the char-level path may reduce those contortions.
Chunking on a shared timeline: keeping long passages coherent
TontaubeV1 doesn’t just split text into chunks; it teaches the model that text and audio co-exist on the same timeline. During training, tokens for text, semantic audio, and acoustic codebooks are serialized physically but assigned logical position IDs so that elements representing the same moment in time align — text advances one position per character; audio advances at 12.5 frames per second.
Because speaking rates drift, TontaubeV1 adds paired split markers and reserves an extra 25 character positions at each boundary. This prevents positions from “leaking” into the next chunk, while keeping positions monotonic and roughly linear across the whole passage. The semantic model sees the previous chunk’s text/audio, the current text, and a short lookahead into the next text, then slides its window forward.
Why it matters: long-form narration often degrades as systems lose local context or fail to align timing. TontaubeV1’s position scheme tries to anchor text-to-audio alignment while containing memory growth — a pragmatic balance for audiobook chapters, tutorials, or podcasts.
Streaming without seams: an overlap-and-reencode trick
Chunked decoders can leave audible “clicks” or prosody discontinuities. TontaubeV1’s pipeline decodes overlapping windows with DualCodec, re-encodes them into a VibeVoice acoustic space, keeps the stable middles, and decodes with a shared causal VibeVoice state. The intent: let audio start streaming before the full passage is done, while masking boundaries between chunks.
Developers building live reading, dubbing, or interactive assistants can use this to reduce latency without sacrificing continuity. If you’ve ever had to stitch generated clips in post, the approach here hints at a cleaner real-time path.
Zero-shot cloning and multilingual training — with caveats
TontaubeV1 supports zero-shot voice cloning from up to 60 seconds of reference audio. It’s trained on seven languages but is primarily targeted at English and German for quality. That’s a compelling range for prototypes involving personal narrators, branded voices, or internal content tools.
Responsible use matters: secure consent for any speaker you clone, consider watermarking or disclosure policies, and keep an eye on organizational guidelines. Open-weight models make local experimentation easy — and also make governance your job.
Hardware, speed, and the path to smaller footprints
The current release needs GPUs with ≥24 GB VRAM (low-VRAM or balanced profiles) or 32 GB (high-throughput). A big chunk of that footprint comes from vLLM’s KV-cache reservations and a multi-engine serving setup tuned for concurrency and latency. If you’re deploying on-prem with PyTorch and CUDA, that’s achievable on prosumer GPUs, but not exactly ultralight.
The team plans quantized variants for smaller memory budgets and on-device scenarios, plus fine-tuning support. For many readers, that’s the inflection point: once quantized models land, laptop-class inference and edge use cases (assistive devices, kiosk narration) get a lot more realistic.
Benchmarks: promising signals, honest disclaimers
On a 400-passage LLM-as-a-judge audiobook test, TontaubeV1’s prosody scored 50.1% against ElevenLabs Flash v2.5 and was preferred over Fish Audio S2 Pro, Gradium, and Cartesia Sonic 3. The methodology details and confidence intervals live in the technical report, with the sensible caveat that human listening is the gold standard. Planned submissions to public TTS arenas should help triangulate quality.
Our read: treat this as a green light to evaluate — especially for long-form — rather than a definitive leaderboard shakeup. Prosody and fatigue-free delivery over hours are where the model is positioning itself.
Practical ways developers might use TontaubeV1
- Audiobooks and courseware: consistent character voices across chapters with fewer join artifacts.
- Product tutorials and docs: fine-grained control over punctuation, acronyms, and code snippets via character-level mapping.
- Assistive reading and offline agents: open-weight and local inference for privacy-sensitive deployments.
- Streaming narration: overlap-and-reencode reduces seams while you pipe audio progressively; use Ctrl+C to stop a long read during dev loops.
Getting started
If you want to kick the tires in minutes, start with the Hugging Face demo space. For local work, check the inference code on GitHub, and review the technical report for the chunking layout and position-ID scheme. The model page on Hugging Face lists profiles and VRAM guidance.
Pro tip: if your text includes rare symbols, try feeding the raw characters rather than “helpfully” expanding them — the character-level approach is designed to cope without you sanitizing inputs. And if you’re measuring latency, isolate vLLM cache behavior from codec decoding to see what’s actually gating throughput.
Questions worth exploring
- How well does character-level tokenization generalize to languages with complex scripts or rich diacritics beyond the two primary targets?
- Can the position scheme be extended to handle singing, where timing deviates sharply from normal speech rates?
- What’s the tradeoff curve of quantization (e.g., 4–8 bit) on prosody and seam masking in long-form streaming?
- Would a lightweight pronunciation lexicon on top of char-level tokens further reduce edge-case mispronunciations?
Bottom line: if your work involves long-form speech generation and you’ve been burned by tokenization oddities or chunking artifacts, TontaubeV1 offers a fresh angle. The open-weight release, the focus on expressive narration, and the frank discussion of tradeoffs make it a practical system to evaluate. As always, the proof is in the listening — but this is one model where the design decisions alone should make developers pause and think, “We should try that.”
Recommended Resources
As an Amazon Associate, I earn from qualifying purchases.
