A confirmed vocoder bug.
Two unresolved quality issues.

We independently recomputed WER and CER for all 1,088 records of CosyVoice, Llasa and Dia. The published corpus calculations match. We then compared generation and waveform decoders against independent implementations.

Which CosyVoice?

FunAudioLLM/Fun-CosyVoice3-0.5B-2512: CosyVoice 3, 0.5B, December 2025 base checkpoint, llm.pt (not RL). Revision 29e01c4e8d000f4bcd70751be16fa94bf3d85a18.

CosyVoice: distortion introduced by HiFT

The native decoder clamped log magnitude before exponentiation and used the wrong final LeakyReLU slope. Both operations now match the publisher. Across all archived recordings, 52.04% of waveform samples reached the internal ±0.99 limiter. The existing clipping diagnostic used ±0.999, so it missed this saturation.

Three publisher-parity regression cases failed before the repair and passed after it. A controlled eight-text ablation kept LM tokens, mel features and RNG state identical:

2.30% is a diagnostic result, not the corrected full-split score. Examples were selected across WER strata, including extreme failures. Full resynthesis uses the same 1,088 texts and original generation/ASR configuration.

Llasa: sampling alone did not resolve the errors

The native default lacked the publisher example’s inherited top_k=50. An eight-text sampling ablation remained poor. Four additional texts generated with the independent Transformers FP32 LM and publisher settings had 61.40% diagnostic WER.

Eight FP32 prefix comparisons matched closely (maximum logit difference 7.63e-5). The legacy publisher XCodec2 decoder, loaded with its original checkpoint, matched the native decoder closely on the same speech codes; latent values matched exactly. A substantial decoder discrepancy does not explain these tested failures. Voice conditioning, configuration and ASR behavior remain possible factors; the full score stays under review.

Dia: failed speech and exaggerated ASR output

Three native/Transformers comparisons used the same checkpoint. Initial logits matched exactly; the first two generated outputs matched, while the third diverged later. Both implementations failed on the two selected bad cases. An independent Transformers DAC decoder check also produced closely matching waveforms. These bounded tests did not establish a port defect.

The full run has 183 empty transcripts. For one six-word target, Whisper inserted 185 extra words, producing 3,183.33% WER. Enabling VAD diagnostically returned an empty transcript and 100% WER. ASR repetition amplifies some failures, but removing it does not restore the target speech. The archived no-VAD scoring protocol is preserved.

Short inputs were more difficult: 187.55% WER below 40 characters, 71.46% at 40–79, and 30.60% at 80+. Dia’s publisher warns about very short text and recommends audio conditioning. This supports investigating protocol fit; it does not establish the complete cause.

Inspect the evidence

Full methods, revisions and limitations · All-record metric verification · CosyVoice ablation · Llasa independent LM results · Llasa codec parity · Dia transcripts and VAD diagnostic · Dia codec parity

Recognition: pinned Whisper-large-v3, English, CUDA FP16, beam 5, temperature 0, no VAD or previous-text conditioning; whisper_english normalization. VAD experiments are labeled separately. This benchmark uses fixed provider voices/references, not the official per-prompt speaker-identity protocol.