ARTIFICIAL INTELLIGENCE
Dialogue audio and the problem of a human voice
Dia2’s streaming dialogue and Kyutai’s 2026 Pocket TTS expose the tradeoffs between expressive speech, latency, stable voices, and interactive listening.
Research checked
Newest cited publication
A real voice spectrogram displays how speech energy changes across time and frequency, providing context for generated conversational audio.
Dvortygirl and Mysid · Image source · CC BY-SA 3.0
A synthetic voice can pronounce every word correctly and still feel difficult to talk with. It may begin too late, pause in the wrong place, or continue speaking after an interruption. Open dialogue-audio systems make those differences easier to study. Nari Labs' Dia2, released in November 2025, is a useful streaming reference; Kyutai's January 2026 Pocket TTS provides a newer example of speech synthesis aimed at modest hardware. As of October 18, 2026, the substantive question is how these components behave in conversation, rather than whether a curated sentence sounds impressively human.
Different releases answer different questions
Dia2's official repository provides one-billion- and two-billion-parameter checkpoints for English dialogue generation, with a documented context limit of about two minutes. It describes generating audio before the complete input text is available and conditioning on conversational audio. These are project capabilities, not a universal measurement of response latency on every machine. The model card also distinguishes the available model from server features still listed as upcoming.
Kyutai's current project page identifies Pocket TTS as a January 2026 release with one hundred million parameters, designed for real-time CPU use, and records additional languages introduced in April. That is a different deployment target from a larger accelerator-oriented system. Parameter count alone cannot tell a user which will sound better or respond sooner. The answer depends on the language, voice, device, text delivery pattern, and whether timing is measured from the first word or from the completed sentence.
Why the waveform needs an intermediate language
A speech waveform contains many samples for each second of audio. Directly treating every sample as an independent prediction target would make long conversations expensive. Neural codecs instead compress sound into a lower-rate representation from which a decoder reconstructs audio. Dia2 documents using Kyutai's Mimi codec, while the earlier Dia release uses a different audio-generation stack. They are separate implementations, so details from one should not be casually attributed to the other.
Moshi's research and implementation provide background for streaming audio tokens. An original way to think about the bottleneck is a musical score with richer instructions: enough information must survive to preserve what was said, how it was said, and who appeared to say it. Compression may preserve intelligibility while losing subtle breath or timing cues. Better text modeling cannot recover information that the acoustic representation systematically discards. Conversely, a beautiful codec cannot correct a sentence the language component misunderstood.
Streaming is a property of the whole path
Dia2's repository describes incremental text input and provides an inference interface, while separately listing a dedicated TTS server as upcoming. This is a useful distinction. A model can support incremental generation without a particular hosted application delivering low-latency audio to a listener. Loading weights, preparing audio prefixes, scheduling accelerator work, encoding the stream, and network buffering can all intervene.
An original timing example makes the point. Suppose text becomes available after 200 milliseconds, model preparation takes another 150, and the player waits for a 300-millisecond buffer. Even an immediately generated first audio fragment cannot be heard before those delays are paid. Reducing one component by fifty milliseconds helps, but does not remove the rest. An honest benchmark should report time to first audible output and sustained generation speed separately. It should also distinguish a warm process from a cold start and specify the actual processor used.
Consistency requires more than a speaker tag
The original Dia project supports dialogue speaker tags and audio conditioning, while warning that voices can vary across generations. Dia2 similarly recommends prefixes or fine-tuning for stable output. A tag establishes a role in the script; it does not by itself prove a persistent acoustic identity. The distinction matters for any application that joins separately generated passages into a longer conversation.
Imagine generating the same fictional narrator in twelve short segments. Each clip might sound natural in isolation while their sequence reveals changes in accent, pitch, pacing, or microphone character. A useful independent test would compare segment boundaries and ask listeners whether the speaker still seems consistent. Similarity should be assessed separately from intelligibility and expressive quality. For a real person's reference recording, permission and clear labeling are part of a responsible product workflow. A voice model's ability to reproduce acoustic traits is not evidence that the person endorsed the resulting words.
Listening and interruption are separate capabilities
Moshi's September 2024 preprint studies a speech-text framework for real-time, full-duplex dialogue. Full duplex means that listening and speaking can overlap, rather than forcing every exchange into a complete user turn followed by a complete system turn. That background helps locate a speech synthesizer within a larger conversation system. Text-to-speech alone does not supply reliable interruption handling, intent recognition, or decisions about when to take a turn.
For an original evaluation, ask a user to interrupt a generated explanation with a short correction. Measure whether audio playback stops, whether already queued content is discarded, and whether the next response incorporates the correction. A system can have excellent speech quality while failing all three. Another test could introduce a long pause inside an unfinished sentence and check whether the assistant barges in too early. These are coordination failures experienced by the listener, even if each generated word is acoustically flawless.
Test the experience that survives repeated use
The openly available Dia2 artifacts let researchers inspect model usage and compare settings, while Kyutai's 2026 release broadens the range of hardware on which experiments can be attempted. Neither project page establishes a single winner across all dialogue tasks. Claims should be tied to a language, context length, hardware configuration, and evaluation protocol, with sample selection disclosed.
An independent test set should contain names, numbers, ambiguous punctuation, emotional shifts, and conversations approaching the documented context limit. Use complete outputs, including awkward endings, rather than trimming every sample to its strongest moment. Ask listeners to score word accuracy, conversational timing, voice continuity, and fatigue after several minutes. Those measurements can disagree, and that disagreement is useful. It reveals whether the next improvement should target acoustic fidelity, input handling, or the conversation manager. The goal is a voice that makes interaction easier over time, with its synthetic nature clear, rather than a single clip optimized to fool a listener.
An unfinished sentence contains unfinished information
Streaming introduces a tradeoff that can be demonstrated without a benchmark leaderboard. Read the phrase I did not say she stole the money aloud several times, stressing a different word each time. The words remain unchanged while the implied correction changes. A synthesizer receiving only the first few words must begin choosing rhythm and emphasis before later context resolves the intended meaning. Waiting for more text can improve planning while increasing the perceived delay.
An original test could deliver the same sentence in complete form and in progressively smaller chunks, keeping the voice and device fixed. Listeners would judge both responsiveness and whether the prosody matches the final meaning. Add punctuation late in another condition to expose the cost of committing too soon. The ideal amount of look-ahead will depend on the application: a narrated article can tolerate preparation that a rapid exchange cannot. Treating this as a measurable tradeoff helps explain why immediate sound and natural speech are related goals, rather than interchangeable definitions of success.
Sources & further reading
Primary papers, project documentation, and research releases used in this article. Publication dates below refer to the sources themselves.
- Dia2Nari LabsPublished Checked
- Dia2-2BNari LabsChecked
- DiaNari LabsChecked
- Text-to-speechKyutaiChecked
- Moshi: a speech-text foundation model for real-time dialoguearXivPublished Checked
- Pocket TTS: a high-quality TTS with voice cloning that runs on CPUKyutaiPublished Checked
FOLLOW THE QUESTION
Study engineering foundations