Pipecat Text to Speech: Fish Audio Pipeline (2026)

Pipecat text to speech can use Fish Audio through FishAudioTTSService, a WebSocket integration that streams generated speech into a Pipecat pipeline. The service supports custom voice reference IDs, model selection and runtime prosody controls.

For production, the important work is not the import statement. It is placing TTS correctly after the LLM, choosing a transport-compatible audio format and measuring time to first audible speech through the real call path.

Use the AI voice agent guide for the surrounding speech loop and the text to speech API guide for credentials, retries and request handling.

Does Pipecat support Fish Audio text to speech?

Yes. Pipecat ships FishAudioTTSService, which connects to Fish Audio's real-time WebSocket API and produces audio frames inside a Pipecat pipeline.

The service accepts a Fish Audio API key, a voice reference ID and settings for model, latency, prosody and chunking. It is a direct integration, so you manage Fish Audio billing and credentials while Pipecat manages frame flow, interruption and transport.

How do you install and configure FishAudioTTSService?

Install the Fish Audio extra, expose FISH_API_KEY through a server-side environment variable, and construct the service with a voice reference ID. The default model is s2.1-pro.

import os

from pipecat.services.fish.tts import FishAudioTTSService

tts = FishAudioTTSService(
    api_key=os.getenv("FISH_API_KEY"),
    settings=FishAudioTTSService.Settings(
        voice="your-voice-reference-id",
        model="s2.1-pro",
        latency="balanced",
        prosody_speed=1.0,
        prosody_volume=0,
    ),
)

Use the current Pipecat package and service settings rather than copying a complete production agent from an old tutorial. The project's supported-services index is the authoritative list of integrations.

Where should TTS sit in a Pipecat pipeline?

Place TTS after the LLM and before transport output. The LLM emits text frames, TTS turns them into audio, and the output transport sends that audio to the user. The assistant context aggregator should also see the spoken text frames so the conversation history matches what the caller heard.

pipeline = Pipeline([
    transport.input(),
    stt,
    context_aggregator.user(),
    llm,
    tts,
    transport.output(),
    context_aggregator.assistant(),
])

This ordering comes from Pipecat's TTS pipeline guidance. Moving TTS after the assistant aggregator can make the model believe unspoken text entered the conversation, while omitting the output frame can delay playback.

Which audio format should you use?

Use PCM for the normal Pipecat transport path unless you deliberately decode another format. FishAudioTTSService supports PCM, Opus, MP3 and WAV, but Pipecat transports play raw PCM audio frames. Requesting MP3 without applying a decoder downstream creates noise or silence.

Confirm the pipeline sample rate before changing TTS output settings. A matching sample rate avoids unnecessary resampling, while telephone transports may require a different rate from browser audio.

How do you tune Fish Audio latency in Pipecat?

Start with the balanced latency setting, then compare low and normal using the same scripts and transport. Lower latency can trade away some delivery quality, while quality-first generation can make turn-taking feel slow.

Text aggregation is the second control. Pipecat can wait for complete sentences or begin synthesis from earlier token boundaries. Sentence mode can produce a more natural read; earlier synthesis may improve responsiveness but risks fragments and awkward prosody.

Chunk length and minimum chunk length are the third. They determine how long text is grouped before Fish Audio begins synthesis. Tune these values with the prompt's typical answer length rather than only with one-word greetings.

The LiveKit text to speech guide covers the same latency and model tradeoffs for LiveKit Agents, while the real-time text to speech guide explains how STT, LLM, TTS and playback budgets combine.

How do prosody settings affect a voice-agent reply?

Speed and volume are useful when a single voice must serve different contexts, such as a calm answer and an urgent confirmation. Keep them close to the voice's natural delivery unless the role genuinely changes.

Text normalization improves numbers, dates and abbreviations in supported languages. It does not guarantee the pronunciation of product names, account identifiers or mixed-language terms. Test the exact strings used by the agent and add application-level pronunciation rules where Pipecat supports them.

Pipecat reports that changing a service setting through TTSUpdateSettingsFrame reconnects the Fish Audio WebSocket. Avoid changing speech settings on every turn, because reconnecting during a conversation adds avoidable latency.

How should you handle interruption and fallback?

Let Pipecat own interruption. When the caller speaks, the pipeline should cancel in-flight LLM and TTS work, emit the correct speech boundaries and preserve a clean transcript. Do not simulate barge-in only by discarding audio on the client.

Configure a fallback TTS service with a voice that preserves the same language, role and approximate pace. Alert when fallback usage rises. The Retell TTS fallback guide and Vapi text to speech guide show why fallback belongs to the agent platform rather than the prompt.

How should you test a Pipecat and Fish Audio agent?

Measure p50 and p95 from the caller's final syllable to the first audible reply. Record STT finalization, LLM first token, Fish Audio time to first audio, transport network and client playback as separate timestamps.

Run short answers, long explanations, interruptions and provider failures. Include names, prices, dates, account identifiers and escalation language. Review one telephone-quality recording because browser speakers hide codec and handset problems.

Keep the voice ID, model, latency mode and prosody settings under version control. A model or voice change is a product change, not a cosmetic deployment detail.

Key takeaways

• Pipecat provides a native Fish Audio WebSocket service with custom voices and prosody controls.
• Put TTS after the LLM, before transport output and before the assistant context aggregator.
• Use PCM unless the transport has an explicit decoder for another format.
• Tune latency, text aggregation and chunking together, then measure p95 through the real transport.
• Test interruption, pronunciation and fallback before trusting a successful sample generation.

Frequently asked questions

Does Pipecat support Fish Audio text to speech?

Yes. Pipecat provides FishAudioTTSService, a real-time WebSocket integration for Fish Audio with custom voice models and prosody controls.

Where does FishAudioTTSService belong in a Pipecat pipeline?

Place TTS after the LLM and before transport output. Keep the assistant context aggregator after the spoken-text frames so conversation history reflects what the caller actually heard.

Which audio format should Pipecat request from Fish Audio?

Use PCM unless your transport decodes another format. Pipecat transports expect raw PCM audio frames, while MP3, WAV and Opus require explicit decoding downstream.

How do I lower Fish Audio latency in Pipecat?

Use a low or balanced latency setting, tune text aggregation and sentence boundaries, benchmark from the real transport, and measure p95 time to first audible audio rather than isolated TTS response time.

Can Pipecat use a custom Fish Audio voice?

Yes. Configure FishAudioTTSService with your Fish Audio API key and voice reference ID, then keep the voice and consent record with the agent version that uses it.