Agora Conversational AI TTS: Fish Audio Guide (2026)

Agora Conversational AI can use a custom text-to-speech service through GenericTTS and its generic_http vendor. Fish Audio connects through a small server-side adapter that follows Agora's OpenAI-compatible /audio/speech contract and streams PCM audio back to the agent.

This is a real custom-TTS integration, not a claim that Fish Audio appears in Agora's native provider selector. Agora owns the conversational pipeline; your adapter owns the Fish Audio request, credential boundary and PCM response.

How does Agora support Fish Audio text to speech?

Agora's custom TTS documentation defines a generic_http TTS path. The service must accept an HTTP POST, expose an OpenAI-compatible audio speech endpoint, authenticate with a bearer key and return PCM audio.

Fish Audio becomes the synthesis engine behind that endpoint. The adapter receives text and voice settings from Agora, calls Fish Audio with a server-side key, converts the response to the required PCM sample rate and streams audio back without buffering the whole answer unless the transport requires it.

The architecture is:

1. Agora produces the agent's text response.
2. GenericTTS calls your /audio/speech adapter.
3. The adapter selects the approved Fish Audio voice and model.
4. Fish Audio generates speech.
5. The adapter returns PCM to Agora.
6. The transport plays audio to the caller.

What request contract must the adapter implement?

The adapter should accept POST requests with a JSON body containing input, model, voice, speed, sample_rate, response_format and optional delivery instructions. Validate the fields against the current Agora contract rather than assuming every OpenAI-compatible parameter has identical semantics.

A practical configuration keeps Fish Audio's model and voice as server-side deployment defaults. Agora can pass a voice identifier, but the adapter should map it through an allowlist instead of forwarding arbitrary values to Fish Audio.

{
  "tts": {
    "vendor": "generic_http",
    "url": "https://tts.example.com/audio/speech",
    "params": {
      "model": "s2.1-pro",
      "voice": "approved-voice-id",
      "sample_rate": 16000,
      "response_format": "pcm"
    }
  }
}

Use Agora's Agent SDK or REST configuration as the source of truth for exact field names. The Agora custom TTS page includes the current request body, error shape and generic_http reference.

How should the Fish Audio adapter authenticate requests?

Use two separate credentials. Agora authenticates to your adapter with a scoped bearer key or approved header, while the adapter authenticates to Fish Audio with a server-side Fish Audio API key.

Never pass the Fish Audio key through the public Agora request. Rotate the adapter credential independently and restrict the endpoint by environment, allowed voice IDs, request size and concurrency. The Fish Audio API guide covers key creation and staging/production separation.

Which audio format should Agora receive?

Agora's custom TTS response is a PCM stream. Start with the sample rate Agora requests, such as 16 kHz, and verify the channel count, signedness and byte order expected by the deployment.

Do not return a WAV header or MP3 bytes while declaring the stream as PCM. Those mismatches often appear as noise, pitch shift or silence even though the adapter returned HTTP 200. Test the raw output with a short sentence before connecting the agent to telephony.

How do you keep Fish Audio latency predictable?

Stream from Fish Audio to the adapter and from the adapter to Agora. Avoid waiting for the entire response, parsing the complete audio in memory or applying a slow format conversion after generation.

Measure four timestamps: Agora request received, Fish Audio request sent, first audio byte received and first audio byte returned to Agora. Then measure from the caller's final syllable to the first audible response through the real transport.

Use a low or balanced Fish Audio latency setting only after listening tests. A lower-latency mode can change delivery quality, so compare the same sentences and voice rather than changing several variables at once.

The real-time text to speech guide covers streaming, cancellation and p95 latency planning in more detail.

How should interruption and cancellation work?

Agora should cancel the active speech request when the caller interrupts. The adapter must propagate that cancellation to Fish Audio and stop forwarding bytes immediately.

Keep request identifiers so logs can connect an Agora turn to the Fish Audio request without storing the caller's full conversation by default. If cancellation arrives after generation starts, record whether it reduced audible output or only discarded queued audio.

Test interruptions at the beginning, middle and end of a long response. Confirm that the transcript and context aggregator reflect what the caller actually heard, not text that was generated and cancelled.

How should the integration fall back?

Configure a second Agora TTS path or a controlled human transfer for Fish Audio timeout, quota exhaustion, invalid voice and malformed-response failures. The adapter should distinguish those cases rather than returning an empty success or waiting until the caller gives up.

Keep the fallback voice close in gender, role, language and pace. If the fallback is noticeably different, consider ending the turn with a short transfer message instead of continuing the conversation in a different persona.

The voice-agent TTS fallback guide provides a provider-neutral fault matrix and test plan.

What should the production pilot include?

Test short confirmations, long tool-driven answers, numbers, names, account identifiers and two languages. Include a cancellation, an adapter timeout, a Fish Audio failure and a sample-rate mismatch.

Measure first-audio latency, completion rate, voice consistency and cost per completed call. Review one real telephone recording because a browser clip does not show codec, handset or background-noise problems.

At fishaudio.lol, one Fish Audio credit covers about 1,000 characters and API access is included in the Studio pack. Agora usage and transport costs remain separate. Use the TTS API pricing guide for character-volume planning.

Key takeaways

• Agora supports custom TTS through GenericTTS and generic_http.
• Fish Audio connects through an OpenAI-compatible PCM adapter, not a claimed native Agora provider.
• Keep Fish Audio credentials server-side and map voice IDs through an allowlist.
• Stream both directions, return raw PCM at the requested sample rate and test interruption.
• Validate provider failure, fallback, voice continuity and p95 first-audio latency before launch.

Frequently asked questions

Does Agora support a custom text-to-speech service?

Yes. Agora Conversational AI documents GenericTTS and the generic_http vendor for an HTTP TTS service that follows the OpenAI audio speech contract and returns PCM audio.

Does Fish Audio connect directly to Agora?

Use a thin server-side adapter that accepts the Agora generic_http request and calls the Fish Audio API. The adapter keeps Fish Audio credentials private and returns PCM in the format Agora expects.

What audio format should the Fish Audio adapter return?

Return a PCM stream at the sample rate configured in Agora. A 16 kHz mono stream is a practical starting point, but verify the audio pipeline and transport before production.

How should Fish Audio credentials be handled?

Store the Fish Audio key in the adapter or a secret manager, never in the browser or Agora request parameters exposed to callers. Authenticate the public adapter with its own scoped credential.

What should I measure before launch?

Measure p50 and p95 time from the caller finishing a turn to the first audible reply, then test interruption, sample-rate mismatch, provider timeout and fallback.