Using the Realtime Text to Speech API

Endpoint and Prerequisites

Use a regional realtime endpoint where the feature is available. The WebSocket path is /ws/synthesize/stream. For commercial OCI regions, the endpoint pattern is:

wss://realtime.aiservice.<region>.oci.oraclecloud.com/ws/synthesize/stream

Replace <region> with a supported region identifier. Use the endpoint domain for your realm.

You need permission to use Speech synthesis and list voices in the selected compartment. See About Speech Policies for information about Speech access policies. Select a voice compatible with your model and language using ListVoices.

The examples use placeholders for your compartment, voice, and authentication values. Send client messages as JSON text messages.

Authenticate the Connection

Send an authentication object as the first WebSocket text message. Opening the connection alone doesn't authenticate the session.

1. Convert the WebSocket URL to its HTTPS signing equivalent by replacing wss with https. Preserve the exact host, port, path, and query string.
2. Use an OCI signer to sign a GET request for that URL. See Request Signatures.
3. Copy every header produced by the signer into the authentication object's headers map. Include host and uri if the signer hasn't added them. Set uri to the full HTTPS signing URL.
4. Send the object with authenticationType set to CREDENTIALS. Use the same compartment for authentication and the synthesis request.

{
  "authenticationType": "CREDENTIALS",
  "compartmentId": "<compartment-ocid>",
  "headers": {
    "authorization": "<authorization-header-from-OCI-signer>",
    "date": "<date-header-from-OCI-signer>",
    "host": "realtime.aiservice.<region>.oci.oraclecloud.com",
    "uri": "https://realtime.aiservice.<region>.oci.oraclecloud.com/ws/synthesize/stream"
  }
}

Include the signer's date or x-date header and any additional headers it produces. The endpoint doesn't support TOKEN as the value of authenticationType.

Wait for both the authenticated and connected events before starting synthesis. They can arrive in either order.

{
  "event": "authenticated",
  "sessionId": "<authentication-session-id>"
}

{
  "event": "connected",
  "sessionId": "<synthesis-session-id>"
}

The two session IDs can differ. Use the ID from connected to identify subsequent synthesis events.

Start Synthesis

Send one start action per connection. Include nonblank initial text, set isStreamEnabled to true, and select a compatible model and voice. This example uses Natural synthesis with US English and MP3 output:

{
  "action": "start",
  "request": {
    "compartmentId": "<compartment-ocid>",
    "text": "Welcome to the garden tour. ",
    "isStreamEnabled": true,
    "configuration": {
      "modelFamily": "ORACLE",
      "modelDetails": {
        "modelName": "TTS_2_NATURAL",
        "voiceId": "<compatible-voice-id>",
        "languageCode": "en-US"
      },
      "speechSettings": {
        "textType": "TEXT",
        "outputFormat": "MP3",
        "sampleRateInHz": 24000,
        "speechMarkTypes": []
      }
    }
  }
}

For Standard synthesis, set modelName to TTS_1_STANDARD, select a compatible voice, and omit languageCode from modelDetails.

Wait for session_started before appending text:

{
  "event": "session_started",
  "sessionId": "<synthesis-session-id>",
  "outputFormat": "MP3",
  "contentType": "audio/mpeg"
}

Append Text and Finish Input

After session_started, send additional text while continuing to receive audio:

{
  "action": "append_text",
  "text": "Our first stop is the fern collection. "
}

Preserve the spaces between segments and send each segment only once. Don't repeat text from the start request.

For typed or generated text, buffer characters through the latest whitespace boundary before sending them. Keep any trailing word fragment for the next message. When input is complete, send the remaining text, including a final word without trailing whitespace. Punctuation alone can occur inside a word or value, so it isn't a reliable boundary.

After all text has been sent, send end_input:

{
  "action": "end_input"
}

Don't append text after this action. Continue processing audio until completed. The service can send an input_ended acknowledgement, but your client must not depend on receiving it to process audio.

Receive Audio and Check Completion

The service sends audio in JSON text events. Each chunk contains base64-encoded bytes from the same MP3 stream:

{
  "event": "audio_chunk",
  "sessionId": "<synthesis-session-id>",
  "chunkIndex": 0,
  "encoding": "base64",
  "contentType": "audio/mpeg",
  "chunk": "<base64-encoded-mp3-data>",
  "audioDurationMs": 740.0
}

For each audio event:

1. Check that sessionId matches the ID from connected.
2. Check that chunkIndex starts at zero and increases by one for each chunk.
3. Check that encoding is base64 and contentType is audio/mpeg.
4. Decode chunk and verify that the decoded audio isn't empty.
5. Append the decoded bytes in order. To save the audio, write the combined bytes to an .mp3 file.
6. Check that the cumulative audioDurationMs doesn't decrease.

Keep receiving events after end_input. A successful session ends with completed, for example:

{
  "event": "completed",
  "sessionId": "<synthesis-session-id>",
  "chunksProduced": 12,
  "audioDurationMs": 3840.0
}

Before accepting the audio as complete, verify that you received at least one nonempty chunk, that chunksProduced matches your received chunk count, and that the final duration is positive and no less than the last chunk's duration. The values in these event examples are illustrative.

Treat error and fatal_error events as failures. Process these errors before checking the session ID because an error event might not include one. If the connection ends without a completion event, any audio received can be incomplete.

Cancel a Session

To stop an active session, send:

{
  "action": "cancel"
}

The service can respond with cancelled. Don't send further text or control actions after cancellation. Open and authenticate a new connection for another synthesis session.

Input and Session Limits

• The combined text in start.request.text and all append_text.text messages must not exceed 10,000 characters. The Console has a separate limit of 5,000 characters.
• The initial text must be a nonblank JSON string. Appended text must also be a JSON string.
• Use a compatible model, language, voice, and sample rate. The sample rate must be a whole number from 8,000 through 48,000 Hz.
• Send one start action per WebSocket connection. Don't reuse a completed or cancelled session.
• Send JSON text messages. Binary client messages aren't supported.

Troubleshooting

Correct malformed requests or authorization failures before retrying. If audio stops unexpectedly, check for an error event and retain the connection's close reason for troubleshooting.