Skip to content

Docs

Build a voice agent

Install an SDK, point it at an agent you published, and call connect. Everything the agent knows lives on the server, so changing its behaviour never means shipping a new build.

Install

Five SDKs share one wire protocol, so an agent configured once behaves the same on every platform.

WebTypeScriptnpm install @voxera/sdk-web
React NativeTypeScriptnpm install @voxera/sdk-react-native
iOSSwift.package(url: "https://github.com/voxera-voice/voxera-ios.git", from: "1.1.42")
AndroidKotlinPublic release in progress
FlutterDartPublic release in progress

Quickstart

Create an agent in the dashboard, issue a publishable key, then:

voice.ts
import { VoxeraClient } from "@voxera/sdk-web";

const client = new VoxeraClient({
  appKey: "vx_pk_live_...",          // publishable key, safe in a bundle
  serverUrl: "https://rtc.voxera-voice.com",
  agentId: "your-agent-id",          // the agent you published in step 2
});

await client.connect();
client.startConversation();

The SDK captures the microphone and plays the reply itself. If a browser blocks autoplay it emits autoplayBlocked rather than failing quietly, so you can retry on the next gesture.

Credentials

Three credentials on three trust levels. Getting this wrong is a security bug rather than a style choice, so each one is accepted at exactly one endpoint.

CredentialLivesCan do
vx_pk_*In your bundleStart a voice session, from an allowed origin only
vx_sk_*Your serverMint session tokens. Refused if presented by a client
session tokenOne callStart one session, then expires

Set an origin allowlist on every publishable key. An empty allowlist skips the check entirely, and the key then works from any site on the internet — against your voice minutes. Secrecy is not what protects a publishable key; the allowlist is.

For stricter deployments, mint a short-lived token server-side and never ship a key at all:

server.ts
// Your backend. The secret key stays here, never in the app bundle.
const response = await fetch(
  "https://api.voxera-voice.com/api/v1/sessions/token",
  {
    method: "POST",
    headers: {
      "content-type": "application/json",
      authorization: `Bearer ${process.env.VOXERA_SECRET_KEY}`,
    },
    body: JSON.stringify({ userId: user.id, metadata: { plan: user.plan } }),
  },
);

const { token } = await response.json();

Agents

An agent is a published, immutable version holding a system prompt, a speech-to-text config, a model config, a text-to-speech voice, and a set of tools. Clients name it by id. Publishing a new version changes behaviour everywhere at once, with no client release.

A client may also send its own prompts and tools at session start, but only if the agent's runtime policy allows it — both flags default to closed. If a client sends tools and nothing happens, that policy is the first thing to check.

Agents answer in whatever language the caller speaks. Pin a language on the transcription config when a deployment really is single-language: it is more accurate and slightly faster than detection.

Tool calling

Tools let an agent look something up or take an action mid-sentence. The server pauses the reply until the result arrives, which makes one rule absolute:

Answer every tool call exactly once. A call you never answer does not degrade the conversation, it ends it — the agent stops mid-sentence with nothing to show why.

tools.ts
client.on("toolTriggered", async ({ actionId, name, arguments: args }) => {
  const result = await runYourTool(name, args);

  // Free text, read by the model verbatim. Say what happened in terms it
  // can act on — "booked for Friday 10:00", not "200 OK".
  client.selectAction(actionId, result);
});

Events

Subscribe with client.on(name, handler), which returns an unsubscribe function.

EventPayloadFires when
stateChangedVoxeraClientStateConnection, conversation or speaking status moves. Also fires once on start.
messageConversationMessageA reply streams in, or the caller's turn is transcribed. Prefer fullText over content — content is only the newest delta.
transcript{ text, isFinal }Live speech-to-text while the caller is still speaking.
audioLevel{ level, isAi }Loudness update on a 0–1 scale. isAi says whose voice it is.
toolTriggered{ actionId, name, arguments }The agent called one of your tools. Must be answered with selectAction.
remoteStreamMediaStreamThe agent's audio is ready. Only needed if you set autoPlayRemoteAudio: false.
remoteTrackMediaStreamTrackAn individual remote track arrived, for callers doing their own mixing.
localStreamMediaStreamThe microphone was captured, for a local level meter or preview.
autoplayBlocked{ stream }The browser refused to play audio unprompted. Retry resumeRemoteAudio() on the next user gesture.
errorVoxeraProtocolErrorA protocol-level failure, carrying a code you can branch on.
rawEvent{ event, data }Every server event, unparsed. An escape hatch for anything the typed map does not cover yet.

Errors

Every failure arrives on the error event with a code. Branch on the code, not the message — messages are written for people and change.

AUTHENTICATION_FAILED

The key was rejected, the origin is not on its allowlist, the agent is not published, or the monthly voice-minute allowance is spent.

What to do: Do not retry. The message says which. Check the key's allowed origins first — that is the most common cause in a browser.

CAPACITY_LIMIT

The tenant is already running as many concurrent calls as the plan allows.

What to do: Retry shortly, or upgrade. The credential is fine.

MEDIA_ACCESS_DENIED

The browser or the person refused microphone access.

What to do: Ask again from a user gesture. Check the page is on HTTPS or localhost — outside a secure context the request is refused before anyone is asked.

CONNECTION_FAILED

The socket never opened.

What to do: Check serverUrl and that the network allows WebSocket.

WEBRTC_ERROR

Signalling succeeded but media did not, usually a transport or ICE failure.

What to do: Most often a restrictive network. The SDK falls back to TURN on its own; a proxy that blocks UDP entirely will still fail.

TIMEOUT

A step took longer than connectionTimeoutMs, which defaults to 30s.

What to do: Retry. Raise the timeout only if you know the network is slow.

INVALID_CONFIG

The session config was rejected before anything was attempted.

What to do: A required field is missing or malformed. The message names it.

NETWORK_ERROR

The connection dropped mid-session.

What to do: Reconnect. Set autoReconnect to have the SDK do it for you.

SERVER_ERROR

The server failed for a reason it did not attribute to the client.

What to do: Retry once. If it persists it is worth reporting.

Limits and billing

A voice minute is wall-clock time on a connected session, metered in seconds. Time spent connecting or waiting is not billed. Calls in progress count against the allowance, so a burst of concurrent sessions cannot overshoot it.

The allowance resets monthly, anchored to your billing day rather than the first of the month. Sessions are refused once it is spent; calls already running are allowed to finish.

Concurrency is enforced across every server, not per process, so CAPACITY_LIMIT means the account really is at its ceiling. See pricing for the per-plan numbers.

Other platforms

The same agent, from each SDK. Every sample is checked against the SDK source.

VoiceScreen.tsx
import { useVoxera } from "@voxera/sdk-react-native";

export function VoiceScreen() {
  const { start, leave, isActive, messages, speakingState } = useVoxera();

  return (
    <View>
      <Text>{speakingState === "ai" ? "Agent speaking" : "Listening"}</Text>
      <Button
        title={isActive ? "End call" : "Start call"}
        onPress={() =>
          isActive
            ? leave()
            : start({
                appKey: PUBLISHABLE_KEY,
                serverUrl: "https://rtc.voxera-voice.com",
                userId: "user-123",
              })
        }
      />
      {messages.map((message) => (
        <Text key={message.id}>{message.content}</Text>
      ))}
    </View>
  );
}
VoiceView.swift
import SwiftUI
import VoxeraSDK

struct VoiceView: View {
  // The view model takes its configuration up front — there is no
  // separate configure() step.
  @StateObject private var voxera = VoxeraViewModel(
    config: VoxeraConfig(
      appKey: publishableKey,
      serverUrl: "https://rtc.voxera-voice.com",
      userId: "user-123"
    )
  )

  var body: some View {
    VStack(spacing: 12) {
      Text(voxera.isConnected ? "Connected" : "Idle")

      ForEach(voxera.conversationMessages) { message in
        Text(message.content)
      }

      Button("Start call") {
        voxera.connect()
        voxera.startConversation()
      }
    }
  }
}
VoiceActivity.kt
import com.voxera.sdk.ConversationMessage
import com.voxera.sdk.VoxeraClient
import com.voxera.sdk.VoxeraConfig
import com.voxera.sdk.VoxeraListener

// Configuration and the listener are passed to the constructor;
// connect() itself takes no arguments.
private val voxera = VoxeraClient(
  context,
  VoxeraConfig(
    appKey = PUBLISHABLE_KEY,
    serverUrl = "https://rtc.voxera-voice.com",
    userId = "user-123",
    listener = object : VoxeraListener {
      override fun onMessage(message: ConversationMessage) {
        binding.transcript.append(message.content)
      }
    },
  ),
)

voxera.connect()
voxera.startConversation()
voice_page.dart
import 'package:voxera_flutter/voxera_flutter.dart';

final voxera = VoxeraClient(const VoxeraConfig(
  appKey: publishableKey,
  serverUrl: 'https://rtc.voxera-voice.com',
  userId: 'user-123',
));

// Messages arrive as typed events, not as a list.
final sub = voxera.on('message').listen((event) {
  setState(() => _transcript.add(event.data['content'] as String));
});

await voxera.connect();
await voxera.startConversation();