Voice Service

Voice bot platform with WebRTC and telephony support — STT, TTS, and ADK LLM integration via Pipecat

voice

voicewebrtctelephonypipecatsttttstwilioelevenlabsazure-speech

Updated 2026-11-03

Verified 2026-11-03

The Voice Service (cosmos-voice-v3) is a voice bot platform supporting both telephony (Twilio) and WebRTC connections. Built on FastAPI and Pipecat, it integrates multiple AI services for speech-to-text, text-to-speech, and conversational AI via the ADK service.

Architecture

Framework: FastAPI + Pipecat 1.14 (voice pipeline framework)
Dependency Management: uv (pyproject.toml + uv.lock), installed into /app/.venv in the image
Transports: WebRTC (primary), Telephony (Twilio)
STT Providers: Azure Speech, Deepgram, ElevenLabs, Google, Smallest.ai
TTS Providers: Azure Speech, Cartesia, ElevenLabs, Google, Sarvam
Provider credentials: Resolved per app through Configurator bring-your-own-token (BYOT) entries, not service environment variables. Azure Speech uses the BYOT codes azure_speech_service (STT) and azure_speech_synthesis (TTS) and connects through an explicit Cognitive Services endpoint (a private endpoint or the BYOT base_url).
LLM: Cosmos ADK via WebSocket, using the cosmos.voice.v2 protocol
ICE: Stunner TURN gateway in Kubernetes, with TURN over TCP on port 443 as a fallback for restrictive networks; public STUN for local development
Runtime: Uvicorn with exactly one worker per container; local port 8106 maps to container port 8000

Pipeline Architecture

Voice pipeline

Source

Loading diagram...

Core Features

Feature	Description
First Message	Bot speaks first when call connects (configurable greeting)
ADK End Conversation	AI-driven call termination based on ADK response
Max Duration	Automatic call termination after specified duration
Idle Handling	Pipecat idle detection ends calls when the user stays silent
RTVI Protocol	Real-time voice & interaction protocol support
Metrics	STT, TTS, and LLM metrics via observability

Voice sessions use the modern Pipecat pipeline only. Apps configured with the legacy voice profile are rejected, and the legacy silence_timeout setting is replaced by idle handling.

API Overview

The routes depend on the transport mode the service runs in.

WebRTC mode

Method	Endpoint	Description
GET	`/api/ice-servers`	ICE/TURN server configuration
POST	`/api/offer`	Send the SDP offer and start a voice session
PATCH	`/api/offer`	Send trickled ICE candidates for an existing connection

Telephony mode (Twilio)

Method	Endpoint	Description
POST, GET	`/`	Twilio voice webhook for incoming calls
WS	`/ws/events/{call_id}`	Call event stream
WS	`/ws/{call_id}/{stream_token}`	Twilio media stream
POST	`/status-callback/{call_id}/{callback_token}`	Twilio call status callback

Both modes

Method	Endpoint	Description
GET	`/health/`	Liveness check
GET	`/ready/`	Readiness check

Errors use {"status": "error", "code": "...", "message": "..."}. See API Conventions.

Session Flow

User clicks "Start Call" on frontend
Opens cosmos-voice-ui in new tab with params (app_id, version, org_id, name)
cosmos-voice-ui connects to Voice service WebRTC endpoint
Voice service creates Pipecat pipeline with ADK LLM service
Conversation flows: Browser ↔ WebRTC ↔ Pipecat ↔ ADK ↔ LLM

ADK creates session IDs (not the client). The session ID is sent to the client via bot-started RTVI event after the first query.

Configuration

Variable	Description
`CONFIGURATOR_BASE_URL`	Configurator API for app, agent, and BYOT configuration
`ADK_WS_URL`	ADK WebSocket URL for LLM interactions
`USER_SERVICE_URL`	User Service URL for token validation and service tokens
`COSMOS_SERVICE_CLIENT_ID`, `COSMOS_SERVICE_CLIENT_SECRET`	OAuth service client credentials (`cosmos-voice-v3`)
`KAFKA_BOOTSTRAP_SERVER`	Kafka broker address
`REDIS_*`	Redis connection
`STUNNER_AUTH_URL`, `STUNNER_SERVICE`	Stunner TURN credential service (Kubernetes)

Integration Points

Service	Integration
ADK	WebSocket-based LLM conversations
Configurator	App/agent configuration and speech-provider BYOT credentials
Kafka	Observability event production
Redis	Session caching
Frontend	`cosmos-voice-ui` (Next.js on port 3001)

Troubleshooting

ICE server errors in local development

The Stunner service only exists inside Kubernetes clusters. Without it, local development uses public STUN servers.

Stale dependencies after update

The image builds its own virtual environment, so rebuild the image and recreate the container:

bash

./manage.sh dev build voice
./manage.sh dev restart voice

The local docker-compose.dev.yml mounts a named volume (voice-venv) at /app/.venv. Docker only seeds a named volume from the image when the volume is empty, so if the container still reports old packages after a rebuild, the volume is holding the previous environment. Re-seed it from the rebuilt image with ./manage.sh dev rebuild voice, which removes the volume and recreates the container.

ℹ️ Info: Voice is an optional service. Start it explicitly with ./manage.sh dev start voice.