Chat Service

Chat session management and message routing with WebSocket streaming support

chat

chatwebsocketstreamingfastapisessions

Updated 2026-11-03

Verified 2026-11-03

The Chat Service (cosmos-chat-v3) manages chat sessions and routes messages between the frontend and the backend agentic framework. It transforms frontend parameters, adds required fields for the ADK service, and provides both REST and WebSocket endpoints for real-time conversation streaming.

Architecture

Framework: FastAPI + Uvicorn
Streaming: WebSocket at /api/v1/converse_stream
Authentication: Manual JWT middleware requiring Authorization + X-Org-Id on every route except /health/. WebSocket clients can send the token in the Authorization header or as a Sec-WebSocket-Protocol value.
Token Forwarding: Forwards the user's JWT, organization, and project context to ADK so downstream permissions stay user- and project-scoped
Event Consumer: Separate chat-consumer container for Kafka events
Dependencies: The image installs Python dependencies with pip into /app/.venv; after a dependency change, rebuild with ./manage.sh dev build chat and restart
Telemetry: Cloud deployments start the webserver and consumer under opentelemetry-instrument, which reports to Azure Monitor through APPLICATIONINSIGHTS_CONNECTION_STRING

Components

Component	Description
REST API Server	HTTP endpoints for chat interactions
WebSocket Server	Real-time streaming for conversations
Event Consumer	Kafka consumer for agent/app configuration events
Transformation Service	Converts frontend params to ADK-compatible format

API Overview

Method	Endpoint	Description
WS	`/api/v1/converse_stream`	WebSocket streaming conversation
POST	`/api/v1/converse`	Synchronous chat message
POST	`/api/v1/converse/whatsapp`	WhatsApp channel message
POST	`/api/v1/trig-add-event`	Add an event to a session
POST	`/api/v1/trig-end-session`	End a session
POST	`/api/v1/upload-file`	Upload a file for use in a conversation
GET	`/health/`	Health check

→ Full API reference: Chat API

Configuration

Variable	Description
`COSMOS_SERVICE_CLIENT_ID`	OAuth service client ID; compose sets `cosmos-chat-v3`
`COSMOS_SERVICE_CLIENT_SECRET`	OAuth client secret; sourced from `COSMOS_CHAT_SERVICE_CLIENT_SECRET`
`ADK_HOST_URL`	ADK service URL
`CONFIG_HOST_URL`	Configurator service URL
`USER_SERVICE_URL`	User Service URL
`KAFKA_BOOTSTRAP_SERVER`	Kafka broker address
`KAFKA_TOPIC_CONSUMER`, `KAFKA_TOPIC_PRODUCER`	Kafka topics the consumer reads and the service writes
`REDIS_HOST`, `REDIS_PORT`	Redis connection
`HYPERSCALER`	`local`, `azure`, or `gcp`
`CORS_ALLOWED_ORIGINS`	Allowed browser origins
`CHAT_V2_TEST_RELAY_ENABLED`	Enables the agent test-session relay protocol (off by default)

Integration Points

Service	Integration
ADK	Forwards processed messages for agent execution
User Service	JWT token validation
Frontend	Primary consumer via REST and WebSocket
Kafka	Consumes `configurator` topic for app/agent updates

Troubleshooting

WebSocket connection drops

Check Chat service logs: ./manage.sh dev logs chat
Verify ADK service is healthy: curl http://localhost:8103/health
Verify the Chat service WebSocket port is reachable (COSMOS_CHAT_HOST_PORT, default 8108)
Check the close code: 4401 and 4403 are authentication and authorization failures; see Chat Conversations API