Platform Architecture

Cosmos is a multi-service AI agent platform. It provides shared AI infrastructure, such as agent execution, tool management, authentication, and storage, and a family of applications is built on top of it. Each service is developed and deployed independently. Services communicate through REST APIs, WebSockets, and Kafka events, and the web frontend calls each service directly.

High-Level Architecture

The frontend is the browser entry point. It calls each backend service at its own address: a published port locally, or a per-service public URL in deployed environments. There is no shared gateway in front of the services.

Platform Services

Core services

AI services

Optional services

Infrastructure

Local Service Groups

Locally, services run with Docker Compose through ./manage.sh. Default groups start with ./manage.sh dev start; optional services start only when you name them.

./manage.sh dev start voice                # one optional service
./manage.sh dev start knowledge            # the RAGFlow services
./manage.sh dev start observer-webserver observer-worker

./manage.sh dev start all starts the default groups (infra, core, and ai). Media Gateway and Model Garden are not part of the local compose groups. See Deploying Locally.

Data Flow

1. User request path

A user's message reaches the Chat or ADK service. ADK loads the agent configuration from Configurator, builds the agent tree, connects to MCP for tools, and runs the agent loop. MCP executes tool calls against external APIs, databases, or custom integrations.

2. Tool lifecycle

When a tool is created or deployed through Configurator, a Kafka event is emitted on the configurator topic. The MCP tools-consumer writes the tool definition to Redis (DB 11). On the next tool listing, MCP loads the tools from Redis into its in-memory registry, and later calls are served from memory.

3. Observability

MCP and ADK publish observability events on the observer Kafka topic. The Observer worker consumes them and writes them to the analytics backend for the environment: Azure Data Explorer or Google BigQuery, selected by HYPERSCALER.

The web frontend reports browser telemetry (page views, navigation, API dependencies, and exceptions) to Azure Monitor Application Insights in Azure deployments. See Frontend.

4. Caching

The cache is kept current in two ways:

• Proactive — Kafka events update Redis in near real time.
• Reactive (auto-sync) — when MCP gets a request for a tool that is not cached, it fetches the tool from Configurator, updates Redis, and registers it in memory.

Inter-Service Authentication

Services authenticate to each other with OAuth2 client_credentials tokens issued by the User Service at /oauth/token/. Each service has a stable client ID and a secret stored in its environment file or the cloud secret store. Runtime calls send Authorization: Bearer <token> and, for organization-scoped APIs, X-Org-Id: <uuid>.

Service tokens carry actor_type=service and service_id=<client id>. When a service acts for a user, it also sends X-On-Behalf-Of-User-Id.

Service clients are provisioned by ./manage.sh <env> setup-tokens and checked by ./manage.sh <env> verify-tokens. See Service Tokens for client IDs, secret names, and management.

Database Allocation

PostgreSQL

Each service has its own database, created on first start:

ℹ️ Database names use the _dev suffix in the dev environment. Azure and GCP environments use cosmos.

Redis

See Redis Databases.

Kafka Topics

See Kafka Topics.

Port Reference

Local ports are the same in every environment:

API Conventions

• API routes are versioned: /api/v1/.
• Requests need Authorization: Bearer <token> and X-Org-Id: <uuid>.
• Optional services return 503 with {"error": "Service name is currently unavailable"} when they are down.
• Health checks are unauthenticated: GET /health/.

Multi-Cloud (Hyperscaler) Pattern

Several services change behavior based on HYPERSCALER (local, azure, gcp):

Consumer Applications

Consumer applications are products built on Cosmos. They use the platform for agent execution, authentication, storage, and tools, and add their own domain logic and interfaces.

• JumpStart SAP Intelligence — a consulting workbench for SAP, Oracle, and BTP assessment and migration. Teams use it to analyze codebases, generate migration blueprints, and speed up development with AI-assisted code generation, with a canvas-based UI and connectors to SAP and Oracle systems. Learn more
• JumpStart AMS — an application management support portal for project-based ticket intake, triage and routing, and SLA tracking. Learn more
• JumpStart Workbench — an agent workbench that brings Cosmos agents and tools into the developer's environment, as a VS Code extension or a desktop application. Learn more

When the platform gains a capability, such as a new model provider, tool type, or better observability, every consumer application can use it.

SDKs

Two SDKs are available for building on Cosmos:

• TypeScript SDK — for Node.js applications. Covers authentication, agent communication, tool management, storage, and real-time streaming. See TypeScript SDK.
• Python SDK — for Python backends and data pipelines, with the same capabilities and an async-first design. See Python SDK.

Both handle service authentication, WebSocket streaming, and multi-service coordination, so application developers can focus on their domain.

Deployment