DocsArchitectureGateway Design

Gateway Design

How the Mosaic Gateway routes and manages AI requests.

Gateway Design#

The Gateway is the central entry point for all AI requests. It implements a pipeline of middleware stages.

Request Pipeline#

  1. Authentication and rate limiting
  2. Policy evaluation (cost, latency, provider preference)
  3. Provider selection and failover
  4. Request transformation and forwarding
  5. Response streaming and telemetry emission

Configuration Example#

yaml
gateway:
  listen: 0.0.0.0:8080
  providers:
    - name: openai
      priority: 1
      models: [gpt-4o, gpt-4o-mini]
    - name: anthropic
      priority: 2
      models: [claude-sonnet-4-20250514]