Gateway Design
How the Mosaic Gateway routes and manages AI requests.
Gateway Design#
The Gateway is the central entry point for all AI requests. It implements a pipeline of middleware stages.
Request Pipeline#
- Authentication and rate limiting
- Policy evaluation (cost, latency, provider preference)
- Provider selection and failover
- Request transformation and forwarding
- Response streaming and telemetry emission
Configuration Example#
bash
# .env — provider configuration
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODELS=llama3.2,codellama,mistral
GATEWAY_PORT=14242