Important
You can use this project exactly as it is, but consider it merely a starting point—your imagination will surely enable you to create something even better than what I have built. This project is also my protest against the proliferation of AI gateways that lack any standardization.
这个项目你可以原样使用，也是抛砖引玉，你的想象力一定能创造比我更美好的东西。此外本项目是我对于 AI 网关满天飞但是却毫不规范的抗议。
A reusable Docker Compose stack that builds a self-owned AI gateway from separate modules: CLIProxyAPI, Sub2API, AISIX, Apache APISIX, ai-sse-keepalive-proxy, openai-api-cache-keeper, Squid, socat and Cloudflare Tunnel. Each module keeps one job: provider credentials, client API keys, logical-model routing, public routing and Internet ingress stay separate. Components are cleanly separated and no script hides the configuration, so an AI agent can deploy and change it while a human reviews it.
flowchart LR
Client --> Cloudflare --> APISIX[APISIX front door]
APISIX --> Keepalive[SSE keepalive] --> Keeper[cache keeper + WS/SSE bridge] --> Sub2API
Sub2API --> Models[apisix-models] --> CPA[CLIProxyAPI]
Models --> AISIX --> CPA
APISIX -->|versioned /v1/models| Models -->|/v1/models| Catalog[model-catalog]
Operator[localhost / Tailscale :9083] -->|/models-table| APISIX -->|/models-table| Catalog
Catalog -.->|read-only| CPA
Catalog -.->|read-only| AISIX
Sync[cpa-model-sync] --> CPA
CPA --> Squid[Squid egress allowlist] --> Providers
Full graph: docs/architecture.md.
Warning
Every intentionally public host port needs an external firewall or cloud security group. Tracked defaults bind host ports to 127.0.0.1; never change a *_BIND_ADDRESS to 0.0.0.0. See docs/security.md.
All components live under services/, grouped by role; operations tooling lives under ops/.
| Component | Where | What it does | How to use |
|---|---|---|---|
| Cloudflare Tunnel | service cloudflared |
Internet ingress to the front APISIX | Public Hostname origin http://apisix-ingress:9080 (installation) |
| APISIX front door | services/firewall/apisix/, 127.0.0.1:9080; operator 9083 |
Public AI route allowlist, limits, opaque 404; catalog intersection; models table on 9083 |
Public API, catalog path |
| Egress proxy | services/firewall/egress-proxy/ |
Fail-closed Squid allowlist; the only Internet egress | Edit data/egress-proxy/policy.json (egress policy) |
| One-way relays | services/firewall/one-way-relay/ |
socat relays and namespace owners for every directed edge | Security model |
| AI SSE keepalive proxy | services/middleware/ai-sse-keepalive-proxy/, operator 127.0.0.1:8087 |
Heartbeats for silent AI SSE streams | No configuration |
| Cache keeper + WS/SSE bridge | services/middleware/ |
Responses history, background replay, WebSocket → HTTP/SSE | Responses path |
| Sub2API | services/sub2api/, admin 127.0.0.1:8086 |
The only client API-key, entitlement and quota authority | Upstream configuration |
| apisix-models | services/routing/apisix-models/, no host port |
Constant pipe: /v1/models → model-catalog, everything else → AISIX |
Model routing |
| AISIX + status page | services/routing/aisix/, aisix-status/; 127.0.0.1:3001/status |
Logical model IDs, ordered fallback, cooldown; no-JS status page | Upstream configuration |
| model-catalog | services/routing/model-catalog/ |
Codex model catalog and /models-table; reads CPA and AISIX read-only |
model-catalog reference |
| CLIProxyAPI (CPA) | services/cpa/config/, 127.0.0.1:8317 |
Provider credentials and pool execution | Management UI with CPA_MANAGEMENT_KEY |
| cpa-model-sync | services/cpa/cpa-model-sync/ |
Syncs model membership of five CPA API-key channel kinds | Its README |
| provider-sidecar (optional) | private compose.override.yaml |
OpenAI-compatible sidecar behind TLS relay and tunnel | Provider sidecar |
| Ansible ops | ops/ansible/ |
Guarded plan/apply deploys and rollbacks to the host | Ansible operations |
| Scripts | ops/scripts/ |
Init, validate, render, host recovery and tests | Operations |
Requirements: Linux with /dev/net/tun, Docker Engine with Compose v2.33.1+ or rootless Podman with a Compose provider, OpenSSL, and a remotely managed Cloudflare Tunnel.
git clone --recurse-submodules https://github.com/xz-dev/AI-gateway.git "$HOME/AI-gateway"
cd "$HOME/AI-gateway"
./ops/scripts/init.sh # private .env, CPA and AISIX files (never overwrites)
# edit .env (ADMIN_EMAIL, CLOUDFLARED_TUNNEL_TOKEN) and services/routing/aisix/resources.yaml
./ops/scripts/init-egress-proxy.sh # render data/egress-proxy/policy.json
./ops/scripts/validate.sh
source ./ops/scripts/container-runtime.sh
"${AI_GATEWAY_COMPOSE[@]}" up -d --buildFull steps: docs/installation.md. Day-2 operations: docs/operations.md.
Guides and reference:
- Architecture — full component graph
- Installation — first install and Cloudflare Tunnel
- Security model — boundaries, one-way edges, hardening
- Egress policy — Squid allowlist rules
- Upstream configuration — CPA, AISIX and Sub2API wiring
- Responses path — Pi Codex transports and cache keeper
- WebSocket termination — topology decision and limits
- Model routing — inner selector and deploy checklist
- Provider sidecar — optional sidecar role
- Public API behavior — routes and opaque responses
- Operations and validation — coupled stack, state,
validate.sh - Gateway health and recovery — readiness and bounded recovery
- Ansible operations — playbook tags, variables, guarded rollouts
- model-catalog — catalog service reference
- Model catalog path — front-door intersection and models table
- Models table scheduled refresh
- Add an OpenAI-compatible provider
- Sub2API error passthrough
- Sub2API Codex manifest defects
- Repository layout —
services/andops/tree
Verification evidence: catalog identity, layered metadata (deployment), output-token fields, models-table SSR, Go enricher read cache (historical), Go provider prefix (historical), catalog round closeout.
Dated deployment records (point-in-time, not current state): 2026-09-07 CPA, 2026-09-07 read cache, 2026-09-08 bare chain, 2026-09-08 positive admission, 2026-09-08 provider prefix, 2026-09-08 models-table SSR, 2026-09-08 output-token fields, 2026-09-08 source chain, 2026-09-15 aihub, 2026-09-16 force refresh, 2026-09-30 tunnel 103, 2026-09-30 keepalive late error, 2026-10-01 header wait, 2026-10-09 upgrades (round 2), 2026-10-09 memory/swap, 2026-10-11 upgrades, 2026-10-12 CPA, 2026-10-15 upgrades, 2026-10-20 one-way edges, 2026-10-23 batch.
This deployment template is licensed under the MIT License. Container images and upstream applications retain their own licenses.