VI
AI
Live

SPARK AI Gateway

Self-hosted, OpenAI-compatible API gateway that brokers between multiple tenants and OpenClaw agents.

Design & development

4

Products using it

The problem

Many products in the SPARK DEV ecosystem need their own AI assistant — the Core CMS store chatbot, the SPARK Finance financial assistant, the OrderManager group-chat agent, the assistant on this very portfolio. Calling an LLM provider directly from each app means every app handles its own API keys, rate limits, conversation history and tools. SPARK AI Gateway brings all of that into one place.

The solution

  • OpenAI-compatible API (/v1/chat/completions, /v1/models): any app that speaks the OpenAI standard can use it — just change the base URL and key.
  • Broker between multiple tenants and OpenClaw agents: each app has its own sk_… key, and each conversation is tied to an agent session, so the agent remembers context without the app managing history itself.
  • Model routing via 9Router to Claude, GPT or Gemini.
  • SSE streaming with the spark_status extension (the agent is thinking / using a tool) and spark_attachment (files produced by tools, stored on Cloudflare R2).
  • Security & operations: HMAC + API key authentication, Redis-backed rate limiting, message history trimming, audit logs, Prometheus metrics.

Architecture

  • NestJS 11 on Node 22, PostgreSQL 16 (TypeORM), BullMQ queues on Redis, pino logging.
  • Split into 3 deployments: the API, a run-processing worker (kept at 1 replica to preserve ordering) and the admin UI.
  • Admin UI: agents, apps, keys, sessions, runs, tools, usage, audit logs, users, playground.

In production

The financial assistant (SPARK Finance), the group-chat agent (OrderManager), the store chatbot (Core CMS) and the "Ask about Bình" assistant on devbyspark.com.