Auto Router

Open-source LLM routing you can self-host. Auto Router gives you one OpenAI-compatible endpoint for many providers and models, with configurable keys, fast routing, and analytics you control.

Built for teams and builders who want provider flexibility without handing their usage data, routing logic, or API keys to another hosted gateway.

What It Does

• Routes chat completion requests across multiple OpenAI-compatible providers.
• Lets you configure providers, models, and API keys from your own deployment.
• Exposes a familiar OpenAI-compatible API surface for easy drop-in usage.
• Tracks request analytics so you can understand model usage, latency, and cost patterns.
• Runs as a self-hosted Next.js app with a Cloudflare Worker in front for low-latency routing.
• Ships as open source under the Apache 2.0 license.

Why Auto Router

LLM infrastructure changes quickly. A model that is best today may be slower, more expensive, rate-limited, or unavailable tomorrow. Auto Router keeps your application pointed at one stable endpoint while letting you change the routing layer behind it.

Use it to:

• Switch providers without changing application code.
• Route traffic by model, provider, availability, or policy.
• Keep API keys in infrastructure you control.
• Compare real-world latency and usage across models.
• Avoid lock-in to a single hosted router or inference gateway.

Core Features

• Multi-provider routing: Connect any provider with an OpenAI-compatible API.
• Model-aware configuration: Define available models and route requests through your preferred providers.
• Configurable API keys: Bring your own provider keys and manage them in your deployment.
• OpenAI-compatible endpoint: Point existing SDKs and tools at Auto Router with minimal changes.
• Near-zero routing overhead: Use Cloudflare Workers for globally distributed request handling.
• Analytics dashboard: View request volume, provider usage, model usage, latency, and routing behavior.
• Self-hostable: Deploy the dashboard and routing layer yourself.
• Open source: Apache 2.0 licensed.

Architecture

Auto Router is split into two parts:

• Next.js dashboard: Configuration, analytics, provider management, and project settings.
• Cloudflare Worker: Lightweight routing layer that receives API requests and forwards them to the selected provider.

This keeps the hot path small. Your application talks to the Worker, the Worker applies routing rules, and provider responses stream back without unnecessary application-server hops.

Your app / SDK
    |
    v
Cloudflare Worker routing endpoint
    |
    v
OpenAI-compatible provider
    |
    v
Selected model

Provider Support

Auto Router is designed around OpenAI-compatible APIs. If a provider supports the OpenAI chat completions format, it should be possible to add it.

Examples of provider types this can support:

• OpenAI-compatible hosted inference providers
• Self-hosted OpenAI-compatible servers
• Local gateways exposed behind a compatible API
• Custom provider adapters that follow the same request and response shape

Quick Start

git clone https://github.com/NazmusSayad/auto-router.git
cd auto-router
pnpm install

Create your environment file:

cp .env.example .env.local

Configure your database, application secrets, and provider keys in .env.local.

Initialize the app:

pnpm run db:init

Run the dashboard locally:

pnpm dev

Deploy the Worker using your Cloudflare account and point your applications at the Worker endpoint.

Example Usage

Because Auto Router exposes an OpenAI-compatible API, existing clients can use it by changing the base URL and API key.

import OpenAI from 'openai'

const client = new OpenAI({
  apiKey: process.env.AUTO_ROUTER_API_KEY,
  baseURL: 'https://your-worker.your-subdomain.workers.dev/v1',
})

const response = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Write a short product update.' }],
})

console.log(response.choices[0]?.message?.content)

Analytics

Auto Router helps you inspect how your LLM traffic behaves in production.

Track:

• Request volume
• Provider distribution
• Model usage
• Latency
• Errors
• Routing decisions

The goal is simple: make routing visible enough that you can improve cost, reliability, and performance without guessing.

Deployment

Auto Router is intended to be deployed with:

• Next.js for the dashboard and control plane
• Cloudflare Workers for the routing plane
• PostgreSQL for persisted configuration and analytics

You own the deployment, keys, database, and request path.

Scripts

• pnpm dev: Start the local development server.
• pnpm build: Build the application.
• pnpm db:init: Initialize the database.
• pnpm lint: Run lint checks.
• pnpm test: Run tests.

Contributing

Contributions are welcome. Good issues and pull requests are practical, focused, and easy to review.

Before opening a pull request:

• Keep changes small and scoped.
• Include tests when behavior changes.
• Update documentation when configuration or usage changes.
• Avoid introducing provider lock-in.

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.