Skip to content
minpeterPublic

About

Kiro proxy gateway

Resources

Stars

31 stars

Watchers

0 watching

Forks

Repository files navigation

Kiro LB

kiro-lb

OpenAI- and Anthropic-compatible gateway for Kiro (Amazon Q Developer / CodeWhisperer), with multi-account load balancing and an operations dashboard.

License

AGPL-3.0 — see LICENSE and NOTICE.md.

Based on jwadow/kiro-gateway (Copyright (C) 2025 Jwadow). This tree adds multi-account routing, dashboard, and related changes (Copyright (C) 2026 minpeter).

Disclaimer

This project is not affiliated with, endorsed by, or sponsored by Amazon Web Services, Inc. or Anthropic. Use of upstream Kiro / Amazon Q services is subject to your AWS / product terms of service. You are solely responsible for how you obtain credentials and for compliance with applicable terms and law.

Quick start

Standalone binary

Download from the releases page:

Platform File
Windows x64 kirolb-windows-x64.exe
Windows ARM64 kirolb-windows-arm64.exe
Linux x64 (static, any distro) kirolb-linux-x64.tar.gz
Linux ARM64 (static, any distro) kirolb-linux-arm64.tar.gz

The Windows files are the executable itself: download and run. The Linux files are a tarball so the executable bit survives the download:

tar -xzf kirolb-linux-x64.tar.gz
./kirolb-linux-x64

On first run it creates .env and .env.example with generated credentials and prints them. Open http://localhost:8000 and add accounts by signing in. Google and GitHub use browser sign-in, which listens on localhost:3128 for the callback (the redirect Kiro allows); if that port is unreachable, paste the address the browser ended on into the dashboard. A device code is still available for headless setups, but approving a second Google/GitHub user that way signs the first one out.

For an organization account, choose Organization (IAM Identity Center), enter your organization's HTTPS start URL and SSO region, and approve the device code in your browser. The dashboard discovers available Kiro profiles in US East (N. Virginia) and Europe (Frankfurt), as kiro-cli does. A single profile is selected automatically; otherwise choose a profile to finish. Your administrator must have assigned access to a Kiro profile. The access token is refreshed in the SSO region, independently of the profile's API region, using the stored OIDC client registration. That registration is kept as issued and is not renewed: when it expires, sign the account in again. No kiro-cli database or filesystem mount is needed.

SHA256SUMS in the release lists every file's checksum.

At startup and every hour, the gateway checks GitHub's latest stable release in the background. A newer release produces a log notice with a download link. Dashboard Info → Service shows the running version and update status; failed checks are shown as unavailable, not up to date. Checks time out after five seconds, send no account credentials, and never install updates automatically. Use Check now in the Info tab to check manually, even when live updates are paused. Concurrent checks share one request, and results less than one minute old are reused to avoid exhausting GitHub's API limit. Opening the tab or refreshing dashboard data alone does not trigger an external check.

On standalone Linux and Windows release binaries (x64/ARM64), Update and restart asks for confirmation, downloads the matching release asset, verifies SHA256SUMS and the candidate's --version, and saves <executable>.previous before replacing the executable. It then drains active requests, flushes runtime state, and restarts with the same arguments, environment and working directory. The dashboard reconnects and reloads the new frontend. The executable's directory must be writable; no privilege escalation is attempted. An install failure keeps the existing process running. A failure to launch the new executable restores the backup and attempts to restart it; a failure occurring after successful launch still requires manual recovery from .previous and is not automatically rolled back. On Windows, a failure during executable replacement disables further installs in that process: restart from the original executable before retrying. If restoring the original path also failed, recover .previous first.

Docker, blue/green managed deployments, debug builds and unsupported platforms only offer checks and update instructions, not executable replacement. Docker must be updated by pulling an image and recreating the container externally. Release binaries expose kirolb --version without opening the database or creating configuration files.

Docker

A multi-arch image (linux/amd64, linux/arm64) is published to GHCR:

docker run -d --name kiro-lb -p 8000:8000   -v "$PWD/data:/app/data" --env-file .env   ghcr.io/minpeter/kiro-lb:main

Or with the bundled compose file, which reads .env and keeps data/ on the host:

docker compose up -d

From source

cd frontend && bun install && bun run build && cd ..
cargo build --release          # target/release/kirolb(.exe)

See .env.example and AGENTS.md for configuration details. Issues: https://github.com/minpeter/kiro-lb/issues

Codex CLI and Claude Code

kirolb client configures either CLI without changing the default Codex profile or requiring a paid inference request for verification. Start the gateway, export one of its data-plane keys, and run setup:

Automatic setup and restore mutate client files on Linux only. They fail closed on macOS and Windows because those platforms do not provide the same verified conditional commit path in this workflow. Read-only diagnose and status remain available there; configure the clients manually using the contracts below. This does not affect running the gateway on those platforms.

export KIROLB_API_KEY='your-gateway-key'
kirolb client setup all --base-url http://127.0.0.1:8000

Use --api-key-stdin to pipe the key instead of putting it in the environment. Use --api-key-env NAME when Codex should read a different environment variable, and --model MODEL to choose the Codex profile's initial model. Never put the key on the command line: command arguments may be visible to other local processes and shell history.

Setup first calls only GET /health and authenticated GET /v1/models. These checks verify reachability, authentication, and model discovery without generating tokens. Discovery must return at least one object with a nonempty string model ID, so setup also detects a gateway with no serving account. Health and discovery JSON responses are limited to 1 MiB each. Nothing is written if either check fails. You can run the same read-only check separately:

kirolb client diagnose --base-url http://127.0.0.1:8000

Plaintext HTTP is accepted only for localhost and loopback IP addresses. If HTTP_PROXY, http_proxy, ALL_PROXY, or all_proxy is set, setup also requires an unambiguous NO_PROXY/no_proxy entry for that exact loopback host in bare form (for example, 127.0.0.1 or ::1) in both the setup environment and the environment used to start the client. Port-qualified, bracketed, wildcard, and case-variant entries are rejected because their behavior is not consistent across clients. Preserved Claude Code env overrides are included in this check. The read-only diagnostic always connects directly for plaintext loopback, so its success alone does not prove that a generated client will bypass a proxy. Use HTTPS for remote gateways so the bearer key is not sent in cleartext.

For Codex, setup writes the dedicated profile $CODEX_HOME/kirolb.config.toml (normally ~/.codex/kirolb.config.toml). Separate profile files require Codex 0.134.0 or later. The profile reads the key from KIROLB_API_KEY; it does not contain the key itself:

codex --profile kirolb

Using an isolated profile is deliberate: setup does not need to parse, reformat, or merge the user's main TOML configuration.

For Claude Code, setup updates ~/.claude/settings.json (or $CLAUDE_CONFIG_DIR/settings.json) with ANTHROPIC_BASE_URL, a bearer token, and gateway model discovery. Existing unrelated JSON settings are retained. The file and the private restoration journal are written with owner-only permissions on Unix. Confirm the active base URL and credential source with /status inside Claude Code.

Inspect or undo setup at any time:

kirolb client status all
kirolb client restore all

Setup is repeatable and serialized across concurrent processes. Writes use a same-directory temporary file and atomic rename. The private journal preserves the exact prior bytes and Unix file mode, including the difference between a missing file and an existing empty file. Restore is idempotent. If a configured file changed after setup, restore refuses to overwrite it; reconcile that file manually rather than deleting the journal. Symlinked configuration files and configuration directories are rejected. Client files and restoration journals are limited to 8 MiB each; setup rejects an oversized generated file or journal before mutating a client file. An interrupted or conflicted operation can retain recovery or .kirolb-*.tmp staging files that preserve original or concurrent bytes needed for reconciliation. Review them before deletion and do not bulk-delete these files by name.

Compatibility limits

The /v1/responses endpoint is a translation facade, not persistent OpenAI Responses storage. Codex must send each turn's complete conversation in input with store=false; previous_response_id is rejected. The generated profile therefore disables standalone web search and WebSocket transport, which this gateway does not expose.

Claude Code uses the Anthropic Messages API and can discover models from /v1/models. The gateway supports Messages streaming and token counting, but it translates requests to Kiro rather than forwarding every current or future Anthropic beta feature unchanged. Features that depend on direct Anthropic services or a claude.ai identity, including cloud sessions and Remote Control, are outside this setup. If a newly introduced Claude Code beta field is rejected, try CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1 and report the incompatibility.

The generated fields and paths follow the current official client contracts:

Credit accounting

Credit metering follows the additive usage-summary policy in official Kiro CLI 2.24.1 (@kiro/agent 0.66.8). Every valid credit or credits event adds to the request total, including repeated values within one upstream response and events from retry or search follow-up generations. A zero contributes zero; it does not erase earlier readings. Missing credits remain unknown, distinct from a measured zero, and each generation's credits stay attributed to its origin account.

This is an explicit official-client compatibility policy, not an independently verified upstream billing contract; see #85 for the source evidence and live-capture limits. Metered credits remain separate from model-cost estimates and cache-token counts. Existing stored totals are not recalculated because individual historical metering events are not retained.

InferX internal engine API

Set INFERX_CONTROL_TOKEN to enable the engine-owned, opt-in API at /internal/inferx/v1/connections/{uuid}. Requests use Authorization: Bearer <token>; this token is intentionally independent from dashboard and /v1 credentials. PUT starts Google/GitHub/AWS Builder ID device login, GET reads status, POST .../poll advances it, and DELETE disconnects it. Every request is additionally bound to the exact ownerId supplied by InferX.

This contract supports the dedicated InferX backend and one running engine instance per SQLite database. Pending device flows expire on restart; registered records remain durable. Marketplace credentials are stored in the existing SQLite database (plaintext at rest, like existing internal credentials) but in a separate table and are never loaded into the unrestricted /v1 account pool. Financial settlement belongs exclusively to InferX's PostgreSQL backend, not this engine or the Cloudflare frontend.

This feature is restricted to a dedicated instance with an empty operator account pool; connection starts reject existing operator sources or accounts. Do not add operator accounts later or expose the dashboard and /v1 publicly. Protect the database and backups; managed encryption is still needed before a public marketplace rollout. Disconnect erases stored credentials, not upstream OAuth grants. Registered status checks read saved metadata, not current upstream health; credentials refresh on demand for seller-scoped serving or diagnostics. Continuous health checks are not included. Initialize a fresh dedicated database for this contract; there is no compatibility path for earlier prototype schemas.

PUT takes {ownerId, provider} with provider equal to github, google, or builder-id; polling takes {ownerId}. GET and DELETE take ownerId in the query. Responses expose only {id, status, authorization, account, diagnostics, models?}. Authorization has url, userCode, expiresAt (Unix milliseconds), and intervalSeconds; account has the verified upstream id and nullable email. Both are nullable. Status is pending, registered, expired, failed, or disconnected. Neither the control token nor provider tokens belong in browser requests.

Registered responses optionally include models: string[] (at most 100 unique IDs), discovered with that seller's authenticated ListAvailableModels request at registration and recheck. IDs are preserved verbatim, including newly released models; plan names, aliases and configuration fallbacks never establish access. Unknown discovery omits models; a successful empty catalogue returns [] and revokes all previous support. Successful refresh replaces, rather than merges, the catalogue, so downgrades remove old models. Failed, malformed, timed-out or oversized discovery preserves the last authoritative catalogue (or remains omitted if none exists). Usage diagnostics are independent of model discovery. GET and cooldown responses use persisted models without upstream requests. Models are owner-scoped, persisted via an additive SQLite column, and cleared on disconnect under the same cancellation/credential-refresh fence.

diagnostics is additive and nullable (null until checked), independent of the lifecycle status: an unhealthy account remains registered. Its shape is:

{
  checkedAt: number; // Unix milliseconds, latest completed diagnostic check
  health: "unknown" | "healthy" | "authentication_failed" | "temporarily_suspended";
  usage: null | {
    plan: string | null;
    used: number | null;
    limit: number | null;
    resetsAt: number | null; // Unix milliseconds
    overage: "enabled" | "disabled" | "unknown";
    updatedAt: number; // Unix milliseconds, latest successful usage reading
  };
}

POST /internal/inferx/v1/connections/{uuid}/recheck takes {ownerId} and returns the usual connection response. It refreshes credentials as needed and calls Kiro's usage and model discovery APIs without generating tokens. A persisted 60-second cooldown per connection returns cached results for repeated requests, including after failed checks or process restarts. The cooldown runs from check completion; an initial claim also prevents immediate retries if the process exits mid-check. Upstream work is bounded to 30 seconds after acquiring the connection lock; it serializes with inference and disconnect. Non-registered connections return their cached response without contacting Kiro. Missing and other-owner UUIDs return the same 404 body.

GET never contacts Kiro. Registration saves its already-fetched usage. Definitive inference credential rejection or suspension also updates health without replacing usage. A management 401 gets one credential refresh and retry before being treated as authentication failure. Explicit suspension reasons/messages are distinct from credential rejection; generic management 403, throttling, timeouts and other unclassified failures yield unknown. Failed diagnostic checks preserve the last successful usage and its updatedAt; missing upstream fields are null, not zero. healthy means the management check succeeded, not that every model is available or that quota remains. Diagnostics contain only the fields above, never raw errors, tokens or profile ARNs. They persist separately with the connection using additive SQLite columns. Disconnect clears diagnostics and credentials and fences delayed work; a cancelled HTTP recheck still saves any rotated credential before releasing that fence. Database and backup protection requirements above also apply here.

Retry ambiguous requests with the same canonical UUID. Ownership is immutable, including after disconnection. Deleting an unknown UUID creates a tombstone to block late starts. Use a new UUID after a terminal state. Verification uses the upstream user ID, not the shared social-login profile ARN or mutable email.

POST /internal/inferx/v1/requests/{uuid} accepts {ownerId, connectionId, request} with text-only OpenAI chat parameters. It durably admits a request ID once and uses only the specified seller connection. The backend must reserve funds before dispatch. stream:true forwards native OpenAI SSE chunks followed by a private inferx.receipt event after persistence; the engine omits [DONE] so the backend can settle before declaring success. Channel saturation emits an error instead of falsely successful truncated output. Execution and receipt persistence continue after a client disconnect. stream:false returns a receipt with a one-time completion; content is not stored.

GET .../requests/{uuid}?ownerId=… reads a saved receipt. DELETE reads it or durably fences an unknown UUID before the backend releases its hold. A late POST cannot execute a fenced request. Restart marks interrupted execution indeterminate; the backend does not bill unverified completions. Preserve the receipt database and drain admitted work on shutdown. Usage is tokenizer-estimated, not an upstream invoice. The InferX output limiter measures text, reasoning and complete tool calls before forwarding. It emits a safe UTF-8 prefix, omits any tool call that cannot fit, drops the upstream body at the budget boundary and succeeds with finish_reason: "length" plus bounded usage. A required tool may be absent at this boundary; the safe prefix need not fill the budget. The backend settles this receipt normally, instead of refunding already-delivered output. Local cancellation does not promise an upstream quota refund. Never expose these endpoints directly to buyers.

Release maintenance

Release-plz opens or updates a release PR after changes reach main. Review its version and changelog, then merge it to approve a release. Use Conventional Commits (fix:, feat:, and ! / BREAKING CHANGE:) for useful release notes. The baseline is v0.2.0; versions and tags are not reset. This is GitHub-only distribution: git_only = true disables publishing to crates.io.

No personal access token or release secret is required. In Settings → Actions → General → Workflow permissions, enable Allow GitHub Actions to create and approve pull requests; keep the default token permissions read-only. The release PR job grants its short-lived GITHUB_TOKEN Contents and Pull requests write access, plus Actions write access to explicitly dispatch validation. It does not approve or merge PRs.

After creating/updating a release PR, the job dispatches Build on that PR's branch. This avoids depending on bot-generated PR events, which may require approval. Dispatched builds check the branch commit and cannot publish Docker images or releases, even when manually dispatched on a tag. Review the Build checks on the release PR before merging. No dispatch is made when release-plz reports no PR changes. The old RELEASE_PLZ_TOKEN secret is no longer read and can be removed after verifying the token-free workflow.

The release workflow uses the default GITHUB_TOKEN to create a tag and a draft release, then directly calls the existing build workflow for that exact tag. It does not depend on a bot-created tag triggering another workflow. Rust/frontend checks and all four binary builds must pass before upload. The publisher downloads the uploaded assets, verifies SHA256SUMS, and only then makes the release public. Published releases are never overwritten.

If a release build or upload fails, the draft remains unpublished. Re-run failed jobs, or run Release automation manually from main with the existing draft tag (for example v0.2.1). Recovery rejects published releases and tags outside main history. It rebuilds the tagged source using the selected workflow's packaging logic. Do not move or reuse published version tags.

The existing main Docker build remains independent; automatic binary releases do not add a versioned Docker image or deploy/restart any running server.

About

Kiro proxy gateway

Resources

Stars

31 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages