Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,12 @@
"homepage": "https://zuplo.com/docs",
"repository": "https://github.com/zuplo/tools",
"skills": [
"./skills/zuplo-ai-gateway",
"./skills/zuplo-cli",
"./skills/zuplo-graphql",
"./skills/zuplo-guide",
"./skills/zuplo-monetization"
"./skills/zuplo-monetization",
"./skills/zuplo-project-setup"
],
"source": "./",
"strict": false
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/plugin-validation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,5 +51,5 @@ jobs:
printf '%s\n' "$zuplo_details"
printf '%s\n' "$zudoku_details"

grep -q 'Skills (3).*zuplo-cli, zuplo-guide, zuplo-monetization' <<< "$zuplo_details"
grep -q 'Skills (6).*zuplo-ai-gateway, zuplo-cli, zuplo-graphql, zuplo-guide, zuplo-monetization, zuplo-project-setup' <<< "$zuplo_details"
grep -q 'Skills (1).*zudoku-guide' <<< "$zudoku_details"
3 changes: 3 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ The **Zuplo Agent Skills** repository provides official agent skills for coding
| `skills/zuplo-guide/` | Comprehensive Zuplo API gateway skill with docs and policies |
| `skills/zuplo-monetization/` | API monetization setup, billing, Stripe integration |
| `skills/zuplo-cli/` | Zuplo CLI command reference and usage guide |
| `skills/zuplo-project-setup/`| Guided gateway setup — starting point, then the core policies |
| `skills/zuplo-graphql/` | GraphQL endpoint — proxy route plus Zudoku docs plugin |
| `skills/zuplo-ai-gateway/` | AI Gateway — providers, teams, apps, policy chains, budgets |

## Specification

Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ Install via **Cursor Settings > Rules > Add Rule > Remote Rule (GitHub)** and en
.cursor/skills/
├── zuplo-guide/SKILL.md
├── zuplo-cli/SKILL.md
├── zuplo-project-setup/SKILL.md
├── zuplo-ai-gateway/SKILL.md
├── zuplo-graphql/SKILL.md
├── zuplo-monetization/SKILL.md
└── zudoku-guide/SKILL.md
```
Expand Down Expand Up @@ -131,6 +134,9 @@ If local docs aren't available and MCP is not configured, skills fall back to fe
| **zuplo-guide** | Comprehensive gateway guide — documentation lookup, request pipeline, route/policy configuration, custom handlers, deployment. Start here for general Zuplo development. |
| **zuplo-monetization** | API monetization — meters, plans, Stripe billing, subscriptions, usage tracking, private plans, tax collection. |
| **zuplo-cli** | CLI reference — local dev, deployment, env vars, tunnels, OpenAPI tools, mTLS, project management. |
| **zuplo-project-setup** | Guided setup for a new or unconfigured gateway — starting point (template, OpenAPI import, or scratch), then API key auth, rate limiting, CORS, and JWT validation in order. |
| **zuplo-graphql** | GraphQL endpoints — the `x-graphql` proxy route plus the Zudoku GraphQL documentation plugin, which must be set up together. |
| **zuplo-ai-gateway** | AI Gateway — providers, teams and apps, `providerLabel/model` addressing, an app's policy chain and request URL, cascading budgets, and diagnosing a rejected AI request. |

### Zudoku (Developer Portal)

Expand Down
166 changes: 166 additions & 0 deletions skills/zuplo-ai-gateway/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
---
name: zuplo-ai-gateway
description: "Use when working with a Zuplo AI Gateway project — connecting LLM providers, organizing teams and apps, wiring an app's policy chain (model filtering, fallback model, budgets, semantic cache), setting usage limits, or calling the gateway from application code. Covers how models are addressed as providerLabel/model, where an app's request URL comes from, how budgets cascade, and how to diagnose a rejected AI request."
license: MIT
metadata:
author: Zuplo
version: "1.0.0"
repository: https://github.com/zuplo/tools
---

# Zuplo AI Gateway

One gateway in front of every LLM an organization calls: provider keys held in
one place, usage metered and capped per team and per app, and an
OpenAI-compatible surface so client SDKs need only a base URL change.

## Critical rule: Read docs before configuring

Read the relevant page before writing configuration — the AI Gateway's policy
names and metadata shapes are exact, and a guess breaks the app's policy chain.
Sources in priority order:

1. **Local docs (preferred):** `node_modules/zuplo/docs/ai-gateway/` when the
`zuplo` package is installed. Check the project root and parent directories.
2. **MCP server tools:** `search-zuplo-docs` and `ask-question-about-zuplo` if
the Zuplo MCP server is connected (names may be prefixed, e.g.
`mcp__*Zuplo*__search-zuplo-docs`).
3. **Fetch by URL:** `https://zuplo.com/docs/ai-gateway/overview` and the pages
under `https://zuplo.com/docs/ai-gateway/`.

## The three layers

| Layer | What it is | What it carries |
| ------------- | -------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| **Providers** | The LLM vendors the gateway may route to (OpenAI, Anthropic, Google, Bedrock, Vertex, an OpenAI-compatible custom endpoint…) | The vendor API key, and the models the gateway may reach through it |
| **Teams** | A hierarchy that groups apps and the people who own them | Shared usage limits, and who may access and manage the apps beneath |
| **Apps** | The pieces of software that call the gateway — a chatbot, an internal agent | Its own request URL, its own Zuplo-managed API key, its own policy chain, and the team it belongs to |

All three live inside one AI Gateway project. Read the layer a question is
about rather than guessing: which models are reachable is a *provider* question,
who is spending is a *team* and *app* question, why one request was rejected is
an *app policy chain* question.

## Models are addressed `providerLabel/model`

**The prefix is the name given to the provider configuration, not the vendor.**
A provider labelled `openai` yields `openai/gpt-5-mini`; label the same vendor
`openai-eu` and its models are `openai-eu/gpt-5-mini`. Two configurations of one
vendor — different keys, regions or accounts — are told apart by exactly this.
Comment on lines +46 to +49

Take model ids from the gateway's own supported-model catalog, never from
memory: a model the gateway does not list is not routable, and a plausible id
you invented fails at request time rather than at configuration time.

Unrestricted, an app may call any model reachable through the project's
providers, and each request names its own in the `model` field.

## Calling an app

Each app has its own API URL, containing that app's id — the gateway attributes
usage from it. **Read that URL from the app itself; never construct it by hand
or borrow another app's.** The hostname is the deployed environment's, the path
segment identifies the app, and both come from the app, not from a pattern.

Under it sit the OpenAI-compatible services:

| Service | Path |
| ---------------- | --------------------- |
| Chat Completions | `v1/chat/completions` |
| Responses | `v1/responses` |
| Messages | `v1/messages` |

```bash
curl https://<gateway-host>/<app-id>/v1/chat/completions \
-H "Authorization: Bearer $APP_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-5-mini",
"messages": [{ "role": "user", "content": "Hello, world!" }]
}'
```

Client SDKs generally want a base URL ending in `/v1`, so point the SDK at
`https://<gateway-host>/<app-id>/v1` and let it append its own paths. The app's
API key goes in `Authorization: Bearer …`, and is enforced when the
authentication policy is in the app's chain.

## The app's policy chain

An app runs an ordered chain, each entry chosen from the gateway's policy menu,
before the handler calls the provider. An entry can short-circuit the request —
a cache hit answers it, a guardrail blocks it.

| Policy | Declaration | What it does |
| -------------------- | ---------------------------------------- | -------------------------------------------------------- |
| Model Filtering | `ai-gateway-model-filtering-v2-inbound` | Restricts which models the app may call |
| Fallback Model | `ai-gateway-fallback-model-v2-inbound` | Routes to a fallback when a budget is spent or upstream fails |
| Budgets and Costs | `ai-gateway-metering-v2-inbound` | Meters cost, tokens and requests, and enforces the app's own limits |
| Semantic Cache | `ai-gateway-semantic-cache-v2-inbound` | Serves a semantically equivalent cached response |
| Authentication | `ai-gateway-auth-v2-inbound` | Enforces the app's API key |

**The recommended order is Model Filtering → Fallback Model → Budgets and Costs
→ Semantic Cache, and the order is load-bearing:** budgets after the fallback so
an exceeded budget can route to the fallback model instead of failing, and the
cache after budgets so cache hits still count toward request limits.
Comment on lines +102 to +105

A Model Filtering allow list does double duty: it restricts the models **and its
first entry becomes the default** when a request omits `model`.

## Usage limits cascade

Limits exist at three levels, and **all of them apply at once — a request is
blocked when any one is exceeded**:

| Level | Covers | Configured at |
| ----------- | ---------------------------------------- | ---------------------------------------------- |
| **Gateway** | Every team and app in the project | Settings → Usage Limits |
| **Team** | All of that team's apps together | The team's Usage & Limits tab |
| **App** | That one app | The Budgets and Costs policy in its chain |

Gateway and team limits are **inherited by every app beneath them**, whether or
not the app has Budgets and Costs in its own chain — so an app with no budget
policy is still capped. The app's policy is the only place per-metadata budgets
live (a budget per user or per customer, keyed on an expression).

Each rule chooses a scope (the whole app, or one budget per distinct metadata
value) and carries limit rows of **meter** (cost, tokens, requests) × **period**
(hourly, daily, weekly, monthly) × **limit** × **action** (block or warn). A
warn row and a block row on the same meter and period give warn-then-block.

Over a limit, a request goes to the app's quota fallback model if one is
configured; otherwise it is rejected with **HTTP 429** and a `budget` object
naming what ran out.

## Keys and secrets

- **The vendor's API key belongs to the provider configuration**, stored as a
project secret. It is never in application code, a request header, or a
committed file — the point of the gateway is that callers never hold it.
- **An app's key is issued and rotated by Zuplo.** Application code carries only
that app key, which the gateway trades for provider access.
- Never echo a key value back to a user or write one into a file. Refer to the
environment variable or the app, not the secret.

## Diagnosing a rejected request

| Symptom | Where to look |
| ------- | ------------- |
| "No provider" / model not available | The provider layer: is a provider connected, and does it enable that model? Check the `providerLabel/` prefix matches a provider's actual label |
| `429` with a `budget` object | Which level ran out — gateway, team, or app. The object names the meter and period; a quota fallback model turns this into a slower answer instead of a failure |
| The model is ignored or substituted | Model Filtering's allow list — its first entry is the default when `model` is omitted |
| `401` / `403` | The app's API key, and whether the authentication policy is in that app's chain |
| A `404` on the request URL | The URL was constructed rather than read from the app. Get it from the app itself |
| Costs attributed to the wrong app | Two callers sharing one app URL — usage is attributed by the app id in the URL, so each caller needs its own app |

## Getting it wrong

| Mistake | Consequence |
| ------- | ----------- |
| Prefixing a model with the vendor instead of the provider's label | The model does not resolve, and the error looks like a missing provider |
| Inventing a model id instead of reading the catalog | Configuration saves, then every request using it fails |
| Constructing an app's request URL from a pattern | A hard 404, or usage billed to another app |
| Putting Budgets and Costs before the Fallback Model | An exceeded budget fails the request instead of falling back |
| Putting Semantic Cache before Budgets and Costs | Cache hits escape the request count |
| Assuming an app with no budget policy is uncapped | Gateway and team limits still apply to it |
| Putting a vendor API key in application code | The secret the gateway exists to hold is now in your repo |
134 changes: 134 additions & 0 deletions skills/zuplo-graphql/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
---
name: zuplo-graphql
description: "Use when adding or fixing a GraphQL endpoint on a Zuplo API gateway, which always means two things together — a proxy route in routes.oas.json marked x-graphql so GraphQL-aware policies apply, and the Zudoku GraphQL plugin registered in the developer portal config so the schema and playground are documented. Covers the route handlers, the plugin options, and the GraphQL-specific policies."
license: MIT
Comment on lines +2 to +4
metadata:
author: Zuplo
version: "1.0.0"
repository: https://github.com/zuplo/tools
---

# Zuplo GraphQL Endpoints

A GraphQL API on Zuplo is **two things that must be set up together**:

1. **The gateway endpoint** — a route that proxies GraphQL requests to the
upstream API.
2. **The documentation plugin** — the Zudoku developer portal plugin that
renders the schema and an interactive playground.

**The developer portal does not document GraphQL automatically the way it
documents OpenAPI routes.** OpenAPI operations come from `routes.oas.json`,
which the portal already reads; a GraphQL schema lives behind the endpoint and
has to be introspected by a plugin. So a GraphQL endpoint is not done until
BOTH parts are in place. Never add the gateway route without also adding the
documentation plugin.

## Read the docs first

Before writing any configuration, read `https://zuplo.com/docs/articles/graphql`
— it is the source of truth for both halves. Prefer local docs when the `zuplo`
package is installed (`node_modules/zuplo/docs/articles/graphql.md`), or the
Zuplo MCP server's `search-zuplo-docs` if it is connected. Do not configure
GraphQL from memory.

Then read the project's current state before changing it: the existing routes
in `config/routes.oas.json` and the portal config in
`docs/zudoku.config.tsx`.

## 1. Find the upstream

Determine the upstream GraphQL URL to proxy to, and the path to mount the
endpoint on (`/graphql` by default). Ask the user for both if they have not
said — never invent an upstream URL.

## 2. Add the gateway route

Add the route to `config/routes.oas.json` under `post` (GraphQL is a POST API),
marked `"x-graphql": true`, with a proxy handler:

- `urlRewriteHandler` — rewrite the incoming path to the full upstream GraphQL
URL. Use this for a single fixed endpoint.
- `urlForwardHandler` — forward to a base URL, preserving the path. Use this
when the upstream shares a base with other routes.

```json
{
"paths": {
"/graphql": {
"post": {
"summary": "GraphQL Endpoint",
"operationId": "graphql",
"x-graphql": true,
"x-zuplo-route": {
"handler": {
"module": "$import(@zuplo/runtime)",
"export": "urlRewriteHandler",
"options": { "rewritePattern": "https://api.example.com/graphql" }
},
"policies": { "inbound": [] }
}
}
}
}
}
```

**`"x-graphql": true` is required**, not decorative: it is how the gateway
knows the route carries GraphQL, and the GraphQL-aware policies below only
apply to routes that have it.

### GraphQL-specific policies

| Policy | What it does |
| ---------------------------------------- | ------------------------------------------------------------- |
| `graphql-analytics-outbound` | Reports the operations being called to analytics |
| `graphql-disable-introspection-inbound` | Blocks introspection queries — usual for a production endpoint |

Read each policy's doc page before configuring it. Note the interaction with
step 3: if introspection is disabled on the gateway, the documentation plugin
cannot introspect the schema through it — point the plugin at a schema file or
at the upstream directly.

## 3. Add the documentation plugin (never skip)

Read `docs/zudoku.config.tsx` and **merge** the plugin into the existing config
— do not overwrite the file. Import `graphqlPlugin` from
`@zudoku/plugin-graphql` and add an instance to the config's `plugins` array,
creating the array if it does not exist. Add **one plugin instance per GraphQL
API**.

```tsx
import { graphqlPlugin } from "@zudoku/plugin-graphql";

// merge into the existing config object:
plugins: [
graphqlPlugin({
type: "url", // introspect the schema from the input URL
input: "https://api.example.com/graphql",
path: "/graphql", // where the docs + playground mount in the portal
options: { title: "GraphQL API" },
}),
],
```

If `@zudoku/plugin-graphql` is not already a dependency in
`docs/package.json`, add it there too — the portal build fails on the missing
import otherwise.

## 4. Verify and deploy

Build the project and read the result; if it fails, read the logs, fix the
configuration, and build again. Then summarize what was configured and
**confirm both halves explicitly** — the gateway route and the documentation
plugin — before offering to deploy.

## Getting it wrong

| Mistake | Consequence |
| ------- | ----------- |
| Route without the documentation plugin | The endpoint works but is undocumented, with no playground — the portal shows nothing |
| Omitting `"x-graphql": true` | GraphQL-aware policies do not apply to the route |
| Overwriting `docs/zudoku.config.tsx` | Every other portal setting is destroyed |
| Adding the plugin without the dependency in `docs/package.json` | The portal build fails on the missing import |
| Disabling introspection while the plugin introspects through the gateway | The schema cannot be loaded and the docs render empty |
Loading
Loading