The gateway

The gateway is NextOS from your own code: one API key opens an OpenAI-compatible model endpoint and a set of endpoints for running your agents and reading their results. If you already have an OpenAI SDK integration, pointing it at the gateway is usually a one-line change.

See the API reference for the full, generated list of routes, request bodies and response shapes with runnable examples for every endpoint. This guide covers how the pieces fit together.

Getting a key

Create a scoped, revocable API key from Settings > Developer. A key can be limited to only the scopes it needs (for example, read-only access to runs, or the ability to invoke models but not start agent runs), so a key that leaks doesn't hand over more than it has to.

Calling a model

Point any OpenAI-compatible SDK at the gateway's base URL and send your key as a bearer token:

from openai import OpenAI

client = OpenAI(
    base_url="https://www.jonkum.in/api/v1",
    api_key="ngk_live_your_key_here",
)

resp = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Summarize my week."}],
)
print(resp.choices[0].message.content)

const response = await fetch('https://www.jonkum.in/api/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer ngk_live_your_key_here',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'anthropic/claude-sonnet-5',
    messages: [{ role: 'user', content: 'Summarize my week.' }],
  }),
});
console.log(await response.json());

curl -X POST "https://www.jonkum.in/api/v1/chat/completions" \
  -H "Authorization: Bearer ngk_live_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{"model": "anthropic/claude-sonnet-5", "messages": [{"role": "user", "content": "Summarize my week."}]}'

Model ids are provider/model, so you can pin a specific provider. List everything currently routable with GET /v1/models.

Paying for a call

You can bring your own provider key (send it in an X-Provider-Key header, and you pay the provider directly - the gateway never marks it up) or omit it and draw down your plan's hosted allowance, which is metered and billed through NextOS at list price plus a platform markup.

Caching

Two independent caches keep you from paying for the same answer twice:

• Exact-match caching is opt-in per request: send an X-Kumin-Cache-Ttl header with how long a hit should be considered valid, and an identical request within that window is served for free.
• Semantic caching (available to teams, set from Governance policy) serves a cached answer for a request that's merely similar, not identical - useful when many slightly different prompts are really asking the same question.

Routing & fallback

If your team's governance policy allows more than one provider or model, the gateway can fall back automatically when your first choice is unavailable or over budget, retrying safely so a stream never restarts mid-answer once content has already been sent to you.

Structured output

Ask for a specific JSON shape with an OpenAI-style response_format:

{
  "response_format": {
    "type": "json_schema",
    "json_schema": { "name": "weather", "schema": { "type": "object", "properties": { "summary": { "type": "string" } }, "required": ["summary"] } }
  }
}

The gateway validates the reply against your schema and retries once automatically if the model's first attempt doesn't match.

Cost caps

Set a hard ceiling on what a single call may cost with an X-Kumin-Max-Cost-Usd header. By default the call is refused if the estimate exceeds it; add X-Kumin-On-Cap: downgrade to have the gateway automatically substitute the cheapest model that still fits your cap, instead of refusing outright.

Rate limits

Every key has its own request-per-minute and token-per-minute limits, visible and adjustable from Settings > Developer. A team's governance policy can additionally cap the whole org or an individual member, whichever is stricter applies.

Running your own agents from your code

Beyond the model endpoint, the gateway lets you drive the agents you've already configured in NextOS:

• POST /v1/tasks hands the runner a one-off task without a preconfigured agent.
• GET /v1/agents lists the agents you can run server-side; POST /v1/agents/:id/runs starts one.
• GET /v1/runs and GET /v1/runs/:id let you list and poll runs; GET /v1/runs/:id/events streams a run live over Server-Sent Events.

See the API reference for the exact request and response shape of each.

What's next

• Read the full, generated API reference.
• See what leaves your browser and what doesn't.