Chat and copilots, under control.
Keep streaming fast while every conversation is priced, attributed to one of your users, held to their plan’s limits and stored only as much as you allow.
import osfrom openai import OpenAI client = OpenAI( base_url=import OpenAI from "openai" const client = new OpenAI({ baseURL: "https://gateway.worksi.ai/v1", apiKey: process.env.AIW_API_KEY,}) export async function answer( userIdHash, userPlan, messages,) { const headers = { // a pseudonymous id, not an email "X-AIW-End-User": userIdHash, // your own plan name, e.g. "free" "X-AIW-End-User-Plan": userPlan, } const res = await client.chat .completions.create( { model: "gpt-5-mini", messages }, { headers }, ) return res.choices[0]?.message.content}A conversation that outgrew its budget.
Each turn sends the whole conversation again, and this user keeps pasting long documents into it, so the prompt balloons while the answers stay short. This is how Worksi records the growth, flags the call and holds the user to a limit.
Example values. Detector rules are the product defaults, and the response is in the gateway’s format.
- 01
Four turns, one growing prompt
Every call of the conversation is attributed to support-bot and one end user, and every turn sends all the turns before it again, plus what the user pasted.customer-chat · one conversation - workflow_runcustomer-chatsupport-bot · one end user
support-bot · one end user
- llm_callturn 1 · 1,420 tokenschat.completions gpt-5-mini
chat.completions gpt-5-mini
- llm_callturn 2 · 3,910 tokenschat.completions gpt-5-mini
chat.completions gpt-5-mini
- llm_callturn 3 · 19,150 tokenschat.completions gpt-5-mini
chat.completions gpt-5-mini
- llm_callturn 4 · 71,170 tokenschat.completions gpt-5-mini
chat.completions gpt-5-mini
- 02
An outlier for this agent
token_anomalyfires when a call passes 60,000 tokens, or 4× the agent’s p95. In our benchmark, context stuffing opened an incident 0.97 s after the call (measured — how we measured time to detect).Incident · detector - detector
- token_anomaly
- rule
- > 60,000 tokens or > 4× agent p95
- this call
- 71,170 tokens
- severity
- medium
- 03
Input up, answers flat
The call is measured against the agent’s own seven‑day baseline. The input grew with every paste while the answers stayed about the same length.Incident · evidence - input tokens
- 70,900 (turn 1: 1,180)
- output tokens
- 270, flat
- agent p95, 7 days
- 9,800 tokens
- this call
- 7.3× the p95
- 04
Over the plan: 429
Give every user a plan. The free plan allows 50,000 tokens a day, and the fourth turn took this user past it. The next call gets 429 withRetry-After, and your provider is never called.Response# the request carriesX-AIW-End-User: <hash of your user id>X-AIW-End-User-Plan: free # the gateway answersHTTP/1.1 429 Too Many RequestsRetry-After: <seconds until the limit resets>{ "error": { "message": "end user \"u_…\" reached the tokens per day limit of plan \"free\" (95650 of 50000; resets …, policy \"Plan limits for app users\")", "type": "rate_limit_error", "code": "end_user_quota_exceeded", "param": null, "aiw_request_id": "req_…", "details": { "plan": "free", "limit": "tokens_per_day", "window": "day", "max": 50000, "current": 95650, "resets_at": "…" } }}Example response. The body format is the gateway’s.
What else watches a conversation.
Five of the eleven built‑in detectors, with the conditions they fire on.
| Detector | Fires when |
|---|---|
Token anomalytoken_anomaly | A call uses far more tokens than the agent normally does |
Provider failureprovider_failure | A provider or model’s error rate spikes, or calls fail in a row |
Timeouttimeout | Model or tool calls time out or run past their thresholds |
Error rateerror_rate | An agent’s error rate crosses the threshold over a window |
Cost anomalycost_anomaly | Cost per request jumps versus the agent’s 7-day baseline, or a run exceeds its cost limit |
Stream answers, per user.
Streams pass straight through the gateway, and usage is recorded for every one. The SDK adds a run and an outcome around each answer.
Pro fits most chat products: 1M gateway requests and 10M spans a month, 30‑day retention, Slack and webhook alerts. $49 / month.
SDK packages are distributed by Worksi. Request access. Any OpenAI or Anthropic SDK works with the gateway without them.
import osfrom openai import OpenAI client = OpenAI( base_url="https://gateway.worksi.ai/v1", api_key=os.environ["AIW_API_KEY"],) stream = client.chat.completions.create( model="gpt-5-mini", messages=conversation, stream=True, extra_headers={ "X-AIW-End-User": user_id_hash, "X-AIW-End-User-Plan": user_plan, },)# chunks pass through as they arrive; usage is recordedfor chunk in stream: if chunk.choices: send(chunk.choices[0].delta.content or "")from aiworkforce import AIW, EndUserQuotaExceededError aiw = AIW(service="support-bot")client = aiw.openai() def reply(user: str, plan: str, conversation: list) -> None: # every call in the run carries the end user and plan with aiw.run("customer-chat", agent="support-bot", end_user=user, end_user_plan=plan) as run: try: stream = client.chat.completions.create( model="gpt-5-mini", messages=conversation, stream=True, ) except EndUserQuotaExceededError as err: # over the plan's limit: offer an upgrade instead send(upgrade_prompt(err.plan, err.resets_at)) return for chunk in stream: if chunk.choices: send(chunk.choices[0].delta.content or "") run.outcome("question_answered", success=True)Put a limit on every user.
Send a pseudonymous user id with each call, then give each of your plans its limits in one policy.
- 3‑day trial with Pro features, no card.
Keep readingB2C appsOpenAI‑compatible APICapture modes
curl -N https://gateway.worksi.ai/v1/chat/completions \ -H "Authorization: Bearer $AIW_API_KEY" \ -H "X-AIW-End-User: $USER_ID_HASH" \ -H "Content-Type: application/json" \ -d '{"model": "gpt-5-mini", "stream": true, "messages": [{"role": "user", "content": "Hi"}]}'