See inside every RAG answer.

1. 01

   40 chunks in one prompt

   Retrieval is a tool span with its data source, generation a model call with its prompt size, so the trace shows what one question pulled in.

   • workflow_rundocs-qaone question

     one question

     • agent_stepretrieve
       • llm_callembeddingstext-embedding-3-small · 12 tokens

         text-embedding-3-small · 12 tokens

       • tool_callsearch_docs · top 40docs-index

         docs-index

     • agent_stepgenerate
       • llm_call64,800 input tokenschat.completions gpt-5-mini

         chat.completions gpt-5-mini

2. 02

   Past both thresholds

   token_anomaly fires when a call passes 60,000 tokens, or 4× the agent’s p95. This prompt passed both.

   detector

   token_anomaly

   rule

   > 60,000 tokens or > 4× agent p95

   this call

   65,230 tokens

   agent p95, 7 days

   12,100 tokens

3. 03

   Input grows, output doesn’t

   Answer by answer, the input grows with the number of chunks while the output stays the same length.

   answer 1

   5 chunks · 9,640 tokens

   answer 2

   12 chunks · 21,300 tokens

   answer 3

   25 chunks · 43,900 tokens

   answer 4

   40 chunks · 64,800 tokens

4. 04

   A daily token cap

   Put a ceiling on it. A token_limit policy’s daily cap stops the agent once its tokens for the day are spent, before the call reaches your provider; its per‑request cap limits the max_tokens a call may ask for.
   Response

   HTTP/1.1 403 Forbidden{  "error": {    "message": "daily token limit of … for agent reached (used …, policy \"…\")",    "type": "permission_error",    "code": "token_limit_exceeded",    "param": null,    "aiw_request_id": "req_…"  }}

   Example response. The body format is the gateway’s.

SDK packages are distributed by Worksi. Request access. Any OpenAI or Anthropic SDK works with the gateway without them.

import osfrom openai import OpenAI client = OpenAI(    base_url="https://gateway.worksi.ai/v1",    api_key=os.environ["AIW_API_KEY"],    default_headers={"X-AIW-Agent": "docs-assistant"},) def answer(question: str) -> str:    # embeddings are priced and traced like chat calls    query = client.embeddings.create(        model="text-embedding-3-small",        input=question,    )    chunks = search_docs(query.data[0].embedding, top_k=5)    reply = client.chat.completions.create(        model="gpt-5-mini",        messages=build_prompt(question, chunks),    )    return reply.choices[0].message.content or ""

from aiworkforce import AIW aiw = AIW(service="docs-assistant")client = aiw.openai() def answer(question: str) -> str:    with aiw.run("docs-qa", agent="docs-assistant") as run:        with aiw.step("retrieve"):            query = client.embeddings.create(                model="text-embedding-3-small",                input=question,            )            with aiw.tool("search_docs", args={"top_k": 5},                          data_source="docs-index") as t:                chunks = search_docs(query.data[0].embedding)                t.set_result({"chunks": len(chunks)})        with aiw.step("generate"):            reply = client.chat.completions.create(                model="gpt-5-mini",                messages=build_prompt(question, chunks),            )        run.outcome("question_answered", success=True)        return reply.choices[0].message.content or ""