Per-customer policies

Beta: The LLM Gateway is in beta.

A spend policy can separate a default limit by custom request header, so each header value gets an independent limit. Spend and rate limit policies can also match one specific header value. Use these options to cap your own end customers, tenants, or teams without issuing a separate LangSmith API key for each one. For example, a default workspace spend limit separated by X-Gateway-Customer-Id gives acme and globex independent limits within each workspace. An explicit policy with the condition X-Gateway-Customer-Id: acme limits only requests that carry that exact value.

Matchable headers

The gateway matches on request headers prefixed with X-Gateway-, and on keys inside the X-Gateway-Metadata JSON header. No other request header is matchable. Header names are normalized before matching: the X-Gateway- prefix is stripped, the remainder is lowercased, and every character outside a-z, 0-9, and _ is replaced with _. The headers X-Gateway-Customer-Id, x-gateway-customer_id, and X-Gateway-CUSTOMER.ID all resolve to the matcher key customer_id. Header values are compared as exact, case-sensitive strings, with no wildcard or pattern matching. The gateway stamps caller identity itself and ignores client attempts to override it. Headers that resolve to organization_id, workspace_id, workspace_handle, user_id, user_email, api_key_id, api_key_short, auth_mode, user_agent, applied_policy_ids, or applied_policy_names, and any header whose normalized name starts with gateway, are discarded.

The gateway trusts the X-Gateway-* headers on an incoming request. Set the header in your own backend after you authenticate the end user, and do not distribute the gateway API key to end users. A caller that controls both the key and the header can choose which limit to use.

Creating and managing policies requires the organization:manage permission. For the full permissions breakdown, refer to Traces, Engine, and access control.

Separate a default spend limit by header

A default spend limit applies the same cap to every member of a subject dimension. Separating it by a header applies that cap independently to every subject and header-value pair, without requiring a policy for each value. Default spend bucketing follows these rules:

One header per default: Enter the header name only. The request supplies the value that identifies the bucket.
Independent limits: Each subject and header-value pair receives the configured spend limit.
Fallback limit: Requests without the configured header share a fallback limit for their subject.

To separate a default spend limit by header:

Go to Settings → Gateway → LLM Gateway and select Cost Controls.
Click Create spend limit.
Select Workspace, User, or API Key, then select the option to apply the limit to every subject of that type by default.
Select Separate limits by custom header.
Enter the Header name without its X-Gateway- prefix. For example, enter Customer-Id for the X-Gateway-Customer-Id request header.
Set the spend limit, then click Create spend limit.

The policy table displays the header used to separate the default limit. You can also edit an existing default spend limit to add, change, or remove the header. Use an explicit policy instead when different header values need different limits.

Add an explicit header condition

An explicit spend or rate limit policy can match one exact header value. Use an explicit policy to assign different limits to different header values. Explicit header conditions follow these rules:

One condition per policy: A policy accepts one header name and one value.
One subject scope: Combine the condition with an organization, workspace, user, or API key scope. The subject side accepts several values and matches any of them.
Missing headers do not match: A request without the configured header does not match the policy.
Every matching policy is enforced: A request that matches both a plain subject policy and a policy with a header condition counts against both, and either one can block it.
At most 10 conditions: A policy carries no more than 10 subject conditions in total.

Go to Settings → Gateway → LLM Gateway.
Click Create policy.
Select the policy type and subject scope, then set the limits.
Under Custom header condition (optional), enter the Header name without its X-Gateway- prefix (for example, Customer-Id) and the Header value to match (for example, acme).
Save.

You cannot edit the header condition in the UI after the policy is created. To change it, delete the policy and create a new one, or update subject_matchers through the API.

Cap spend per end customer

A reseller or multi-tenant application usually calls the gateway from its own backend, using one workspace-scoped API key on behalf of many end customers. Use an explicit header condition when each customer needs a different cap. If every customer uses the same cap, separate one default spend limit by header instead.

Step 1. Send a customer header on every call

Attach the header to each request your backend makes on behalf of an end customer:

curl https://gateway.smith.langchain.com/openai/v1/chat/completions \
    -H "Authorization: Bearer $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -H "X-Gateway-Customer-Id: acme" \
    -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"ping"}]}'

import os

from openai import OpenAI

client = OpenAI(
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["LANGSMITH_API_KEY"],
)
customer_id = "acme"
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "ping"}],
    extra_headers={"X-Gateway-Customer-Id": customer_id},
)
print(response.choices[0].message.content)

If your LangSmith account is on a regional instance, use the corresponding regional gateway.

Step 2. Create a cap for one customer

Create one spend policy per end customer through the LangSmith REST API:

curl -X POST "https://api.smith.langchain.com/v1/platform/gateway-policies" \
    -H "X-Api-Key: $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
          "name": "customer-acme-monthly-cap",
          "policy_type": "spend_cap",
          "action": "block",
          "subject_matchers": [
            {"key": "workspace_id", "value": "0b1c2d3e-4f56-7890-abcd-ef1234567890"},
            {"key": "customer_id", "value": "acme"}
          ],
          "config": {"window": "monthly", "limit_usd": 250}
        }'

import os

import httpx

response = httpx.post(
    "https://api.smith.langchain.com/v1/platform/gateway-policies",
    headers={"X-Api-Key": os.environ["LANGSMITH_API_KEY"]},
    json={
        "name": "customer-acme-monthly-cap",
        "policy_type": "spend_cap",
        "action": "block",
        "subject_matchers": [
            {"key": "workspace_id", "value": "0b1c2d3e-4f56-7890-abcd-ef1234567890"},
            {"key": "customer_id", "value": "acme"},
        ],
        "config": {"window": "monthly", "limit_usd": 250},
    },
    timeout=30.0,
)
response.raise_for_status()

The matcher key is the normalized name customer_id, not the header name X-Gateway-Customer-Id. The policy belongs to the organization that owns the API key. Posting a policy whose subject_matchers already exist updates that policy instead of adding a duplicate, so this call is safe to repeat.

Step 3. Sync policies with your customer list

Because each end customer needs its own policy, keep the policy set in step with your customer list. The following script creates or updates a cap for every current customer, then deletes the caps of customers that are gone:

import os

import httpx

API_URL = "https://api.smith.langchain.com/v1/platform/gateway-policies"
WORKSPACE_ID = os.environ["LANGSMITH_WORKSPACE_ID"]

# Your source of truth: end customer identifier mapped to a monthly cap in USD.
CUSTOMER_CAPS = {"acme": 250.0, "globex": 1000.0, "initech": 50.0}


def matchers_for(customer: str) -> list[dict[str, str]]:
    return [
        {"key": "workspace_id", "value": WORKSPACE_ID},
        {"key": "customer_id", "value": customer},
    ]


def existing_caps(client: httpx.Client) -> dict[str, dict]:
    """Return the current per-customer spend caps, keyed by customer identifier."""
    response = client.get(API_URL, params={"policy_type": "spend_cap"})
    response.raise_for_status()
    return {
        matcher["value"]: policy
        for policy in response.json()
        for matcher in policy["subject_matchers"]
        if matcher["key"] == "customer_id"
    }


def sync() -> None:
    headers = {"X-Api-Key": os.environ["LANGSMITH_API_KEY"]}
    with httpx.Client(headers=headers, timeout=30.0) as client:
        existing = existing_caps(client)

        # Posting an existing matcher set updates that policy, so this both
        # creates caps for new customers and corrects caps that changed.
        for customer, limit_usd in CUSTOMER_CAPS.items():
            client.post(
                API_URL,
                json={
                    "name": f"customer-{customer}-monthly-cap",
                    "policy_type": "spend_cap",
                    "action": "block",
                    "subject_matchers": matchers_for(customer),
                    "config": {"window": "monthly", "limit_usd": limit_usd},
                },
            ).raise_for_status()

        # Deletes any per-customer cap missing from CUSTOMER_CAPS, including
        # caps created outside this script.
        for customer, policy in existing.items():
            if customer not in CUSTOMER_CAPS:
                client.delete(f"{API_URL}/{policy['id']}").raise_for_status()


if __name__ == "__main__":
    sync()

Run the script whenever a customer signs up, churns, or moves to a different plan.

Step 4. Read spend per customer

Each spend policy returned by the API reports current_spend_usd, the spend accumulated in the policy’s active window. Use it to show each end customer their usage, or to warn them before they reach the cap. The field is omitted when the spend lookup fails, so treat a missing value as unknown rather than as zero. The list endpoint narrows by a subject matcher key only when that key is paired with a value, so list the spend caps and select the per-customer ones in your own code:

import os

import httpx

response = httpx.get(
    "https://api.smith.langchain.com/v1/platform/gateway-policies",
    headers={"X-Api-Key": os.environ["LANGSMITH_API_KEY"]},
    params={"policy_type": "spend_cap"},
    timeout=30.0,
)
response.raise_for_status()

for policy in response.json():
    customer = next(
        (m["value"] for m in policy["subject_matchers"] if m["key"] == "customer_id"),
        None,
    )
    if customer is None:
        continue  # A cap on the workspace itself, not on one end customer.
    # current_spend_usd is absent when the spend lookup fails.
    print(customer, policy.get("current_spend_usd"), policy["config"]["limit_usd"])

Limit throughput per end customer

Rate limits use the same subject matchers. Swap policy_type and config to give an end customer its own request and token allowance:

curl -X POST "https://api.smith.langchain.com/v1/platform/gateway-policies" \
    -H "X-Api-Key: $LANGSMITH_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
          "name": "customer-acme-rate-limit",
          "policy_type": "rate_limit",
          "action": "block",
          "subject_matchers": [
            {"key": "workspace_id", "value": "0b1c2d3e-4f56-7890-abcd-ef1234567890"},
            {"key": "customer_id", "value": "acme"}
          ],
          "config": {
            "version": 1,
            "limits": [
              {"metric": "requests", "window": "minute", "value": 100},
              {"metric": "tokens", "window": "hour", "value": 1000000}
            ]
          }
        }'

The sync script in step 3 applies to rate limits with the same two substitutions. Spend caps and rate limits are separate families, so an end customer can hold one of each on the same header value.

Next steps

Spend policies: set cost caps for organizations, workspaces, users, and API keys.
Rate limit policies: limit requests and tokens in a rolling window.
Traces and access control: understand where gateway traces land and who can configure policies.

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.

Core capabilities

Administration and governance

Advanced

Matchable headers

Separate a default spend limit by header

Add an explicit header condition

Cap spend per end customer

Step 1. Send a customer header on every call

Step 2. Create a cap for one customer

Step 3. Sync policies with your customer list

Step 4. Read spend per customer

Limit throughput per end customer

Next steps

​Matchable headers

​Separate a default spend limit by header

​Add an explicit header condition

​Cap spend per end customer

​Step 1. Send a customer header on every call

​Step 2. Create a cap for one customer

​Step 3. Sync policies with your customer list

​Step 4. Read spend per customer

​Limit throughput per end customer

​Next steps

Matchable headers

Separate a default spend limit by header

Add an explicit header condition

Cap spend per end customer

Step 1. Send a customer header on every call

Step 2. Create a cap for one customer

Step 3. Sync policies with your customer list

Step 4. Read spend per customer

Limit throughput per end customer

Next steps