AI Cost Tracking: Find Where Your LLM Bill Actually Goes

AI cost tracking is the practice of answering one question before it becomes a problem: which model, feature, or user is responsible for your LLM bill. Most teams know their total spend, but not its shape. They can tell you the number on the invoice, but not that one power user drives 40% of it, or that a single agent feature quietly doubled its frontier-model usage last month.

This guide covers how to track AI costs with a tagging schema, a small Python meter you can wire into an OpenAI-compatible client, and the alerts that catch a runaway bill before it lands. It is the companion to the LLM API pricing guide — that one helps you pick cheaper models, this one helps you see where the money goes after you have picked.

Why the Invoice Is Not Enough

An AI bill has no line items by feature. A provider sends you a monthly total and a per-model breakdown at best. What it does not tell you is the allocation that matters for running a product:

• Which feature is the expensive one. A customer-facing chatbot, an internal agent, and a nightly batch job all go through the same key unless you separate them.
• Which model tier is absorbing the spend. You may be routing 5% of requests to a frontier model but generating 60% of the cost there.
• Which user or tenant drives the spend. A small number of power users routinely account for a disproportionate share of token usage.

Without tags, every investigation starts from scratch: you export raw logs, guess at the source, and hope. With tags, the answer is a query.

The Tagging Schema

Attach three tags to every request, and you can answer almost any cost question with a single aggregation.

The model tag is free — the response already returns the model that served the request. The stage and user tags are yours to add, and they are where the insight lives. A request sent to the same model for the same price tells you two different stories depending on whether it is a chat turn from a free-tier user or an agent step from your biggest customer.

The rule that keeps the data honest: tag the request at the point you create it, not when you bill it. If you try to infer the stage later from the prompt text, you will misclassify edge cases. The stage should be a field your application already knows.

A Cost Meter in Python

The OpenAI-compatible client gives you the usage object on every response. Wrap the call to accumulate cost and emit a tagged record.

import os
from collections import defaultdict

from openai import OpenAI

# Per-million-token prices, keyed by model ID. Keep this in one place.
RATES = {
    "gpt-5.6-sol": {"in": 2.00, "out": 10.00},
    "claude-opus-4-8": {"in": 2.00, "out": 10.00},
    "deepseek-flash": {"in": 0.18, "out": 0.72},
}

client = OpenAI(
    api_key=os.environ["OURTOKEN_API_KEY"],
    base_url="https://api.ourtoken.ai/v1",
)

spend = defaultdict(float)


def track(model: str, usage, stage: str, user: str):
    r = RATES.get(model)
    if not r:
        # Unknown model: log and skip rather than silently misprice.
        print("unpriced model:", model)
        return
    cached = getattr(usage, "prompt_tokens_details", None)
    cached_tokens = getattr(cached, "cached_tokens", 0) if cached else 0
    uncached = usage.prompt_tokens - cached_tokens

    cost = (
        uncached / 1_000_000 * r["in"]
        + cached_tokens / 1_000_000 * r["in"] * 0.1  # cache read ≈ 10% of input
        + usage.completion_tokens / 1_000_000 * r["out"]
    )

    key = (model, stage, user)
    spend[key] += cost
    # Emit a structured record for your metrics system here.
    print({"model": model, "stage": stage, "user": user, "cost": round(cost, 6)})


resp = client.chat.completions.create(
    model="deepseek-flash",
    messages=[{"role": "user", "content": "Summarize this request in one line."}],
    max_tokens=64,
)

track(resp.model, resp.usage, stage="chat", user="user-42")
print(spend)

Three details matter more than the arithmetic. First, cached tokens are a subset of input — the meter above subtracts them from prompt_tokens before pricing, so you never double-count a cached prefix. The prompt caching guide covers the accounting rule in depth. Second, prices live in one registry — when a provider cuts rates, you update one dictionary instead of hunting through the codebase. Third, unknown models are skipped loudly, not priced silently — mispricing an unfamiliar model is worse than not pricing it.

Where the 80% Actually Is

Once the tags are in place, run the aggregation. Two patterns show up in almost every real system.

Per-model: the frontier tail dominates. A workload that sends 90% of requests to a cheap route and 10% to a frontier model will routinely spend more on the frontier 10%. Pull the per-model totals and you will see it immediately: the model routing guide explains why this happens and how to re-route, but the tracker is what makes the skew visible in the first place.

Per-user: power users are the bill. Sort by the user tag and a handful of accounts will be responsible for the bulk of frontier spend. That is not a problem to punish — it is the signal for a pricing decision: give those users a higher tier, or build a cheaper specialized path for the one feature they hammer.

The point of the tags is not to produce a prettier dashboard. It is to convert a vague "our AI costs are too high" into a specific "the agent stage for the enterprise tenant spends $X on claude-opus-4-8 this month, and here is why."

Alerts That Catch a Runaway Bill

Tracking that only produces a monthly report is too slow. The bill that surprises you on day 30 could have been caught on day 2. Three alerts cover most failures.

1. Daily burn over a threshold. Set a per-day budget per stage. A spike means an agent loop is stuck, a batch job is misconfigured, or a feature went viral. Day-level granularity is the difference between a small fix and a large invoice.
2. Model-tier drift. Alert when the share of frontier-model tokens crosses a threshold. A classifier that starts over-escalating, or a fallback chain that silently degrades upward, will show up here as a slow climb in strong-tier usage before it becomes a cost problem.
3. Zero-cost anomalies. A stage that should be spending money and suddenly is not — or a stage that should be free and is not — is usually a bug, not a win. A dropped key, a stuck queue, or a misrouted model all show up as an unexpected flatline or a surprise line item.

Each alert fires on a tagged aggregate, which is why the schema comes first. An alert on "total spend" is noise; an alert on "agent stage, claude-opus-4-8, enterprise tenant" is actionable.

AI Cost Tracking Composes With the Rest of Your Stack

Cost tracking is one layer in a stack, and it composes with the others rather than replacing them.

• Routing decides which model serves a request; tracking tells you whether that decision was right. The model routing guide and the tracker use the same model and stage tags.
• Prompt caching cuts the input cost; tracking shows the cached-token share over time, so you know whether the caching guide's stable-prefix discipline is actually being followed.
• Batch processing cuts cost for async work; tracking separates batch as a stage so its discount shows up as a lower effective rate rather than disappearing into the average. The batch API guide covers the mechanics.

If you adopt only one, adopt tracking first. It is the layer that tells you whether the other optimizations are worth doing at all.

Start Small

You do not need a full observability platform to start. Three steps get you 80% of the value:

1. Add a stage tag at the three places your application creates a request (chat, agent, batch).
2. Wrap the client with a cost meter like the one above, pricing the model from a single registry.
3. Emit one aggregated number per day per (model, stage, user) triple, and alert on the daily-burn and tier-drift thresholds.

From there, the questions you can answer get sharper every week, because the data you need is already being collected. The DeepSeek V4 API pricing guide has a more detailed cost-calculator pattern if you want to extend the meter into a full per-route budget model.

Conclusion

AI cost tracking is the difference between knowing your bill and understanding it. Tag every request with model, stage, and user; price it from one registry; and alert on daily burn and tier drift instead of waiting for the monthly invoice. The result is not just a lower bill — it is a specific one, where you can point at the exact stage, model, and user that drives the spend and make a targeted change.

When your usage outgrows a hand-rolled meter, the OurToken API Keys page and the usage view in the dashboard give you the per-model and per-request record without building it yourself. The tagging discipline you develop now is what makes that data useful when you look at it.

FAQ

What is AI cost tracking?

Instrumenting every model request with tags — model, stage, and user — so you can aggregate spend by feature, tier, and account instead of seeing only a monthly total.

Which tags should I add to track AI costs?

Model (free, from the response), stage (chat, agent, batch, embedding), and user or tenant. Model tells you the tier; stage and user tell you the why.

How do I avoid double-counting cached tokens?

Subtract prompt_tokens_details.cached_tokens from prompt_tokens before pricing. Cached tokens are a subset of input and bill at a lower rate.

What alerts catch a runaway LLM bill?

A daily-burn threshold per stage, a model-tier drift alert when frontier usage climbs, and a zero-cost anomaly alert for stages that unexpectedly go silent or appear.

Do I need a full observability platform to start?

No. Add a stage tag at request creation, wrap the client with a cost meter, and emit one daily aggregate per (model, stage, user) triple with two alerts. That covers most of the value.