OpenAI Prompt Caching API: Reduce LLM Input Costs with Cached Tokens

OpenAI prompt caching is one of the most practical ways to reduce LLM input costs when your application repeats long instructions, tool definitions, retrieval context, or agent state across requests. It does not make every request cheaper. It makes repeated prompt prefixes cheaper when the model can reuse work it has already performed.

That distinction matters. A support assistant with a 4,000-token system prompt, a RAG app with stable citation rules, and a coding agent with repeated tool schemas can waste real money by sending the same prefix over and over. The OpenAI Prompt Caching API helps by reporting cache reads as cached_tokens and, for GPT-5.6 family models and later, cache writes as cache_write_tokens. Those fields let you measure whether your prompt architecture is actually saving money.

This guide explains how prompt caching works, how to structure prompts for cache hits, how to read usage fields safely, and how to estimate cost with cached input tokens. The examples use OpenAI's current Prompt Caching documentation as the source of truth, then show how to apply the same accounting discipline when testing GPT-5.6 routes through OurToken. If you are already using OurToken's unified endpoint, the GPT-5.6 Terra API page is the natural model page to keep open while you test cached input and cache-write pricing.

OpenAI Prompt Caching API Basics

Prompt caching works when a request repeats an exact prompt prefix that the service can reuse. The prefix can include static instructions, examples, tool definitions, images, files, or structured output schemas. The dynamic part of the request should come later.

OpenAI's official Prompt Caching guide documents three details that matter for production teams:

1. Prompt caching is enabled automatically for eligible requests.
2. Cache hits require exact prefix matches.
3. Requests under 1,024 tokens still report cached_tokens, but the value is zero.

For GPT-5.6 models and later model families, OpenAI also documents a more explicit caching model. You can use prompt_cache_key to improve routing for related traffic, add explicit prompt_cache_breakpoint markers after stable prompt content, and use prompt_cache_options.mode to control whether implicit or explicit breakpoints are used. This is important because GPT-5.6 cache writes have their own cost model: cache_write_tokens are billed at 1.25x the uncached input token rate.

What gets cached

The reusable part is not a vague semantic summary. It is an exact prefix. If two prompts have the same intent but different bytes near the beginning, the cache may miss.

A good mental model is simple:

Stable prefix: instructions + examples + tools + schema + reusable files
Cache breakpoint: end of the reusable section
Variable suffix: current user message + retrieved snippets + request metadata

If your variable suffix appears before the stable prefix ends, you are asking the cache to match a moving target.

Prompt caching vs prompt compression

Prompt caching is not the same as summarization, truncation, or prompt compression.

You can combine them. For example, a RAG app can cache stable system instructions and tool schemas, compress retrieved snippets, and route easy questions to a lower-cost model. If you are standardizing this kind of app behind one endpoint, the Responses API migration guide is useful because Responses is the primary surface for newer OpenAI features such as tools, structured outputs, and stateful request patterns.

Build Prompts and Python Tests for Cache Hits

A prompt caching rollout usually fails for mundane reasons. The team knows caching exists, but the prompt builder adds a changing timestamp at the top. Or the tool serializer changes property order. Or every tenant shares one broad cache key, causing noisy traffic to compete for the same cache bucket.

The fix is to treat prompt layout as production code.

Stable prefix first

Here is a simplified prompt layout for a support assistant:

[system]
You are a support triage assistant.
Use the following categories: billing, account, bug, feature_request, other.
Escalate urgent finance, security, and availability issues.
Return concise answers.

[tool definitions]
- search_knowledge_base
- get_invoice_status
- create_escalation_ticket

[structured output schema]
{ ... stable JSON schema ... }

[user]
Customer message for this request.

That layout gives the model a long stable prefix. If the same assistant handles many customer messages, the beginning can be reused. If you instead put request_id, current_time, or a per-user profile before the system prompt, each request starts differently and the reusable prefix shrinks.

For apps that already use structured JSON responses, the OpenAI Structured Outputs JSON Schema guide pairs naturally with prompt caching. JSON schemas are often stable, long enough to matter, and repeated across thousands of requests.

Use prompt cache keys carefully

OpenAI documents prompt_cache_key for requests that share long common prefixes. The key helps route related traffic so a prefix match is more likely. For GPT-5.6 models and later, OpenAI says the key is required to use the more reliable matching for implicit and explicit caching.

A good cache key describes the reusable prompt family, not an individual user request:

support-triage-v4
rag-policy-legal-docs-v2
coding-agent-tools-v6
tenant-acme-knowledge-base-v3

A poor cache key changes too often:

request-1785399942
user-491-current-session
support-triage-2026-07-30T12:01:33Z

OpenAI also notes that traffic per key should stay around 15 requests per minute. For higher-volume workloads, partition traffic across more stable keys. That means a large SaaS app may want one key per tenant and workflow version rather than one global key for every request.

Add explicit breakpoints for GPT-5.6

For GPT-5.6 family models and later, the OpenAI docs describe explicit cache breakpoints. A breakpoint marks the end of the stable prefix. Content after the breakpoint can change without invalidating the cached section before it.

OpenAI's current documentation shows this request shape for Responses API content blocks:

{
  "model": "gpt-5.6",
  "prompt_cache_key": "tenant:acme:knowledge-base-v1",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": [
        {
          "type": "input_file",
          "file_id": "file_123",
          "prompt_cache_breakpoint": {
            "mode": "explicit"
          }
        },
        {
          "type": "input_text",
          "text": "Answer the current question."
        }
      ]
    }
  ]
}

That example is an OpenAI-native request shape. When testing through a compatibility gateway, first confirm that the selected route accepts prompt_cache_key, prompt_cache_options, and explicit breakpoints. The OurToken GPT-5.6 model page documents the Responses endpoint and cached input/cache-write pricing, but your production route should still be smoke-tested with the exact optional fields you plan to use.

Python example: measure cached tokens

The simplest safe implementation is not a clever abstraction. It is a repeatable test that sends the same stable prefix more than once, reads usage, and computes cost without double counting cached tokens.

First Responses API request

This example uses the OpenAI Python SDK. To test against OurToken, set base_url to https://api.ourtoken.ai/v1, use an OurToken API key from the OurToken API Keys page, and set the model to a route that supports Responses API, such as gpt-5.6-terra.

pip install openai

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["OURTOKEN_API_KEY"],
    base_url="https://api.ourtoken.ai/v1",
)

STABLE_PREFIX = """
You are a support triage assistant.
Classify the request, summarize it in one sentence, and recommend the next action.
Categories: billing, account, bug, feature_request, other.
Escalate billing disputes, security issues, and production outages.
Keep the response concise.
""".strip()

response = client.responses.create(
    model="gpt-5.6-terra",
    instructions=STABLE_PREFIX,
    input="Customer: I was charged twice after upgrading. Please fix it before finance close.",
    max_output_tokens=300,
)

print(response.output_text)
print(response.usage)

Run the same request family repeatedly with different customer messages but the same stable prefix. The first request may write cache. Later requests may report cached input tokens.

Optional OpenAI-native cache controls

For OpenAI-native GPT-5.6 requests, the docs support cache keys and breakpoints. In a compatibility environment, use this as a validation target, not a blind assumption:

response = client.responses.create(
    model="gpt-5.6-terra",
    instructions=STABLE_PREFIX,
    input="Customer: I need the invoice for last month.",
    max_output_tokens=300,
    prompt_cache_key="support-triage-v1",
    prompt_cache_options={"mode": "explicit", "ttl": "30m"},
)

If a route rejects prompt_cache_options, remove explicit cache controls and rely on the route's documented automatic behavior. Do not retry endlessly. A 400 for an unsupported optional field is a configuration signal, not a transient error.

Normalize usage fields

Usage field names can differ between API surfaces and compatibility layers. OpenAI's docs say Responses API returns cached_tokens in usage.input_tokens_details, while Chat Completions returns it in usage.prompt_tokens_details. OurToken model pages often expose compatibility fields such as prompt_tokens, completion_tokens, and prompt_tokens_details.cached_tokens.

Use a normalizer before you calculate cost:

from typing import Any


def to_dict(value: Any) -> dict:
    if value is None:
        return {}
    if isinstance(value, dict):
        return value
    if hasattr(value, "model_dump"):
        return value.model_dump()
    return dict(value)


def normalize_usage(usage: Any) -> dict[str, int]:
    data = to_dict(usage)
    input_details = to_dict(
        data.get("input_tokens_details") or data.get("prompt_tokens_details")
    )

    input_tokens = data.get("input_tokens", data.get("prompt_tokens", 0)) or 0
    output_tokens = data.get("output_tokens", data.get("completion_tokens", 0)) or 0
    cached_tokens = input_details.get("cached_tokens", 0) or 0
    cache_write_tokens = input_details.get("cache_write_tokens", 0) or 0

    return {
        "input_tokens": int(input_tokens),
        "output_tokens": int(output_tokens),
        "cached_tokens": int(cached_tokens),
        "cache_write_tokens": int(cache_write_tokens),
    }

The most important rule: cached tokens are a breakdown of input tokens, not extra tokens to add on top. If prompt_tokens is 10,000 and cached_tokens is 8,000, the request did not use 18,000 input tokens. It used 10,000 input tokens, of which 8,000 were served from cache.

Prompt Caching Cost Math

Cost math is where many prompt caching articles go wrong. They say caching reduces token count. Usually it does not. It reduces the price applied to part of the input. The model still receives the relevant context; some of it is just billed at a cached-input rate.

At the time this article was prepared, the live GPT-5.6 Terra page on OurToken listed:

Always check the live GPT-5.6 Terra API page before publishing budget numbers. Model pricing can change.

Correct cost formula

For GPT-5.6 family routes with separate cache reads and writes, calculate input-side cost like this:

uncached_input_tokens = input_tokens - cached_tokens - cache_write_tokens
input_cost = uncached_input_tokens * input_rate
cached_cost = cached_tokens * cached_input_rate
cache_write_cost = cache_write_tokens * cache_write_rate
output_cost = output_tokens * output_rate
request_cost = input_cost + cached_cost + cache_write_cost + output_cost

Clamp uncached_input_tokens at zero if the provider reports a shape you did not expect, and log the anomaly. Do not silently produce a negative cost.

def estimate_cost_usd(
    usage: dict[str, int],
    *,
    input_per_million: float,
    cached_per_million: float,
    cache_write_per_million: float,
    output_per_million: float,
) -> dict[str, float]:
    input_tokens = usage["input_tokens"]
    cached_tokens = usage["cached_tokens"]
    cache_write_tokens = usage["cache_write_tokens"]
    output_tokens = usage["output_tokens"]

    uncached_tokens = max(input_tokens - cached_tokens - cache_write_tokens, 0)

    input_cost = uncached_tokens / 1_000_000 * input_per_million
    cached_cost = cached_tokens / 1_000_000 * cached_per_million
    write_cost = cache_write_tokens / 1_000_000 * cache_write_per_million
    output_cost = output_tokens / 1_000_000 * output_per_million

    total = input_cost + cached_cost + write_cost + output_cost

    return {
        "uncached_input_cost": input_cost,
        "cached_input_cost": cached_cost,
        "cache_write_cost": write_cost,
        "output_cost": output_cost,
        "total_cost": total,
    }

Example: first request vs repeated request

Assume a support assistant request has:

Using the Terra prices above:

The first request can be more expensive because it writes the prefix. The benefit appears when that prefix is reused. This is why caching works best for repeated workloads, not one-off prompts.

Break-even thinking

A cache write is an investment. With Terra's example prices, writing a token costs $0.625/M, reading it later costs $0.05/M, and processing it uncached costs $0.50/M. A later cache hit saves $0.45/M for each cached token compared with full-price input.

That means the first write pays off only after reuse. If a prefix is never reused, caching may add cost. If it is reused many times, the write cost becomes tiny compared with repeated cached reads.

This is the practical prompt caching rule:

Cache stable prefixes that are large, reused, and likely to be reused soon.
Do not optimize one-off variable context for caching.

Production Architecture for LLM Prompt Caching

A good prompt caching implementation is not just a parameter. It is a small architecture decision around prompt assembly, routing, telemetry, and cost accounting.

Application request
  -> workload classifier
  -> stable prompt builder
  -> deterministic tool/schema serializer
  -> cache key selector
  -> model route selection
  -> Responses API call
  -> usage normalizer
  -> cache hit and cost dashboard

The key is to keep stable prompt construction separate from variable request data. The model call should receive both, but your code should know which part is supposed to be reusable.

What to log

At minimum, log these fields per route:

Do not log raw API keys or sensitive customer prompts. The goal is cost and cache observability, not prompt surveillance.

Where OurToken fits

OurToken is useful when you want to compare GPT-5.6 Sol, Terra, and Luna routes while keeping the integration surface consistent. The pricing gap matters for caching strategy:

Create a key through OurToken API Keys, run a short uncached baseline, then repeat the same prompt family enough times to see whether cached_tokens appears in usage. If you are migrating from Chat Completions to Responses, use the Responses API migration guide to separate API-shape changes from caching changes. Do not change model, prompt, endpoint, and cache policy in the same test; you will not know which change caused the result.

Common mistakes

The boring implementation wins: deterministic prefix, stable key, clear usage normalization, and a dashboard that separates cache reads from writes.

Conclusion and FAQ

OpenAI prompt caching is most valuable when your application has long repeated prefixes: system instructions, tools, files, structured schemas, RAG policies, and agent setup. The API can reduce cost and latency, but only if your prompt layout gives the cache something stable to match.

For GPT-5.6 models and later, treat prompt caching as a measurable cost system. Read cached_tokens, read cache_write_tokens, calculate uncached input correctly, and remember that cache writes need reuse before they pay back. If you are testing through OurToken, start with a documented GPT-5.6 route such as Terra, keep the prompt prefix stable, and verify usage fields on your own account before relying on a forecast.

FAQ

What is OpenAI prompt caching?

OpenAI prompt caching reuses repeated prompt prefixes so supported requests can reduce latency and bill reused input at a cached-input rate. It is automatic for eligible prompts, and GPT-5.6 models add more explicit cache controls.

What are cached tokens in OpenAI?

cached_tokens are input tokens read from cache. They are part of the total input token count, not extra tokens to add on top.

What are cache write tokens?

For GPT-5.6 models and later, cache_write_tokens report prompt tokens written to cache. OpenAI documents cache writes at 1.25x the uncached input token rate for those model families.

Does prompt caching reduce token count?

Usually no. It reduces the cost applied to repeated input tokens. Your app may still send the same amount of context, but reused prefix tokens can be billed differently.

When does prompt caching not help?

It helps less when prompts are short, requests are one-off, the variable part appears early, or reuse happens outside the provider's cache retention window.

Can I use prompt caching with OurToken?

OurToken's GPT-5.6 model pages list cached input and cache-write pricing, and the Responses endpoint is documented for routes such as GPT-5.6 Terra. Test the exact optional cache fields and usage response on your account before production rollout.