> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aihubmix.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt Caching

Prompt Caching is an important mechanism used to reduce model inference costs. By caching previously processed prompt content, it can be reused in subsequent requests, thereby reducing redundant computations, lowering costs, and improving response efficiency.

## Principle

When you send a request with prompt caching enabled, the system checks if the prompt prefix has been cached from recent queries. If found, it uses the cache, reducing processing time and costs; otherwise, it processes the full prompt and caches the prefix after the response begins. This is particularly useful in the following scenarios:

* Prompts containing numerous examples
* Extensive context or background information
* Repetitive tasks with consistent instructions
* Long multi-turn conversations

## Core Mechanism

Different model providers have varying support for caching:

### Automatic Caching

Automatic caching requires no additional configuration; the system automatically identifies and caches reusable content, applicable to models like OpenAI, DeepSeek, etc.

#### OpenAI

* Minimum prompt length: 1024 tokens
* Cost: Writing to cache is free; reading from cache costs 0.25x to 0.5x the original price

#### Gemini

* Implicit context caching is enabled by default, and caching is automatically effective without manual configuration.
* Caching is only effective when the content, model, and parameters are identical; any differences will be treated as a new request and will not hit the cache.
* The cache validity period is set by the developer, and it can also be left unset. If unspecified, it defaults to 1 hour. There are no minimum or maximum duration limits, and costs depend on the number of cached tokens and cache duration.

#### DeepSeek / Grok / Moonshot / Groq

* Cost: Writing to cache is free or at the same price, reading from cache is below the original price

### Claude Model Display Caching

* Requires manual specification of the cache location via `cache_control`
* Allows fine-grained control over caching granularity
* Applicable to Anthropic Claude models

#### OpenAI Compatible Interface

You can set caching breakpoints in `system`, `user` (including images), and `tools` using the `cache_control` field. The following examples only show the key structure:

**System Message Caching (default 5 minutes TTL):**

```json theme={null}
{
  "model": "claude-opus-4-5",
  "messages": [
    {
      "role": "system",
      "content": [
        {"type": "text", "text": "You are an AI assistant"},
        {
          "type": "text",
          "text": "(long context)",
          "cache_control": {"type": "ephemeral"}
        }
      ]
    },
    {
      "role": "user",
      "content": [{"type": "text", "text": "Hello"}]
    }
  ]
}
```

**User Message Caching (1 hour TTL):**

```json theme={null}
{
  "model": "claude-opus-4-5",
  "messages": [
    {
      "role": "system",
      "content": [{"type": "text", "text": "You are an AI assistant"}]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "(long context)",
          "cache_control": {"type": "ephemeral", "ttl": "1h"}
        },
        {"type": "text", "text": "Hello"}
      ]
    }
  ]
}
```

**Image Message Caching:**

```json theme={null}
{
  "role": "user",
  "content": [
    {
      "type": "image_url",
      "image_url": {"detail": "auto", "url": "data:image/jpeg;base64,/9j/4AAQ..."},
      "cache_control": {"type": "ephemeral"}
    },
    {"type": "text", "text": "What's this?"}
  ]
}
```

**Tool Definition Caching:**

Place the `cache_control` at the top level of the tool object (at the same level as `type` and `function`):

```json theme={null}
{
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get current weather for a location",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    },
    "cache_control": {"type": "ephemeral", "ttl": "1h"}
  }]
}
```

#### Anthropic Compatible Interface

<CodeGroup>
  ```shellscript cURL theme={null}
  curl https://aihubmix.com/v1/messages \
    -H "content-type: application/json" \
    -H "x-api-key: $AIHUBMIX_API_KEY" \
    -H "anthropic-version: 2023-06-01" \
    -d '{
      "model": "claude-opus-4-6",
      "max_tokens": 1024,
      "system": [
        {
          "type": "text",
          "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
        },
        {
          "type": "text",
          "text": "<the entire contents of Pride and Prejudice>",
          "cache_control": {"type": "ephemeral"}
        }
      ],
      "messages": [
        {
          "role": "user",
          "content": "Analyze the major themes in Pride and Prejudice."
        }
      ]
    }'

  # Call the model again with the same input until the caching checkpoint
  curl https://aihubmix.com/v1/messages # rest of input
  ```

  ```python Python theme={null}
  import anthropic

  client = Anthropic(
    api_key="<AIHUBMIX_API_KEY>",  
    base_url="https://aihubmix.com"
  )

  params = {
      "model": "claude-opus-4-6",
      "max_tokens": 1024,
      "system": [
          {
              "type": "text",
              "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n",
          },
          {
              "type": "text",
              "text": "<the entire contents of 'Pride and Prejudice'>",
              "cache_control": {"type": "ephemeral"},
          },
      ],
      "messages": [
          {
              "role": "user",
              "content": "Analyze the major themes in 'Pride and Prejudice'.",
          }
      ],
  }
  response = client.messages.create(**params)
  print(response.usage.model_dump_json())

  # Call the model again with the same input until the caching checkpoint
  response = client.messages.create(**params)
  print(response.usage.model_dump_json())
  ```
</CodeGroup>

#### Caching Duration

* Default: 5 minutes
* Optional: 1 hour ("ttl": "1h")

> For more information, please refer to: [Claude Prompt Caching](https://docs.aihubmix.com/en/blogs/aihubmix-openai-upgrade-claude#2-prompt-caching)

## Usage Recommendations

1. **Maintain Stable Prefixes**

Place fixed content at the beginning of the prompt, recommended structure:

```text theme={null}
[System Settings / Long Text / RAG Data] 
[User Question (Variable Part)]
```

2. **Cache Large Texts**

Prioritize caching the following content:

* RAG data
* Long texts
* CSV / JSON data
* Role settings

3. **Control TTL**

* Short sessions → 5 minutes
* Long sessions → 1 hour (more cost-effective)

4. **Reduce Cache Writes**

## Avoid frequently changing content from entering the cache. Do not cache timestamps, user input variables, high-frequency changing data, etc.

Last updated: 2026-06-01