What is Prompt caching?
Prompt caching reuses a model's computation for repeated prompt prefixes, billing the cached input at a fraction of the normal rate.
Many applications send the same long prefix on every request — a system prompt, a tool schema, a document, or a conversation history. Prompt caching stores the model's internal representation of that prefix so it doesn't have to be recomputed on the next call, which is both faster and cheaper.
Cached input is typically billed at around 10% of the normal input rate — up to a 90% discount on the repeated portion. For agents and chatbots with large stable prefixes, this is often the single biggest lever on cost.
Caching is most effective when the cacheable prefix is large and stable and the variable part (the user's new message) is small — exactly the shape of coding agents, RAG pipelines, and long-running chats.
How Kyma does it
Kyma forwards caching hints to models that support it and normalizes the savings, billing cached input at 10% of the input rate. Every response reports usage.cost and the cache discount so you can see exactly what caching saved.