Prompt Caching
Prompt Caching
A technology that temporarily saves instructions sent to an AI so repeat requests process faster and cost less.
In Simple Terms
Prompt caching is a mechanism that temporarily stores part of the text you send to an AI so it can be processed faster the next time around. For example, if you ask an AI about a book, resending and reprocessing the entire book's data from scratch every time takes a lot of time. If that book's data is cached in advance, then from the second request onward the AI can skip re-reading it and jump straight to answering your new question. That cuts processing time and keeps usage costs lower too.
Behind the Name
The name blends two computing terms: a "prompt" is the instructions sent to an AI, while "caching" means temporarily storing something so it can be reused instead of recomputed. When the same prompt is sent again, the system reuses the already-processed opening section instead of analyzing it from scratch, cutting both processing time and cost.
Take a Closer Look!
Prompt caching is a technology that temporarily stores, on the system side, the analysis results for long stretches of text repeated across prompts sent to an AI, then reuses that stored analysis in later requests. Put simply, it lets the AI system skip re-analyzing "long context data you keep reusing" from scratch every time.
Normally, when you send a request to an AI, you resend supporting material or conversation history along with it, and the AI reanalyzes all of that from the beginning before it starts generating a reply. That takes real time and money. With prompt caching, the system can instead reuse the analysis results for the shared portion of a long prompt, cutting the time and cost that analysis would otherwise take.
Services generally handle this "shared long text" in one of two ways. One approach resends the entire prompt, including the long text, every time as before, and the system simply reuses its analysis of the matching opening section. The other registers the long text with the system in advance and returns an ID; from the second request onward, you just reference that ID instead of resending the data itself.
Either way, the AI generates a fresh answer to your new question each time. That makes prompt caching different from a "response cache," which reuses stored answers for identical or similar questions.
This technology is a big help to developers and companies building AI-powered apps and services: by skipping duplicate analysis work on the server side, it shortens the wait for a reply and keeps usage costs down.