Definition
Semantic caching stores the responses to previous AI queries indexed by the meaning of the query, so that a new request similar in intent can be answered from the cache rather than re-invoking the model, reducing cost and latency for repeated or near-duplicate questions.
« Back to Glossary Index
- Cuts cost and latency by reusing answers to semantically similar queries
- Matches by meaning, so paraphrased questions still hit the cache
- Reduces load on expensive model inference for common questions
- Improves response speed for users asking familiar things