The rapid adoption of large language models (LLMs) has created demand for faster responses and lower costs. Semantic caching, reusing semantically similar requests via their embeddings, addresses t...
<p><strong>Tác giả:</strong> Dvir David Biton, Roy Friedman</p>
<p>The rapid adoption of large language models (LLMs) has created demand for faster responses and lower costs. Semantic caching, reusing semantically similar requests via their embeddings, addresses this need but breaks classic cache assumptions and raises new challenges. In this paper, we explore offline policies for semantic caching, proving that implementing an optimal offline policy is NP-hard, and propose several polynomial-time heuristics. We also present online semantic aware cache polici...</p>
<p>📄 DOI: <a href="https://doi.org/https://doi.org/10.1145/3799682.3840580" target="_blank" rel="noopener">https://doi.org/10.1145/3799682.3840580</a></p>
<p>📖 <a href="https://doi.org/https://doi.org/10.1145/3799682.3840580" target="_blank" rel="noopener">Xem bài báo đầy đủ</a></p>
Bài viết được biên soạn lại bởi LIFVietnam. Nguồn tham khảo: OpenAlex
