Skip to content

⌘ K

Getting Started

Installation
Quickstart Examples
TroubleShoot
FAQ

KV Cache offloading and sharing

CPU RAM
Local storage
Redis
InfiniStore
MoonCake
ValKey

Disaggregated prefill

Using NIXL
- 1p1d
- XpYd
Using shared storage

KV Cache management

LMCache Controller
Lookup the KV cache
Persist the KV cache
Clear the KV cache
Move the KV cache
Compress the KV cache
Check finish of a control event

KV Cache Optimizations

Compression
- CacheGen
Blending

Use LMCache in production

Docker deployment
Kubernetes deployment

Developer Guide

Contributing guide
Dockerfile

API Reference

Configuring LMCache
Adding new storage backends

Community

Community meetings
Blogs

/

Quickstart Examples

/

Example: Share KV cache across multiple LLMs

Example: Share KV cache across multiple LLMs#

Coming soon…

Example: Offload KV cache to CPU

Example: Disaggregated prefill

© 2024, The LMCache Team Built with Sphinx 8.2.3