Skip to content

⌘ K

Getting Started

KV Cache Operations

Recipes
- Uniform Attention Models
- Hybrid Attention Models

Secondary KV Storage
- Supported Backends
  - NIXL
  - File & Block
  - Remote & Distributed
  - DAX
  - Mock
  - Fault Inject
- KV Cache Compression
  - CacheGen

Distributed KV Cache

Use LMCache in Production

Observability
- Metrics
- Logging
- Tracing

Community
- Community meetings
- Blogs

KV Cache Optimizations
- CacheBlend
- Segmented Prefill

Developer Guide

Non-KV Caching
- Encodings
- Hidden states

Legacy (In-Process Mode)

/

Legacy (In-Process Mode)

/

Using shared storage

Using shared storage#

Coming soon…

© 2024, The LMCache Team Built with Sphinx 8.2.3