Skip to content

⌘ K

Getting Started

KV Cache Operations

Recipes
- Uniform Attention Models
- Hybrid Attention Models

Secondary KV Storage
- Supported Backends
  - NIXL
  - File & Block
  - Remote & Distributed
  - DAX
  - Mock
- KV Cache Compression
  - CacheGen

Distributed KV Cache

Use LMCache in Production

Observability
- Metrics
- Logging
- Tracing

Community
- Community meetings
- Blogs

KV Cache Optimizations
- CacheBlend
- Segmented Prefill

Developer Guide

Non-KV Caching
- Encodings
- Hidden States

Legacy (In-Process Mode)

/

/

Hidden States

Hidden States#

Note

Hidden-state caching is in progress.

Legacy (In-Process Mode)

© 2024, The LMCache Team Built with Sphinx 8.2.3