Skip to content

⌘ K

Getting Started

KV Cache Operations

Recipes
- Uniform Attention Models
- Hybrid Attention Models

Secondary KV Storage
- Supported Backends
  - NIXL
  - File & Block
  - Remote & Distributed
  - DAX
  - Mock
- KV Cache Compression
  - CacheGen

Distributed KV Cache

Use LMCache in Production

Observability
- Metrics
- Logging
- Tracing

Community
- Community meetings
- Blogs

KV Cache Optimizations
- CacheBlend
- Segmented Prefill

Developer Guide

Non-KV Caching
- Encodings
- Hidden States

Legacy (In-Process Mode)

/

Legacy (In-Process Mode)

/

Adding new storage backends

Adding new storage backends#

Coming soon…

KV Caching for Multimodal Models with vLLM

vLLM Dynamic Connector

© 2024, The LMCache Team Built with Sphinx 8.2.3