Skip to content

⌘ K

Getting Started

KV Cache Operations

Recipes
- Uniform Attention Models
- Hybrid Attention Models

Secondary KV Storage
- Supported Backends
  - NIXL
  - File & Block
  - Remote & Distributed
  - DAX
  - Mock
- KV Cache Compression
  - CacheGen

Distributed KV Cache

Use LMCache in Production

Observability
- Metrics
- Logging
- Tracing

Community
- Community meetings
- Blogs

KV Cache Optimizations
- CacheBlend
- Segmented Prefill

Developer Guide

Non-KV Caching
- Encodings
- Hidden States

Legacy (In-Process Mode)

/

Non-KV Caching

Non-KV Caching#

Caching state beyond the attention KV cache.

Encodings
Hidden States

Extending the HTTP API

© 2024, The LMCache Team Built with Sphinx 8.2.3