Skip to content

⌘ K

Getting Started

Platform
- CPU
- CUDA
- HPU
- Cambricon MLU Backend
- musa-aiter with LMCache on MUSA
- Neuron
- NPU
- Rebellions NPU
- ROCm
- SUPA
- XPU

KV Cache Operations

Recipes

Secondary KV Storage
- Supported Backends
  - NIXL
  - Native NIXL
  - File & Block
  - Remote & Distributed
    - Bigtable
    - SageMaker HyperPod
    - S3
    - HF Bucket
    - Mooncake Store
    - RESP (Redis/Valkey)
    - Valkey
    - Aerospike
    - Hf3fs
    - WQS
  - DAX
  - Mock
  - Fault Inject
  - Plugin (Custom / External)
- KV Cache Compression
  - CacheGen
Using Different Storage Backends

Distributed KV Cache

Use LMCache in Production

Observability
- Metrics
- Logging
- Tracing

Community

KV Cache Optimizations

Developer Guide

Non-KV Caching
- Encodings

Legacy (In-Process Mode)

/

Distributed KV Cache

/

Using shared storage

Using shared storage#

Coming soon…

KV Cache Management

Use LMCache in Production

© 2024, The LMCache Team Built with Sphinx 8.2.3