Skip to content

⌘ K

Getting Started

Platform
- CPU
- CUDA
- HPU
- musa-aiter with LMCache on MUSA
- Neuron
- NPU
- RBLN
- ROCm
- SUPA
- XPU

KV Cache Operations

Recipes

Secondary KV Storage
- Supported Backends
  - NIXL
  - Native NIXL
  - File & Block
  - Remote & Distributed
    - Bigtable
    - SageMaker HyperPod
    - S3
    - HF Bucket
    - Mooncake Store
    - RESP (Redis/Valkey)
    - Valkey
    - Aerospike
    - Hf3fs
    - WQS
  - DAX
  - Mock
  - Fault Inject
  - Plugin (Custom / External)
- KV Cache Compression
  - CacheGen
Using Different Storage Backends

Distributed KV Cache

Use LMCache in Production

Observability
- Metrics
- Logging
- Tracing

Community

KV Cache Optimizations

Developer Guide

Non-KV Caching
- Encodings

Legacy (In-Process Mode)

/

/

Neuron

Neuron#

Note

Device-specific documentation will be added by the backend maintainers.

musa-aiter with LMCache on MUSA

© 2024, The LMCache Team Built with Sphinx 8.2.3