Skip to content

⌘ K

Getting Started

KV Cache Operations

Recipes

Secondary KV Storage
- Supported Backends
  - NIXL
  - Native NIXL
  - File & Block
  - Remote & Distributed
    - Bigtable
    - SageMaker HyperPod
    - S3
    - HF Bucket
    - Mooncake Store
    - RESP (Redis/Valkey)
    - Valkey
    - Aerospike
    - Hf3fs
    - WQS
  - DAX
  - Mock
  - Fault Inject
  - Plugin (Custom / External)
- KV Cache Compression
  - CacheGen
Using Different Storage Backends

Distributed KV Cache

Use LMCache in Production

Observability
- Metrics
- Logging
- Tracing

Community

KV Cache Optimizations

Developer Guide

Non-KV Caching
- Encodings

Legacy (In-Process Mode)

/

Legacy (In-Process Mode)

/

Non-KV Caching

Non-KV Caching#

Encoder caching
Hidden states

Encoder caching

© 2024, The LMCache Team Built with Sphinx 8.2.3