Skip to content
LMCache
⌘ K
LMCache
  • Getting Started
    • Overview
    • Installation
    • vLLM and LMCache compatibility
    • Quickstart
    • Configuration Reference
    • Benchmarking
    • KV Cache Size Calculator
  • Platform
    • CPU
    • CUDA
    • HPU
    • Cambricon MLU Backend
    • musa-aiter with LMCache on MUSA
    • Neuron
    • NPU
    • Rebellions NPU
    • ROCm
    • SUPA
    • XPU
  • KV Cache Operations
    • CLI Reference
      • lmcache server
      • lmcache coordinator
      • lmcache describe
      • lmcache ping
      • lmcache query
      • lmcache bench
      • lmcache kvcache
      • lmcache quota
      • lmcache trace
      • lmcache tool
    • HTTP API
    • Frontend Dashboard
    • KV Cache SDK
    • Internal API Server
      • Common APIs
      • vLLM / Inference APIs
  • Recipes
    • Uniform Attention Models
      • MiniMax M2 series
      • Mistral / Devstral
      • Qwen3 MoE
      • Llama
      • Phi-3 / Phi-4
      • Mixtral
    • Hybrid Attention Models
      • Gemma 3
      • Gemma 4
      • gpt-oss
      • Qwen3.5 / Qwen3.6 / Qwen3.8 series
      • Qwen3.8-Flash-Next
      • Kimi-Linear
      • Kimi K3
      • DeepSeek-V4-Flash
      • DeepSeek-V4.1-Flash
      • GLM 5.1/5.2
      • GLM-5.3
      • MiniMax M3
    • Multimodal Models
      • Qwen2.5-VL (multimodal)
  • Secondary KV Storage
    • Supported Backends
      • NIXL
      • Native NIXL
      • File & Block
        • FileSystem
        • FS (native)
        • Raw Block (Rust)
      • Remote & Distributed
        • Bigtable
        • SageMaker HyperPod
        • S3
        • HF Bucket
        • Mooncake Store
        • RESP (Redis/Valkey)
        • Valkey
        • Aerospike
        • Hf3fs
        • WQS
      • DAX
      • Mock
      • Fault Inject
      • Plugin (Custom / External)
    • KV Cache Compression
      • CacheGen
  • Using Different Storage Backends
    • Device-DAX (/dev/dax)
    • Filesystem Backend
    • Hugging Face Buckets Backend
    • RESP (Native Redis/Valkey)
    • S3 Backend
  • Distributed KV Cache
    • Disaggregated Prefill
    • P2P KV Cache Sharing
    • Multi-Server Coordination
    • KV Cache Management
    • Using shared storage
  • Use LMCache in Production
    • Deployment Guide
    • Kubernetes Deployment
    • Docker deployment
    • Observability
    • Kubernetes Operator
    • Runtime Plugins
    • Dynamo Integration
  • Observability
    • Metrics
    • Logging
    • Tracing
  • Community
    • Frequently asked questions
    • Community meetings
    • Blogs
  • KV Cache Optimizations
    • CacheBlend
    • Segmented Prefill
    • Lazy KV Cache Offload
  • Developer Guide
    • Contributing Guide
    • Integration
    • Storage Plugins
    • Usage Data Module
    • Adding Native Backends
    • Adding a New Device Backend
    • Server Module Plugins
    • Extending the CLI
    • Extending the HTTP API
    • Cache identity tags
  • Non-KV Caching
    • Encodings
  • Legacy (In-Process Mode)
    • More Examples
      • Example: Offload KV cache to CPU
      • Example: Share KV cache across multiple LLMs
      • Example: Disaggregated prefill
      • Example: Multimodal KV Cache Support
      • Standalone Starter
    • KV Cache
      • Storage Backends
        • 3FS
        • Azure Blob Storage Backend
        • Google Cloud Bigtable Remote Cache
        • CPU RAM
        • Custom Storage Backends
        • EIC
        • GDS Backend
        • InfiniStore
        • Local storage
        • Maru
        • Mock
        • Mooncake
        • Nixl
        • Redis
        • SageMaker Hyperpod
        • Valkey
        • Weka
      • Async Loading
      • Using Different Caching Policies
      • P2P KV Cache Sharing
    • Using NIXL
      • 1p1d
      • XpYd
    • KV Cache Optimizations
      • Blending
      • Compression
        • CacheGen
      • Layerwise KV Transfer
    • LMCache Controller
      • Clear the KV cache
      • Compress and Decompress the KV cache
      • Check controller health
      • Lookup the KV cache
      • Move the KV cache
      • Pin the KV cache
      • Check finish of a control event
      • Query Worker Info
    • API Reference
      • Configuring LMCache
      • vLLM Dynamic Connector
      • KV Caching for Multimodal Models with vLLM
      • Adding new storage backends
    • Internal API Server
      • Controller APIs
      • Dynamic Backend Management
    • Controller WebUI
      • Freeze Mode
    • Non-KV Caching
      • Encoder caching
      • Hidden states
    • Production
      • Observability
        • Chunk Statistics
        • LMCache Frontend
        • Health Monitor
        • Internal API Server Metrics
        • Metrics Reference
        • Metrics by vLLM API
      • Performance Tuning
      • KV Cache Events
      • Observability
        • Periodic Thread Monitoring API
    • Developer Guide
      • Architecture Overview
      • Basic Check Tool
      • Remote Storage Plugins
LMCache
/
KV Cache Operations

KV Cache Operations#

Ways to inspect, manage, and observe a running lmcache server – from the command line, over HTTP, or through the web frontend.

  • CLI Reference
    • Installation
    • Available Commands
    • Output Formats
    • Adding New Commands
  • HTTP API
    • Where the routes come from
    • Server Configuration
    • Endpoint Overview
    • Liveness and Health
    • Inspection and Status
    • Cache Management
    • Cache Objects And Prefetch
    • Quota Management
    • Runtime L2 Reconfiguration
    • Runtime L1 Reconfiguration (Device-DAX)
    • Observability
    • Diagnostics and Debugging
    • Adding New Endpoints
  • Frontend Dashboard
    • Architecture Overview
    • Components
    • Prerequisites
    • Quick Start (all-in-one on localhost)
    • Dashboard Features
    • CLI Reference
    • Plugin Config Keys
    • Using a Custom Discovery Service
  • KV Cache SDK
    • Why KV Cache SDK
    • How it works
    • Configuration
    • Writing a custom edit function
    • API reference
  • Internal API Server
XPU
CLI Reference

© 2024, The LMCache Team Built with Sphinx 8.2.3