lmcache tool#

The lmcache tool command groups offline analysis utilities bundled with LMCache.

lmcache tool <tool-name> <action> [options]

Currently one tool is available: cache-simulator.

Note

cache-simulator depends on the optional plot extras (sortedcontainers / matplotlib). If they are not installed, the sub-command is silently omitted from the CLI. Install the extras to enable it.

cache-simulator#

Replay LMCache lookup-hash JSONL logs through an LRU cache to measure the KV-cache token hit rate. It has three actions:

lmcache tool cache-simulator {simulate,sweep,gen-dataset} [options]

Action	Description
`simulate`	Replay logs at a fixed cache capacity; print a text report and save a 7-panel statistics PNG.
`sweep`	Sweep across a range of cache capacities and save a hit-rate vs. capacity PNG.
`gen-dataset`	Generate a `vllm bench serve` custom dataset (JSONL) from lookup-hash JSONL logs, preserving prefix-sharing structure.

Each action has its own flags. Run the built-in help for the full list:

lmcache tool cache-simulator simulate --help
lmcache tool cache-simulator sweep --help
lmcache tool cache-simulator gen-dataset --help