lmcache coordinator

The lmcache coordinator command launches the LMCache MP coordinator, a standalone HTTP service that tracks the MP server instances in a deployment. MP servers register with it and send periodic heartbeats; the coordinator evicts any instance whose heartbeat lapses past --instance-timeout.

It replaces python -m lmcache.v1.mp_coordinator. The process runs in the foreground; stop it with Ctrl-C.

lmcache coordinator [options]

Quick start

lmcache coordinator \
    --host 0.0.0.0 --port 9300 \
    --instance-timeout 30 \
    --health-check-interval 10

Options

Flag	Description
--host HOST	Bind address for the coordinator’s HTTP server (default: 0.0.0.0).
--port PORT	HTTP port (default: 9300).
--instance-timeout SECS	Seconds without a heartbeat after which an instance is evicted (default: 30).
--health-check-interval SECS	Seconds between health-check sweeps; 0 disables the loop (default: 10).
--eviction-check-interval SECS	Seconds between L2 eviction sweeps; 0 disables the loop (default: 5).
--eviction-ratio RATIO	Fraction of tracked keys (by count) to evict per cycle, 0.0 to 1.0 (default: 0.2).
--trigger-watermark RATIO	Eviction fires when usage reaches this fraction of the quota, 0.0 (exclusive) to 1.0 (default: 1.0).
--blend-chunk-size N	Tokens per chunk for the global CacheBlend directory; must equal the LMCache chunk size the blend servers use (default: 256).
--blend-probe-stride N	Positions between CacheBlend match probes; 1 probes every offset for full recall (default: 1).
--timeout-keep-alive SECS	Seconds the HTTP server keeps idle connections open before closing them. Must be greater than the MP servers’ heartbeat interval (default 5), otherwise heartbeat requests may hit a closing connection and fail with Server disconnected without sending a response (default: 10).

Configuration

Every flag is optional. Unset flags fall back to the LMCACHE_MP_COORDINATOR_* environment variables (HOST, PORT, INSTANCE_TIMEOUT, HEALTH_CHECK_INTERVAL, EVICTION_CHECK_INTERVAL, EVICTION_RATIO, TRIGGER_WATERMARK, BLEND_CHUNK_SIZE, BLEND_PROBE_STRIDE, TIMEOUT_KEEP_ALIVE), and then to the built-in defaults. A supplied flag always overrides the matching env-derived value, so env-only deployments keep working unchanged.

A second set of env-only knobs controls the startup L2 resync — LMCACHE_MP_COORDINATOR_ENABLE_STARTUP_RESYNC (default True), LMCACHE_MP_COORDINATOR_RESYNC_POLL_INTERVAL (1), LMCACHE_MP_COORDINATOR_RESYNC_MAX_WAIT (60), and LMCACHE_MP_COORDINATOR_RESYNC_PAGE_SIZE (1000). See Multi-Server Coordination for the boot-time resync flow and the active eviction loop.

The coordinator drives fleet-wide L2 eviction by calling each MP server’s DELETE /l2 endpoint, and resync paginates GET /l2/keys on a registered MP server. Both endpoints are documented at L2 Storage Management.

See Multi-Server Coordination for the coordinator’s architecture, registration protocol, and HTTP API.