配置参考

本页面记录了 LMCache 多进程服务器接受的每个 CLI 参数。参数按定义它们的配置模块进行分组。

Per-request LMCache configuration

vLLM clients can attach request-scoped LMCache metadata through the top-level kv_transfer_params field. When vLLM uses LMCacheMPConnector, entries whose keys start with lmcache. are forwarded with the request across the MP scheduler and worker IPC paths. Other kv_transfer_params entries are reserved for the transfer layer and are not forwarded to LMCache.

For example:

curl -X POST http://localhost:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "Qwen/Qwen3-14B",
        "prompt": "Explain KV cache reuse.",
        "max_tokens": 32,
        "kv_transfer_params": {
            "lmcache.tag.tenant": "example-tenant",
            "lmcache.ttl": 60
        }
    }'

The connector carries these values on lookup, prefetch, store, retrieve, and lookup-lock cleanup operations so server-side features can inspect the same request metadata throughout the request lifecycle.

重要

Forwarding a value does not by itself make the MP server act on it or make it part of cache identity. The current MP server treats request configs as metadata. Do not rely on lmcache.tag.*, lmcache.ttl, lmcache.skip_save, or another request config for isolation, expiration, or cache-control behavior in MP mode unless the selected server-side feature explicitly documents support for it. The in-process LMCacheConnectorV1 may interpret these values differently.

MP 服务器

源: lmcache/v1/multiprocess/config.py

参数	默认	描述
--transport	zmq	Request transport exposed by the server. Choices: zmq or grpc.
--instance-id	(未设置，默认 UUID v4)	此 MP 服务器的稳定标识。用作协调器成员身份键，并投影到每条指标和每个 Span 的 OTel service.instance.id 资源属性（从而使遥测数据与协调器成员身份共享同一个 ID）。若未传递该参数，则默认为启动时生成的随机 UUID v4。
--host	localhost	Host address to bind the selected request server.
--port	5555	Port to bind the selected request server.
--chunk-size	256	KV Cache 操作的块大小（以 token 为单位）。
--null-block-id	0	Engine block ID that denotes absent KV data. Keep the default for vLLM-compatible layouts. Engines where block 0 is valid, such as ATOM native PAGE/STATE transfer, can use -1.
--max-workers	1	工作线程的基本数量。为 GPU（亲和性）池和 CPU（正常）池设置默认值。可以通过 --max-gpu-workers 和 --max-cpu-workers 针对每个池进行覆盖。
--max-gpu-workers	(继承 --max-workers)	GPU 亲和性池（STORE/RETRIEVE）的工作线程。来自同一 vLLM 实例的请求始终分发到同一线程，从而消除 GPU 传输锁竞争。
--max-cpu-workers	(继承 --max-workers)	普通 CPU 池（LOOKUP 等）的工作线程。
--grpc-server-workers	32	gRPC request-dispatch threads. Used only with --transport grpc.
--hash-algorithm	blake3	基于 token 的操作所使用的哈希算法。可选项：builtin、sha256_cbor、blake3。
--engine-type	default	Cache engine backend type. default uses standard prefix caching; blend composes the CacheBlend BlendModule into the engine for non-prefix KV reuse and requires --supported-transfer-mode to be lmcache_driven or auto. Choices: default, blend.
--supported-transfer-mode	lmcache_driven	Which worker → server transfer paths the server loads. lmcache_driven (default) enables only the server-driven transfer path (STORE/RETRIEVE, supports both CUDA IPC and CPU SHM); engine_driven enables only the non-GPU (PREPARE/COMMIT) transfer path; auto loads both so workers of either device type can connect without manual configuration. Choices: lmcache_driven, engine_driven, auto.
--isolated-ipc / --no-isolated-ipc	false	Assume engine workers and this server run in containers that share no host IPC namespace (hostIPC) and no common /dev/shm, and use IPC mechanisms that work there: on CUDA, raw CUDA IPC memory handles for KV-cache registration (instead of PyTorch storage IPC, which needs a shared /dev/shm) and timeline-semaphore events (instead of CUDA interprocess event handles). Must match the workers' lmcache.mp.isolated_ipc setting -- the two mechanisms exchange incompatible event handles, and a mismatch fails at event import on whichever side receives the foreign handle. Currently supported by the vLLM MP connector only; the default stays false until the integrations that still create raw CUDA interprocess events (SGLang, TensorRT-LLM, CacheBlend, qstore) migrate.
--runtime-plugin-locations	[]	零个或多个路径，用于运行时插件脚本或目录，以便与服务器一起启动。插件由 MPRuntimePluginLauncher 生成，并通过 LMCACHE_RUNTIME_PLUGIN_CONFIG 环境变量接收完整的服务器配置。
--runtime-plugin-config	"{}"	通过 LMCACHE_RUNTIME_PLUGIN_EXTRA_CONFIG 转发到运行时插件的额外键值配置的 JSON 字符串。示例：'{\"plugin.frontend.heartbeat_url\": \"http://localhost:5000/heartbeat\"}'。
--script-allowed-imports	[]	允许在 HTTP /run_script 端点发布的脚本导入的以空格分隔的 Python 模块名称列表。示例：--script-allowed-imports numpy pandas。
--run-script-api-enabled	false	Enable the POST /run_script HTTP endpoint, which executes caller-supplied Python in-process. The restricted builtins are not a security boundary — treat this as full remote code execution and only enable it on a trusted network.
--shm-name	""	SHM segment name for non-GPU KV transfer (only used when the non-GPU path is loaded, i.e. --supported-transfer-mode is auto or engine_driven). "" (empty string, default): SHM disabled; KV transfer uses the pickle path. Any other value: create a SHM pool and use that exact name for its segment.
--worker-reap-timeout-seconds	120.0	工作线程发送至少一个心跳 PING 后进入静默状态的最长容忍时间（秒），超时后将回收其 KV Cache 注册，并释放泄漏的 GPU 上下文和 CUDA IPC 句柄。0 表示禁用回收。建议将此值保持在引擎适配器 lmcache.mp.heartbeat_interval（默认 10 秒）的至少 3 倍，以免少量心跳丢失导致活跃工作线程被误回收；若未同步调大此值而单独调大心跳间隔，适配器将在启动时发出警告。
--worker-registration-grace-seconds	3600.0	针对已注册但从未发送 PING 的工作线程（仍在预热或在首次请求前已崩溃）的静默容忍时间（秒）。必须大于或等于 --worker-reap-timeout-seconds。默认值较宽松，以避免将缓慢的模型预热误判为工作线程已宕机。
--enable-segmented-prefix	False	CacheBlend (--engine-type blend) 仅适用：在中间前缀 L2 检索失败时，保留缺口前缀，以便缺口后的块保持在 L1 中驻留，只有丢失的缺口被重计算，而不是在缺口处截断前缀。对其他引擎没有影响。有关如何使用它，请参见 故障注入。
--enable-dedup-content	False	--engine-type blend only: skip fingerprint registration for a chunk whose content is already indexed, so the same text stored behind two prefixes is indexed once. No effect for other engines.
--separate-object-groups / --no-separate-object-groups	False	Split a hybrid model's kernel groups into one object group per cross-chunk attention window (full attention, each sliding-window size, mamba/GDN) at KV-cache registration. Off by default; pass --separate-object-groups to enable it. Required for Mamba / linear-attention hybrids (it lets their recurrent state be cached independently, and is what allows --max-num-batched-tokens to exceed twice the block size). For a non-hybrid model it makes no difference — every layer resolves to one object group. See Hybrid Attention 模型.

查找哈希日志记录

源: lmcache/v1/mp_observability/subscribers/logging/lookup_hash.py

启用时，服务器将在 EventBus 上以 MP_LOOKUP 事件的形式发布 lookup() 期间计算的块哈希。LookupHashLoggingSubscriber 会将这些事件写入滚动 JSONL 文件以供离线分析。默认禁用。这些参数属于可观察性配置组。

参数	默认	描述
--lookup-hash-log-dir	"" (禁用)	写入查找哈希 JSONL 文件的目录。空字符串将禁用日志记录。
--lookup-hash-log-rotation-interval	21600 (6 小时)	滚动到新日志文件前的时间间隔（秒）。
--lookup-hash-log-rotation-max-size	104857600 (100 MB)	即使时间间隔未到，触发滚动的最大文件大小（字节）。
--lookup-hash-log-max-files	100	保留的最大日志文件数量。当超过此限制时，最旧的文件将被删除。

HTTP 前端

源: lmcache/v1/multiprocess/config.py

HTTP 前端在运行 lmcache server 时包含在内。

参数	默认	描述
--http-host	127.0.0.1	Host to bind the HTTP (FastAPI/uvicorn) server. The admin API has no authentication; only bind a non-loopback address on a trusted network.
--http-port	8080	绑定 HTTP 服务器的端口。

P2P

源: lmcache/v1/multiprocess/config.py

这些标志配置 MP 服务器之间的点对点 KV Cache 共享（请参见 P2P KV 缓存共享）。它们通过 add_p2p_args() 在 lmcache server 解析器中注册。当设置了 --p2p-advertise-url 时，P2P 被启用，这还需要通过 --coordinator-url``（或 ``LMCACHE_COORDINATOR_URL）提供协调器 URL。

参数	默认	描述
--p2p-advertise-url	"" (P2P 已禁用)	传输通道服务器 host:port 该实例向对等方宣传。设置它可以启用 P2P（还需要 --coordinator-url）。
--p2p-listen-url	""	传输通道服务器 host:port 进行绑定。默认为 --p2p-advertise-url。
--p2p-lookup-timeout	30.0	在查找结果被视为未命中之前的秒数。
--p2p-load-timeout	30.0	在对等加载被视为失败之前的秒数。
--p2p-transfer-engine	nixl	使用的传输通道实现。

L1 内存管理器

来源: lmcache/v1/distributed/config.py

参数	默认	描述
--l1-size-gb	必需	L1 层的大小（以 GB 为单位）。默认情况下，设置固定 DRAM 的 L1 大小，或者在设置了 --gds-l1-path 时，设置 GDS 块文件（请参见下面的GDS L1 层）。
--l1-use-lazy / --no-l1-use-lazy	True	启用或禁用 L1 内存的延迟分配。传递 --l1-use-lazy 以启用（默认）或 --no-l1-use-lazy 以显式禁用。
--l1-init-size-gb	20	使用延迟分配时的初始分配大小（GB）。
--l1-align-bytes	4096	对齐大小（以字节为单位，默认 4 KB）。
--l1-devdax-path	(未设置)	Optional /dev/dax* device or mmap-able file to use as the L1 backing arena. When set, disable lazy allocation with --no-l1-use-lazy and leave --shm-name at its default "" (SHM transfer disabled) because the L1 bytes live in the DAX mapping. If a DAX L2 adapter with the same device_path is registered, that adapter's max_dax_size_gb is used as the L1 Device-DAX overflow size.

GDS L1 层

来源: lmcache/v1/distributed/config.py

选择加入。设置 --gds-l1-path 将 L1 媒介从固定 DRAM 切换到通过 GPUDirect Storage DMA 访问的 NVMe 块文件。然后禁用 CPU 固定 DRAM 层，并且 --l1-size-gb 设置块的大小。当此选项开启时，禁用字节数组 L2 适配器（GDS 层不提供 L1 内存缓冲区供它们注册）。

DMA 路径由平台自动选择：NVIDIA 上使用 cuFile (libcufile.so)，而 AMD ROCm 上使用 hipFile (libhipfile.so，ROCm/hipFile)。这两个平台适用相同的标志；切换供应商无需更改配置。

uGDS (libugds.so) is a third, opt-in backend selected with --gds-l1-backend ugds. It is a user-space GPUDirect Storage library that builds NVMe commands and rings doorbells from user space, so its IO path issues no syscall. LMCache can use uGDS on either NVIDIA CUDA or AMD ROCm. Each deployment must use a libugds.so built for its active platform. Unlike cuFile and hipFile, uGDS does not use a filesystem: the slab is mapped directly onto a raw character device, and --gds-l1-path must name that device (for example /dev/ugds_drv0) rather than a directory. The first --l1-size-gb bytes of the device are the slab, so the device must be at least that large and must not hold anything else.

Phoenix (libphoenix.so) is a fourth opt-in backend selected with --gds-l1-backend phx. Phoenix (phxfs) provides a kernel-mediated user-space NVMe-to-GPU DMA path with a very low software-stack overhead. Like cuFile and hipFile it uses a filesystem slab: --gds-l1-path names an NVMe directory, --gds-l1-use-direct-io applies, and the slab file can share the disk with other data. Each GPU staging buffer is registered with phxfs (phxfs_regmem, 64 KiB-aligned) and the slab is read and written with stream-ordered submissions (phxfs_read_stream / phxfs_write_stream) that keep the DMA ordered with the other work on the stream. A libphoenix build without the stream-ordered API fails to load. Follow the Phoenix installation guide to build libphoenix.so, load the phoenixfs kernel module, and verify the installation.

备注

AMD hipFile 需要 ROCm >= 7.2.0。零拷贝 GPUDirect 快速路径还需要一个使用 CONFIG_PCI_P2PDMA 构建的内核、amdgpu-dkms >= 30.20.1，以及在本地 NVMe ext4/xfs 文件系统上的 slab；如果这些不可用，hipFile 会透明地回退到主机跳跃兼容路径（正确，但不是零拷贝）。

备注

uGDS requires its kernel module loaded and the NVMe device bound to it, and a platform-matching libugds.so reachable through the loader (LD_LIBRARY_PATH or ldconfig). Because the device is claimed by ugds_drv rather than the kernel NVMe driver, it carries no filesystem and cannot be shared with any other consumer while in use. Follow the uGDS installation guide to build and load the kernel module, bind the NVMe device, build libugds.so, and verify the installation.

At startup LMCache queries the namespace capacity through uGDSGetDeviceCapacity and rejects an aligned --l1-size-gb value larger than the device. The installed libugds.so must provide this API; LMCache fails closed with an upgrade message when an older library cannot report capacity.

警告

uGDS requires an entire dedicated SSD whose contents may be destroyed. Ensure the SSD is not used for any other purpose and that its contents are not critical.

备注

Phoenix requires the phoenixfs kernel module loaded and a platform-matching libphoenix.so reachable through the loader (ldconfig or LD_LIBRARY_PATH). It has been validated on NVIDIA GPUs; other platforms require a matching libphoenix build and are not yet tested.

参数	默认	描述
--gds-l1-path	未设置	NVMe directory for the GDS L1 slab, or the raw device path when --gds-l1-backend ugds is used. Setting this enables the GDS L1 tier; with cuFile, hipFile, or phx one shared slab per process lives at <path>/lmcache_gds_slab.bin.
--gds-l1-backend	auto	GDS implementation: auto, cufile, hipfile, ugds, or phx. auto selects cuFile on CUDA and hipFile on ROCm.
--gds-l1-use-direct-io / --no-gds-l1-use-direct-io	True	Open the slab with O_DIRECT (required for the GDS DMA fast path on ext4). Ignored by ugds, whose IO bypasses the kernel entirely.

L1 管理器 TTLs

来源: lmcache/v1/distributed/config.py

参数	默认	描述
--l1-write-ttl-seconds	600	每个对象的写锁的生存时间（秒）。
--l1-read-ttl-seconds	300	每个对象的读取锁的生存时间（秒）。

逐出策略

来源: lmcache/v1/distributed/config.py

参数	默认	描述
--eviction-policy	必需	Eviction policy. Choices: LRU, ARC, IsolatedLRU, noop. ARC adaptively balances recently created keys and frequently accessed keys. It keeps key-only ghost history for completed policy evictions; no KV data is retained in the ghost lists. Use noop for buffer-only mode where L1 acts as a pure write buffer (data is deleted from L1 after L2 store). IsolatedLRU maintains one LRU list per cache_salt and requires per-cache_salt quotas to be configured at runtime via the /quota HTTP endpoints (see 配额管理); a cache_salt with no registered quota has an effective limit of 0 bytes, so its data is evicted at the next eviction cycle (allowlist semantics).
--eviction-trigger-watermark	0.8	触发逐出的内存使用比例 (0.0--1.0)。
--eviction-ratio	0.2	触发时逐出的已分配内存比例 (0.0--1.0)。

L2 策略

来源: lmcache/v1/distributed/config.py

参数	默认	描述
--l2-store-policy	default	L2 存储策略。决定每个键被写入哪些适配器，以及 L2 存储完成后是否从 L1 删除该键。default 策略将所有键写入全部适配器并保留 L1 中的数据。skip_l1 策略将所有键写入全部适配器后从 L1 删除（纯缓冲区模式）。可选项：default、skip_l1。
--l2-prefetch-policy	default	L2 预取策略。当多个适配器均持有某个键时，决定由哪个适配器加载该键。default 策略选择第一个适配器（索引最小）。预取的键是临时的（读取完成后删除）。retain 策略采用相同的加载计划，但将预取的键永久保留在 L1 中。可选项：default、retain。
--l2-prefetch-max-in-flight	8	最大并发预取（L2 加载）请求数量。限制 PrefetchController 同时发起的在途加载数，防止 L1 内存压力过大。
--periodic-notifier-interval-ms	5	定期事件通知器心跳的间隔（毫秒）。一个原生 C++ 后台线程按此间隔向所有已注册的文件描述符写入数据，以唤醒缺少原生异步完成回调的 L2 适配器的控制器轮询循环。

L2 适配器

源: lmcache/v1/distributed/l2_adapters/config.py

L2 适配器通过可重复的 --l2-adapter <JSON> 参数进行配置。每个 JSON 对象必须包含一个 "type" 字段，用于选择适配器类型。--l2-adapter 参数的顺序决定了适配器的顺序（级联）。

Registered adapter types: nixl_store, nixl_store_dynamic, fs, fs_native, mock, mooncake_store, aerospike, bigtable, sagemaker-hyperpod, s3, hfbucket, resp, valkey, plugin, native_plugin, raw_block, dax, fault_inject. (A p2p type is also registered, but it is wired in dynamically by the P2P subsystem rather than configured via --l2-adapter.)

Each adapter type's required and optional fields, plus per-backend examples, are documented on its own page under Secondary KV Storage -- including the adapters not detailed inline here (fs_native, raw_block, dax, mooncake_store, aerospike, bigtable, sagemaker-hyperpod, hfbucket, resp, valkey).

多个适配器（级联）

多次传递 --l2-adapter。适配器按给定顺序使用：

--l2-adapter '{"type": "nixl_store", "backend": "POSIX", "backend_params": {"file_path": "/data/ssd/l2", "use_direct_io": "false"}, "pool_size": 64}' \
--l2-adapter '{"type": "nixl_store", "backend": "GDS", "backend_params": {"file_path": "/data/nvme/l2", "use_direct_io": "true"}, "pool_size": 128}'

可观察性

源: lmcache/v1/mp_observability/config.py

三种模式（指标、日志记录、链路追踪）的完整说明，请参见 可观察性。

参数	默认	描述
--disable-observability	关闭	总开关：完全禁用 EventBus。
--disable-metrics	关闭	跳过指标订阅者（不暴露 Prometheus 端点）。
--disable-logging	关闭	跳过日志订阅者。
--enable-tracing	关闭	注册追踪订阅者。需要 --otlp-endpoint。
--event-bus-queue-size	10000	事件总线队列中最大事件数，超过后将进行尾部丢弃。
--otlp-endpoint	(无)	用于导出指标和链路追踪数据的 OTLP gRPC 端点。
--prometheus-port	9090	Prometheus /metrics 端点的端口。
--metrics-sample-rate	0.01	Fraction of chunks/blocks in (0, 1.0] to track for lifecycle histograms. Counters always count every event regardless of this setting.
--trace-level	(无)	Enable trace recording at the given level. Currently only storage is supported (records StorageManager public-API calls for offline replay via lmcache trace). See 追踪和调试.
--trace-output	(无)	Path to write the trace file. If omitted while --trace-level is set, a timestamped file under $TMPDIR (lmcache-trace-<pid>-<UTC>.lct) is minted and its path is logged at INFO.
--enable-extra-logging	关闭	周期性 INFO 日志：每个 GPU 的 L0<->L1 传输统计和 L1 内存使用情况。请参阅 日志记录。
--extra-logging-interval	10.0	额外日志输出之间的秒数。

vLLM 客户端配置

On the vLLM side, specify the LMCache server host and port via the kv_connector_extra_config parameter. The URL scheme selects the request transport: use tcp:// for ZMQ or grpc:// for gRPC, matching the server's --transport setting. A bare host is accepted and normalized to tcp:// for backward compatibility:

vllm serve Qwen/Qwen3-14B \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both", "kv_connector_extra_config": {"lmcache.mp.host": "127.0.0.1", "lmcache.mp.port": 6000}}'

要从单个 vLLM 部署目标多个 LMCache 服务器，通过 lmcache.mp.server_urls 传递服务器 URL 的列表（或以逗号分隔的字符串）。设置后，server_urls 优先于单服务器的 host / port 键；vLLM 的世界大小必须能被服务器数量整除，每个工作节点仅连接到其本地分配的服务器（全局排名被切分为连续块，每个服务器一个块）。多服务器模式目前仅支持张量并行 -- 管道并行（pp_size > 1）和数据并行（dp_size > 1）会被明确拒绝并报错。

vllm serve Qwen/Qwen3-14B \
    --tensor-parallel-size 4 \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both", "kv_connector_extra_config": {"lmcache.mp.server_urls": "tcp://host1:6667,tcp://host2:6667"}}'

Decode context parallelism (DCP)

--decode-context-parallel-size is supported. Under DCP, vLLM shards the attention KV cache across ranks along the token axis, so each rank holds only a strided 1/dcp slice and one block ID spans block_size * dcp tokens. LMCache stores each rank's opaque page as its own object and a chunk counts as a hit only when every rank's slice is present. Non-trivial --cp-kv-cache-interleave-size values are supported when they evenly divide every resolved attention cache block size. Because interleave changes the token-to-slot byte layout, the connector automatically adds the DCP size and interleave value to its internal cache namespace. This prevents pages written by one interleave layout from being loaded by another. It does not change the model name served by vLLM, but the decorated cache model name is visible in MP metric labels so operators can distinguish incompatible cache layouts.

One configuration change is required: the LMCache chunk size must be a multiple of vLLM's resolved scheduler block size. For a single attention group, that is block_size * decode_context_parallel_size. For a hybrid model, it is the least common multiple of every attention group's DCP-scaled block span and every recurrent-state group's unscaled physical block span. If the chunk size is incompatible, vLLM fails at connector startup with the required multiple in the message (the LMCache server itself starts fine).

The example model also needs a vLLM build that can run it under DCP: Kimi-Linear DCP support landed after v0.27.1 (vLLM commit 63ac04a61e, PR #50484). On stock v0.27.1 the command below fails at startup with Kimi-K3 MultiHeadLatentAttention does not support context parallelism.

# block_size 1024 x dcp 2 -> chunk size must be a multiple of 2048
lmcache server --host localhost --port 6000 --chunk-size 2048 \
    --l1-size-gb 20 --eviction-policy LRU

vllm serve moonshotai/Kimi-Linear-48B-A3B-Instruct \
    --trust-remote-code \
    --tensor-parallel-size 2 \
    --decode-context-parallel-size 2 \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both", "kv_connector_extra_config": {"lmcache.mp.host": "127.0.0.1", "lmcache.mp.port": 6000}}'

Pipeline parallelism and multiple LMCache servers may both be combined with DCP. These combinations are rejected at startup:

Rejected with DCP	Reason
--prefill-context-parallel-size > 1	Adds a second KV shard axis this connector does not map.
Fewer than dcp_size ranks per LMCache server	No server holds a complete set of shards, and lookup takes the minimum hit count across servers, so it reports no hits.

An interleave value that is non-positive, larger than a resolved attention cache block, or does not evenly divide every resolved attention block is rejected at connector startup.

decode_context_parallel_size > tensor_parallel_size is rejected by vLLM itself, so this connector does not re-check it.

LMCacheMPConnector 从 kv_connector_extra_config 中读取以下键：

连接器 extra_config 键

所有连接器级别的选项都通过 kv_connector_extra_config 传递，并使用 lmcache.mp. 前缀。

键	默认	描述
lmcache.mp.server_urls	(未设置)	Multi-server deployment: list (or comma-separated string) of <transport>://<host>:<port> URLs, e.g. "tcp://host1:6667,tcp://host2:6667". The transport prefix may be omitted -- bare host:port entries such as "host1:6667,host2:6667" are normalized to tcp:// by the connector. When set, takes precedence over lmcache.mp.host / lmcache.mp.port; the vLLM world size must be divisible by the number of servers, and each worker connects to its locally-assigned server.
lmcache.mp.host	tcp://localhost	Single-server deployment: request transport and host of the LMCache MP server. Use tcp:// for ZMQ or grpc:// for gRPC. A bare localhost / 127.0.0.1 is normalized to tcp:// by the connector. Ignored when lmcache.mp.server_urls is set.
lmcache.mp.port	5555	单服务器部署：LMCache MP 服务器的端口。必须与服务器的 --port 匹配。当设置了 lmcache.mp.server_urls 时将被忽略。
lmcache.mp.mq_timeout	300.0	阻塞消息队列请求的超时时间（秒），包括初始块大小查询和 KV Cache 注册/注销。如果服务器在此时间窗口内未响应，连接器将在启动时引发 ConnectionError。
lmcache.mp.heartbeat_interval	10.0	连接器向服务器发送周期性心跳 PING 的时间间隔（秒）。
lmcache.mp.nonblocking_lookup_status	true	Poll lookup-status replies without blocking the scheduler by default. Set to false to wait for each status RPC reply in the current callback, for example when long prefill steps delay observation of an already-ready reply. LOOKUP acknowledgement polling remains asynchronous. Available with the current LMCacheMPConnector.
lmcache.mp.eager_prefetch	false	Submit the LMCache lookup when a request enters vLLM's waiting queue, allowing L2-to-L1 KV staging to overlap with scheduler queue wait. Resumable requests are skipped because their token IDs may be incomplete at enqueue time.
lmcache.mp.autostart	false	Whether vLLM worker 0 should start a local lmcache server process before workers connect to it. Other local workers wait for the server to become reachable. Only localhost and 127.0.0.1 are supported. IPv6 endpoints, including ::1, raise ValueError before startup because the MP ZMQ transport does not enable IPv6 sockets. Auto-start supports exactly one server endpoint; configuring multiple lmcache.mp.server_urls raises ValueError during connector initialization.
lmcache.mp.autostart.wait_timeout	90.0	Timeout (seconds) to wait for the auto-started server to respond to ZMQ PING requests. Must be positive and finite.
lmcache.mp.autostart.server_args	""	Extra command-line arguments passed to the auto-started MP HTTP server process. Required server settings such as --l1-size-gb and --eviction-policy must be supplied here. For example, pass --l1-size-gb 20 --eviction-policy LRU. Endpoint flags such as --host, --port, and --http-host are rejected because the auto-started ZMQ and HTTP listeners are bound to the local connector endpoint. If multiple auto-started MP servers run on the same host, pass distinct --http-port values here to avoid HTTP frontend port conflicts.
lmcache.mp.mp_transfer_mode	auto	工作进程与服务器传输上下文的路由模式。可以是 auto``（CUDA -> lmcache_driven，其他 -> engine_driven）、``lmcache_driven``（强制使用 IPC / SHM 零拷贝句柄路径 — LMCache 服务器通过设备句柄拉取数据）或 ``engine_driven``（强制使用工作进程端的收集/分散拷贝路径）。当设置时，会覆盖 ``LMCACHE_MP_TRANSFER_MODE 环境变量。
lmcache.mp.isolated_ipc	false	Assume the vLLM workers and the LMCache server run in containers that share no host IPC namespace (hostIPC) and no common /dev/shm, and use IPC mechanisms that work there: on CUDA, raw CUDA IPC memory handles for KV-cache registration and timeline-semaphore events instead of CUDA interprocess event handles. Set it together with the server's --isolated-ipc flag -- a mismatch fails at event import on whichever side receives the foreign handle.
lmcache.mp.use_vmm_api	false	Set when the engine allocates its KV cache through the CUDA VMM API (vLLM's --enable-cumem-allocator): such memory has no legacy CUDA IPC handle, so KV-cache registration exports it via cuMemExportToShareableHandle instead (a fabric handle when the allocation is fabric-exportable -- requires an IMEX channel device, e.g. NVIDIA_IMEX_CHANNELS=0 -- or a POSIX fd otherwise). Composes with lmcache.mp.isolated_ipc for fabric-exportable pools; a POSIX-fd-only pool under isolated IPC is rejected at registration.
lmcache.mp.lazy_offload	false	Buffer stores on the scheduler and submit them according to the selected lazy-offload policy. Requires vLLM prefix caching. See Lazy KV Cache Offload for behavior, limitations, and tuning guidance.
lmcache.mp.lazy_offload_policy	EVICTION_AWARE	Lazy drain policy. EVICTION_AWARE drains blocks near the GPU free queue's eviction head. Set FIFO explicitly to keep the count-triggered behavior.
lmcache.mp.lazy_offload_horizon_steps	2.5	EVICTION_AWARE only: estimated scheduler steps of block consumption treated as imminent eviction. Must be greater than zero. Larger values store earlier and reduce eviction losses, but may store GPU-resident hot content and increase lower-tier eviction pressure.
lmcache.mp.lazy_offload_max_drain_per_step	64	EVICTION_AWARE only: maximum store operations emitted per scheduler step. A value below the concurrent prefill admission rate can lose buffered operations to eviction.
lmcache.mp.lazy_offload_max_deferral_seconds	0.0	EVICTION_AWARE only: how long a buffered operation may wait before it is emitted regardless of eviction pressure. Not a hard bound: no drain runs on a step that schedules no tokens, a request whose store is already in flight is skipped, and due operations that do not fit in max_drain_per_step wait for a later step. Zero leaves emission entirely to the danger window. Set it below the reuse interval the workload has to beat.
lmcache.mp.lazy_offload_threshold	100	FIFO only: number of finished buffered requests that triggers a drain.
lmcache.mp.lazy_offload_select_count	10	FIFO only: maximum finished requests emitted by one drain.

To let vLLM worker 0 start a local MP server automatically:

vllm serve Qwen/Qwen3-14B \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both", "kv_connector_extra_config": {"lmcache.mp.autostart": true, "lmcache.mp.autostart.server_args": "--l1-size-gb 20 --eviction-policy LRU"}}'

Auto-start is a convenience for single-node, single-server deployments. The MP server is a child of vLLM worker 0, not an independently managed service.

备注

LMCache's adapter shutdown does not explicitly terminate this child, but vLLM's process-tree cleanup may terminate it. Its lifetime depends on the vLLM version and exit path; neither survival nor automatic cleanup is guaranteed. Stop any remaining auto-started server when it is no longer needed.

For servers that must survive vLLM restarts or be shared across vLLM instances, and for multi-node TP/PP deployments, start and manage the server separately. For example, run the server in a separate terminal or service manager and leave auto-start disabled in vLLM:

# Terminal 1: independently managed MP server
lmcache server --host 127.0.0.1 --port 5555 \
    --http-host 127.0.0.1 --l1-size-gb 20 --eviction-policy LRU

# Terminal 2: connect-only vLLM instance
vllm serve Qwen/Qwen3-14B \
    --kv-transfer-config '{
        "kv_connector": "LMCacheMPConnector",
        "kv_connector_module_path":
            "lmcache.integration.vllm.lmcache_mp_connector",
        "kv_role": "kv_both",
        "kv_connector_extra_config": {
            "lmcache.mp.host": "127.0.0.1",
            "lmcache.mp.port": 5555,
            "lmcache.mp.autostart": false
        }
    }'

环境变量

变量	描述
LMCACHE_LOG_LEVEL	LMCache 的日志级别（DEBUG、INFO、WARNING、ERROR）。设置为 DEBUG 以查看 L2 存储活动、预取结果等。
PYTHONHASHSEED	设置为固定值以实现跨进程的可重复哈希（在使用 --hash-algorithm builtin 时相关）。
LMCACHE_TRACK_USAGE	设置为 false 以禁用匿名使用统计（见下文）。
DO_NOT_TRACK	设置为 1 以禁用匿名使用统计信息（跨工具约定）。
LMCACHE_USAGE_TRACK_INTERVAL	Seconds between continuous usage-telemetry flushes (default 600). See 使用统计收集.

完整示例

lmcache server \
    --host 0.0.0.0 \
    --port 6555 \
    --chunk-size 512 \
    --max-workers 4 \
    --max-gpu-workers 2 \
    --hash-algorithm blake3 \
    --engine-type default \
    --lookup-hash-log-dir /data/lmcache/lookup_hashes \
    --lookup-hash-log-rotation-interval 21600 \
    --lookup-hash-log-rotation-max-size 104857600 \
    --lookup-hash-log-max-files 100 \
    --l1-size-gb 100 \
    --l1-use-lazy \
    --l1-init-size-gb 20 \
    --l1-align-bytes 4096 \
    --l1-write-ttl-seconds 600 \
    --l1-read-ttl-seconds 300 \
    --eviction-policy noop \
    --l2-store-policy skip_l1 \
    --eviction-trigger-watermark 0.9 \
    --eviction-ratio 0.1 \
    --l2-prefetch-policy default \
    --l2-prefetch-max-in-flight 8 \
    --periodic-notifier-interval-ms 5 \
    --l2-adapter '{"type": "nixl_store", "backend": "POSIX", "backend_params": {"file_path": "/data/lmcache/l2", "use_direct_io": "false"}, "pool_size": 64}' \
    --prometheus-port 9090 \
    --metrics-sample-rate 0.01 \
    --enable-tracing \
    --otlp-endpoint http://localhost:4317

匿名使用统计

MP 服务器报告匿名使用统计信息：在启动时进行一次环境/配置快照，并每 LMCACHE_USAGE_TRACK_INTERVAL 秒报告一次间隔计数器（检索/存储的令牌、存储的字节、正常运行时间）。绝不会发送提示、密钥、KV Cache 数据、模型名称或 --instance-id，并且报告绝不会影响服务。可以通过 LMCACHE_TRACK_USAGE=false 或 DO_NOT_TRACK=1 选择退出；有关详细信息，请参见 使用统计收集。