配置参考

本页面记录了 LMCache 多进程服务器接受的每个 CLI 参数。参数按定义它们的配置模块进行分组。

Per-request LMCache configuration

vLLM clients can attach request-scoped LMCache metadata through the top-level kv_transfer_params field. When vLLM uses LMCacheMPConnector, entries whose keys start with lmcache. are forwarded with the request across the MP scheduler and worker IPC paths. Other kv_transfer_params entries are reserved for the transfer layer and are not forwarded to LMCache.

For example:

curl -X POST http://localhost:8000/v1/completions \
    -H "Content-Type: application/json" \
    -d '{
        "model": "Qwen/Qwen3-14B",
        "prompt": "Explain KV cache reuse.",
        "max_tokens": 32,
        "kv_transfer_params": {
            "lmcache.tag.tenant": "example-tenant",
            "lmcache.ttl": 60
        }
    }'

The connector carries these values on lookup, prefetch, store, retrieve, and lookup-lock cleanup operations so server-side features can inspect the same request metadata throughout the request lifecycle.

重要

Forwarding a value does not by itself make the MP server act on it or make it part of cache identity. The current MP server treats request configs as metadata. Do not rely on lmcache.tag.*, lmcache.ttl, lmcache.skip_save, or another request config for isolation, expiration, or cache-control behavior in MP mode unless the selected server-side feature explicitly documents support for it. The in-process LMCacheConnectorV1 may interpret these values differently.

MP 服务器

源: lmcache/v1/multiprocess/config.py

参数	默认	描述
--instance-id	(未设置，默认 UUID v4)	此 MP 服务器的稳定标识。用作协调器成员身份键，并投影到每条指标和每个 Span 的 OTel service.instance.id 资源属性（从而使遥测数据与协调器成员身份共享同一个 ID）。若未传递该参数，则默认为启动时生成的随机 UUID v4。
--host	localhost	绑定 ZMQ 服务器的主机地址。
--port	5555	绑定 ZMQ 服务器的端口。
--chunk-size	256	KV Cache 操作的块大小（以 token 为单位）。
--max-workers	1	工作线程的基本数量。为 GPU（亲和性）池和 CPU（正常）池设置默认值。可以通过 --max-gpu-workers 和 --max-cpu-workers 针对每个池进行覆盖。
--max-gpu-workers	(继承 --max-workers)	GPU 亲和性池（STORE/RETRIEVE）的工作线程。来自同一 vLLM 实例的请求始终分发到同一线程，从而消除 GPU 传输锁竞争。
--max-cpu-workers	(继承 --max-workers)	普通 CPU 池（LOOKUP 等）的工作线程。
--hash-algorithm	blake3	基于 token 的操作所使用的哈希算法。可选项：builtin、sha256_cbor、blake3。
--engine-type	default	Cache engine backend type. default uses standard prefix caching; blend composes the CacheBlend BlendModule into the engine for non-prefix KV reuse and requires --supported-transfer-mode to be lmcache_driven or auto. Choices: default, blend.
--supported-transfer-mode	lmcache_driven	Which worker → server transfer paths the server loads. lmcache_driven (default) enables only the server-driven transfer path (STORE/RETRIEVE, supports both CUDA IPC and CPU SHM); engine_driven enables only the non-GPU (PREPARE/COMMIT) transfer path; auto loads both so workers of either device type can connect without manual configuration. Choices: lmcache_driven, engine_driven, auto.
--isolated-ipc / --no-isolated-ipc	false	Assume engine workers and this server run in containers that share no host IPC namespace (hostIPC) and no common /dev/shm, and use IPC mechanisms that work there: on CUDA, raw CUDA IPC memory handles for KV-cache registration (instead of PyTorch storage IPC, which needs a shared /dev/shm) and timeline-semaphore events (instead of CUDA interprocess event handles). Must match the workers' lmcache.mp.isolated_ipc setting -- the two mechanisms exchange incompatible event handles, and a mismatch fails at event import on whichever side receives the foreign handle. Currently supported by the vLLM MP connector only; the default stays false until the integrations that still create raw CUDA interprocess events (SGLang, TensorRT-LLM, CacheBlend, qstore) migrate.
--runtime-plugin-locations	[]	零个或多个路径，用于运行时插件脚本或目录，以便与服务器一起启动。插件由 MPRuntimePluginLauncher 生成，并通过 LMCACHE_RUNTIME_PLUGIN_CONFIG 环境变量接收完整的服务器配置。
--runtime-plugin-config	"{}"	通过 LMCACHE_RUNTIME_PLUGIN_EXTRA_CONFIG 转发到运行时插件的额外键值配置的 JSON 字符串。示例：'{\"plugin.frontend.heartbeat_url\": \"http://localhost:5000/heartbeat\"}'。
--script-allowed-imports	[]	允许在 HTTP /run_script 端点发布的脚本导入的以空格分隔的 Python 模块名称列表。示例：--script-allowed-imports numpy pandas。
--shm-name	""	SHM segment name for non-GPU KV transfer (only used when the non-GPU path is loaded, i.e. --supported-transfer-mode is auto or engine_driven). "" (empty string, default): SHM disabled; KV transfer uses the pickle path. Any other value: create a SHM pool and use that exact name for its segment.
--worker-reap-timeout-seconds	120.0	工作线程发送至少一个心跳 PING 后进入静默状态的最长容忍时间（秒），超时后将回收其 KV Cache 注册，并释放泄漏的 GPU 上下文和 CUDA IPC 句柄。0 表示禁用回收。建议将此值保持在引擎适配器 lmcache.mp.heartbeat_interval（默认 10 秒）的至少 3 倍，以免少量心跳丢失导致活跃工作线程被误回收；若未同步调大此值而单独调大心跳间隔，适配器将在启动时发出警告。
--worker-registration-grace-seconds	3600.0	针对已注册但从未发送 PING 的工作线程（仍在预热或在首次请求前已崩溃）的静默容忍时间（秒）。必须大于或等于 --worker-reap-timeout-seconds。默认值较宽松，以避免将缓慢的模型预热误判为工作线程已宕机。
--enable-segmented-prefix	False	CacheBlend (--engine-type blend) 仅适用：在中间前缀 L2 检索失败时，保留缺口前缀，以便缺口后的块保持在 L1 中驻留，只有丢失的缺口被重计算，而不是在缺口处截断前缀。对其他引擎没有影响。有关如何使用它，请参见 故障注入。
--enable-dedup-content	False	--engine-type blend only: skip fingerprint registration for a chunk whose content is already indexed, so the same text stored behind two prefixes is indexed once. No effect for other engines.
--separate-object-groups / --no-separate-object-groups	False	Split a hybrid model's kernel groups into one object group per cross-chunk attention window (full attention, each sliding-window size, mamba/GDN) at KV-cache registration. Off by default; pass --separate-object-groups to enable it. Required for Mamba / linear-attention hybrids (it lets their recurrent state be cached independently, and is what allows --max-num-batched-tokens to exceed twice the block size). For a non-hybrid model it makes no difference — every layer resolves to one object group. See Hybrid Attention 模型.

查找哈希日志记录

源: lmcache/v1/mp_observability/subscribers/logging/lookup_hash.py

启用时，服务器将在 EventBus 上以 MP_LOOKUP 事件的形式发布 lookup() 期间计算的块哈希。LookupHashLoggingSubscriber 会将这些事件写入滚动 JSONL 文件以供离线分析。默认禁用。这些参数属于可观察性配置组。

参数	默认	描述
--lookup-hash-log-dir	"" (禁用)	写入查找哈希 JSONL 文件的目录。空字符串将禁用日志记录。
--lookup-hash-log-rotation-interval	21600 (6 小时)	滚动到新日志文件前的时间间隔（秒）。
--lookup-hash-log-rotation-max-size	104857600 (100 MB)	即使时间间隔未到，触发滚动的最大文件大小（字节）。
--lookup-hash-log-max-files	100	保留的最大日志文件数量。当超过此限制时，最旧的文件将被删除。

HTTP 前端

源: lmcache/v1/multiprocess/config.py

HTTP 前端在运行 lmcache server 时包含在内。

参数	默认	描述
--http-host	0.0.0.0	绑定 HTTP (FastAPI/uvicorn) 服务器的主机。
--http-port	8080	绑定 HTTP 服务器的端口。

P2P

源: lmcache/v1/multiprocess/config.py

这些标志配置 MP 服务器之间的点对点 KV Cache 共享（请参见 P2P KV 缓存共享）。它们通过 add_p2p_args() 在 lmcache server 解析器中注册。当设置了 --p2p-advertise-url 时，P2P 被启用，这还需要通过 --coordinator-url``（或 ``LMCACHE_COORDINATOR_URL）提供协调器 URL。

参数	默认	描述
--p2p-advertise-url	"" (P2P 已禁用)	传输通道服务器 host:port 该实例向对等方宣传。设置它可以启用 P2P（还需要 --coordinator-url）。
--p2p-listen-url	""	传输通道服务器 host:port 进行绑定。默认为 --p2p-advertise-url。
--p2p-lookup-timeout	30.0	在查找结果被视为未命中之前的秒数。
--p2p-load-timeout	30.0	在对等加载被视为失败之前的秒数。
--p2p-transfer-engine	nixl	使用的传输通道实现。

L1 内存管理器

来源: lmcache/v1/distributed/config.py

参数	默认	描述
--l1-size-gb	必需	L1 层的大小（以 GB 为单位）。默认情况下，设置固定 DRAM 的 L1 大小，或者在设置了 --gds-l1-path 时，设置 GDS 块文件（请参见下面的GDS L1 层）。
--l1-use-lazy / --no-l1-use-lazy	True	启用或禁用 L1 内存的延迟分配。传递 --l1-use-lazy 以启用（默认）或 --no-l1-use-lazy 以显式禁用。
--l1-init-size-gb	20	使用延迟分配时的初始分配大小（GB）。
--l1-align-bytes	4096	对齐大小（以字节为单位，默认 4 KB）。
--l1-devdax-path	(未设置)	Optional /dev/dax* device or mmap-able file to use as the L1 backing arena. When set, disable lazy allocation with --no-l1-use-lazy and leave --shm-name at its default "" (SHM transfer disabled) because the L1 bytes live in the DAX mapping. If a DAX L2 adapter with the same device_path is registered, that adapter's max_dax_size_gb is used as the L1 Device-DAX overflow size.

GDS L1 层

来源: lmcache/v1/distributed/config.py

选择加入。设置 --gds-l1-path 将 L1 媒介从固定 DRAM 切换到通过 GPUDirect Storage DMA 访问的 NVMe 块文件。然后禁用 CPU 固定 DRAM 层，并且 --l1-size-gb 设置块的大小。当此选项开启时，禁用字节数组 L2 适配器（GDS 层不提供 L1 内存缓冲区供它们注册）。

DMA 路径由平台自动选择：NVIDIA 上使用 cuFile (libcufile.so)，而 AMD ROCm 上使用 hipFile (libhipfile.so，ROCm/hipFile)。这两个平台适用相同的标志；切换供应商无需更改配置。

uGDS (libugds.so) is a third, opt-in backend selected with --gds-l1-backend ugds. It is a user-space GPUDirect Storage library that builds NVMe commands and rings doorbells from user space, so its IO path issues no syscall. LMCache can use uGDS on either NVIDIA CUDA or AMD ROCm. Each deployment must use a libugds.so built for its active platform. Unlike cuFile and hipFile, uGDS does not use a filesystem: the slab is mapped directly onto a raw character device, and --gds-l1-path must name that device (for example /dev/ugds_drv0) rather than a directory. The first --l1-size-gb bytes of the device are the slab, so the device must be at least that large and must not hold anything else.

Phoenix (libphoenix.so) is a fourth opt-in backend selected with --gds-l1-backend phx. Phoenix (phxfs) provides a kernel-mediated user-space NVMe-to-GPU DMA path with a very low software-stack overhead. Like cuFile and hipFile it uses a filesystem slab: --gds-l1-path names an NVMe directory, --gds-l1-use-direct-io applies, and the slab file can share the disk with other data. Each GPU staging buffer is registered with phxfs (phxfs_regmem, 64 KiB-aligned) and the slab is read and written with stream-ordered submissions (phxfs_read_stream / phxfs_write_stream) that keep the DMA ordered with the other work on the stream. A libphoenix build without the stream-ordered API fails to load. Follow the Phoenix installation guide to build libphoenix.so, load the phoenixfs kernel module, and verify the installation.

备注

AMD hipFile 需要 ROCm >= 7.2.0。零拷贝 GPUDirect 快速路径还需要一个使用 CONFIG_PCI_P2PDMA 构建的内核、amdgpu-dkms >= 30.20.1，以及在本地 NVMe ext4/xfs 文件系统上的 slab；如果这些不可用，hipFile 会透明地回退到主机跳跃兼容路径（正确，但不是零拷贝）。

备注

uGDS requires its kernel module loaded and the NVMe device bound to it, and a platform-matching libugds.so reachable through the loader (LD_LIBRARY_PATH or ldconfig). Because the device is claimed by ugds_drv rather than the kernel NVMe driver, it carries no filesystem and cannot be shared with any other consumer while in use. Follow the uGDS installation guide to build and load the kernel module, bind the NVMe device, build libugds.so, and verify the installation.

At startup LMCache queries the namespace capacity through uGDSGetDeviceCapacity and rejects an aligned --l1-size-gb value larger than the device. The installed libugds.so must provide this API; LMCache fails closed with an upgrade message when an older library cannot report capacity.

警告

uGDS requires an entire dedicated SSD whose contents may be destroyed. Ensure the SSD is not used for any other purpose and that its contents are not critical.

备注

Phoenix requires the phoenixfs kernel module loaded and a platform-matching libphoenix.so reachable through the loader (ldconfig or LD_LIBRARY_PATH). It has been validated on NVIDIA GPUs; other platforms require a matching libphoenix build and are not yet tested.

参数	默认	描述
--gds-l1-path	未设置	NVMe directory for the GDS L1 slab, or the raw device path when --gds-l1-backend ugds is used. Setting this enables the GDS L1 tier; with cuFile, hipFile, or phx one shared slab per process lives at <path>/lmcache_gds_slab.bin.
--gds-l1-backend	auto	GDS implementation: auto, cufile, hipfile, ugds, or phx. auto selects cuFile on CUDA and hipFile on ROCm.
--gds-l1-use-direct-io / --no-gds-l1-use-direct-io	True	Open the slab with O_DIRECT (required for the GDS DMA fast path on ext4). Ignored by ugds, whose IO bypasses the kernel entirely.

L1 管理器 TTLs

来源: lmcache/v1/distributed/config.py

参数	默认	描述
--l1-write-ttl-seconds	600	每个对象的写锁的生存时间（秒）。
--l1-read-ttl-seconds	300	每个对象的读取锁的生存时间（秒）。

逐出策略

来源: lmcache/v1/distributed/config.py

参数	默认	描述
--eviction-policy	必需	逐出策略。可选项：LRU、IsolatedLRU、noop。noop 用于纯缓冲区模式，此时 L1 仅作为写缓冲区（数据写入 L2 后即从 L1 删除）。IsolatedLRU 为每个 cache_salt 维护独立的 LRU 列表，并要求在运行时通过 /quota HTTP 端点为每个 cache_salt 配置配额（参见 配额管理）；未注册配额的 cache_salt 的有效上限为 0 字节，其数据将在下一个逐出周期被清除（白名单语义）。
--eviction-trigger-watermark	0.8	触发逐出的内存使用比例 (0.0--1.0)。
--eviction-ratio	0.2	触发时逐出的已分配内存比例 (0.0--1.0)。

L2 策略

来源: lmcache/v1/distributed/config.py

参数	默认	描述
--l2-store-policy	default	L2 存储策略。决定每个键被写入哪些适配器，以及 L2 存储完成后是否从 L1 删除该键。default 策略将所有键写入全部适配器并保留 L1 中的数据。skip_l1 策略将所有键写入全部适配器后从 L1 删除（纯缓冲区模式）。可选项：default、skip_l1。
--l2-prefetch-policy	default	L2 预取策略。当多个适配器均持有某个键时，决定由哪个适配器加载该键。default 策略选择第一个适配器（索引最小）。预取的键是临时的（读取完成后删除）。retain 策略采用相同的加载计划，但将预取的键永久保留在 L1 中。可选项：default、retain。
--l2-prefetch-max-in-flight	8	最大并发预取（L2 加载）请求数量。限制 PrefetchController 同时发起的在途加载数，防止 L1 内存压力过大。
--periodic-notifier-interval-ms	5	定期事件通知器心跳的间隔（毫秒）。一个原生 C++ 后台线程按此间隔向所有已注册的文件描述符写入数据，以唤醒缺少原生异步完成回调的 L2 适配器的控制器轮询循环。

L2 适配器

源: lmcache/v1/distributed/l2_adapters/config.py

L2 适配器通过可重复的 --l2-adapter <JSON> 参数进行配置。每个 JSON 对象必须包含一个 "type" 字段，用于选择适配器类型。--l2-adapter 参数的顺序决定了适配器的顺序（级联）。

Registered adapter types: nixl_store, nixl_store_dynamic, fs, fs_native, mock, mooncake_store, aerospike, bigtable, sagemaker-hyperpod, s3, hfbucket, resp, valkey, plugin, native_plugin, raw_block, dax, fault_inject. (A p2p type is also registered, but it is wired in dynamically by the P2P subsystem rather than configured via --l2-adapter.)

Each adapter type's required and optional fields, plus per-backend examples, are documented on its own page under Secondary KV Storage -- including the adapters not detailed inline here (fs_native, raw_block, dax, mooncake_store, aerospike, bigtable, sagemaker-hyperpod, hfbucket, resp, valkey).

多个适配器（级联）

多次传递 --l2-adapter。适配器按给定顺序使用：

--l2-adapter '{"type": "nixl_store", "backend": "POSIX", "backend_params": {"file_path": "/data/ssd/l2", "use_direct_io": "false"}, "pool_size": 64}' \
--l2-adapter '{"type": "nixl_store", "backend": "GDS", "backend_params": {"file_path": "/data/nvme/l2", "use_direct_io": "true"}, "pool_size": 128}'

可观察性

源: lmcache/v1/mp_observability/config.py

三种模式（指标、日志记录、链路追踪）的完整说明，请参见 可观察性。

参数	默认	描述
--disable-observability	关闭	总开关：完全禁用 EventBus。
--disable-metrics	关闭	跳过指标订阅者（不暴露 Prometheus 端点）。
--disable-logging	关闭	跳过日志订阅者。
--enable-tracing	关闭	注册追踪订阅者。需要 --otlp-endpoint。
--event-bus-queue-size	10000	事件总线队列中最大事件数，超过后将进行尾部丢弃。
--otlp-endpoint	(无)	用于导出指标和链路追踪数据的 OTLP gRPC 端点。
--prometheus-port	9090	Prometheus /metrics 端点的端口。
--metrics-sample-rate	0.01	Fraction of chunks/blocks in (0, 1.0] to track for lifecycle histograms. Counters always count every event regardless of this setting.
--trace-level	(无)	Enable trace recording at the given level. Currently only storage is supported (records StorageManager public-API calls for offline replay via lmcache trace). See 追踪和调试.
--trace-output	(无)	Path to write the trace file. If omitted while --trace-level is set, a timestamped file under $TMPDIR (lmcache-trace-<pid>-<UTC>.lct) is minted and its path is logged at INFO.
--enable-extra-logging	关闭	周期性 INFO 日志：每个 GPU 的 L0<->L1 传输统计和 L1 内存使用情况。请参阅 日志记录。
--extra-logging-interval	10.0	额外日志输出之间的秒数。

vLLM 客户端配置

On the vLLM side, specify the LMCache server host and port via the kv_connector_extra_config parameter. The tcp:// transport prefix on lmcache.mp.host is optional -- a bare host is accepted and normalized to tcp:// by the connector:

vllm serve Qwen/Qwen3-14B \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both", "kv_connector_extra_config": {"lmcache.mp.host": "127.0.0.1", "lmcache.mp.port": 6000}}'

要从单个 vLLM 部署目标多个 LMCache 服务器，通过 lmcache.mp.server_urls 传递服务器 URL 的列表（或以逗号分隔的字符串）。设置后，server_urls 优先于单服务器的 host / port 键；vLLM 的世界大小必须能被服务器数量整除，每个工作节点仅连接到其本地分配的服务器（全局排名被切分为连续块，每个服务器一个块）。多服务器模式目前仅支持张量并行 -- 管道并行（pp_size > 1）和数据并行（dp_size > 1）会被明确拒绝并报错。

vllm serve Qwen/Qwen3-14B \
    --tensor-parallel-size 4 \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both", "kv_connector_extra_config": {"lmcache.mp.server_urls": "tcp://host1:6667,tcp://host2:6667"}}'

Decode context parallelism (DCP)

--decode-context-parallel-size is supported. Under DCP, vLLM shards the attention KV cache across ranks along the token axis, so each rank holds only a strided 1/dcp slice and one block ID spans block_size * dcp tokens. LMCache stores each rank's opaque page as its own object and a chunk counts as a hit only when every rank's slice is present. Non-trivial --cp-kv-cache-interleave-size values are supported when they evenly divide every resolved attention cache block size. Because interleave changes the token-to-slot byte layout, the connector automatically adds the DCP size and interleave value to its internal cache namespace. This prevents pages written by one interleave layout from being loaded by another. It does not change the model name served by vLLM, but the decorated cache model name is visible in MP metric labels so operators can distinguish incompatible cache layouts.

One configuration change is required: the LMCache chunk size must be a multiple of vLLM's resolved scheduler block size. For a single attention group, that is block_size * decode_context_parallel_size. For a hybrid model, it is the least common multiple of every attention group's DCP-scaled block span and every recurrent-state group's unscaled physical block span. If the chunk size is incompatible, vLLM fails at connector startup with the required multiple in the message (the LMCache server itself starts fine).

The example model also needs a vLLM build that can run it under DCP: Kimi-Linear DCP support landed after v0.27.1 (vLLM commit 63ac04a61e, PR #50484). On stock v0.27.1 the command below fails at startup with Kimi-K3 MultiHeadLatentAttention does not support context parallelism.

# block_size 1024 x dcp 2 -> chunk size must be a multiple of 2048
lmcache server --host localhost --port 6000 --chunk-size 2048 \
    --l1-size-gb 20 --eviction-policy LRU

vllm serve moonshotai/Kimi-Linear-48B-A3B-Instruct \
    --trust-remote-code \
    --tensor-parallel-size 2 \
    --decode-context-parallel-size 2 \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both", "kv_connector_extra_config": {"lmcache.mp.host": "127.0.0.1", "lmcache.mp.port": 6000}}'

Pipeline parallelism and multiple LMCache servers may both be combined with DCP. These combinations are rejected at startup:

Rejected with DCP	Reason
--prefill-context-parallel-size > 1	Adds a second KV shard axis this connector does not map.
Fewer than dcp_size ranks per LMCache server	No server holds a complete set of shards, and lookup takes the minimum hit count across servers, so it reports no hits.

An interleave value that is non-positive, larger than a resolved attention cache block, or does not evenly divide every resolved attention block is rejected at connector startup.

decode_context_parallel_size > tensor_parallel_size is rejected by vLLM itself, so this connector does not re-check it.

LMCacheMPConnector 从 kv_connector_extra_config 中读取以下键：

连接器 extra_config 键

所有连接器级别的选项都通过 kv_connector_extra_config 传递，并使用 lmcache.mp. 前缀。

键	默认	描述
lmcache.mp.server_urls	(未设置)	Multi-server deployment: list (or comma-separated string) of <transport>://<host>:<port> URLs, e.g. "tcp://host1:6667,tcp://host2:6667". The transport prefix may be omitted -- bare host:port entries such as "host1:6667,host2:6667" are normalized to tcp:// by the connector. When set, takes precedence over lmcache.mp.host / lmcache.mp.port; the vLLM world size must be divisible by the number of servers, and each worker connects to its locally-assigned server.
lmcache.mp.host	tcp://localhost	Single-server deployment: host of the LMCache MP server. A ZMQ transport prefix (e.g. tcp://) is optional -- a bare localhost / 127.0.0.1 is normalized to tcp:// by the connector. Ignored when lmcache.mp.server_urls is set.
lmcache.mp.port	5555	单服务器部署：LMCache MP 服务器的端口。必须与服务器的 --port 匹配。当设置了 lmcache.mp.server_urls 时将被忽略。
lmcache.mp.mq_timeout	300.0	阻塞消息队列请求的超时时间（秒），包括初始块大小查询和 KV Cache 注册/注销。如果服务器在此时间窗口内未响应，连接器将在启动时引发 ConnectionError。
lmcache.mp.heartbeat_interval	10.0	连接器向服务器发送周期性心跳 PING 的时间间隔（秒）。
lmcache.mp.eager_prefetch	false	Submit the LMCache lookup when a request enters vLLM's waiting queue, allowing L2-to-L1 KV staging to overlap with scheduler queue wait. Resumable requests are skipped because their token IDs may be incomplete at enqueue time.
lmcache.mp.lazy_offload	false	Defer store operations and submit finished requests in FIFO batches. Available only with vLLM and LMCacheMPConnector. See Lazy KV Cache Offload for behavior, limitations, and tuning guidance.
lmcache.mp.lazy_offload_policy	FIFO	Policy used to select finished pending requests. FIFO is currently the only supported value. Used only when lazy offload is enabled.
lmcache.mp.lazy_offload_threshold	100	Number of finished pending requests required before a lazy-offload batch becomes eligible for submission.
lmcache.mp.lazy_offload_select_count	10	Maximum number of finished requests selected each time the lazy-offload threshold is met.
lmcache.mp.mp_transfer_mode	auto	工作进程与服务器传输上下文的路由模式。可以是 auto``（CUDA -> lmcache_driven，其他 -> engine_driven）、``lmcache_driven``（强制使用 IPC / SHM 零拷贝句柄路径 — LMCache 服务器通过设备句柄拉取数据）或 ``engine_driven``（强制使用工作进程端的收集/分散拷贝路径）。当设置时，会覆盖 ``LMCACHE_MP_TRANSFER_MODE 环境变量。
lmcache.mp.isolated_ipc	false	Assume the vLLM workers and the LMCache server run in containers that share no host IPC namespace (hostIPC) and no common /dev/shm, and use IPC mechanisms that work there: on CUDA, raw CUDA IPC memory handles for KV-cache registration and timeline-semaphore events instead of CUDA interprocess event handles. Set it together with the server's --isolated-ipc flag -- a mismatch fails at event import on whichever side receives the foreign handle.

环境变量

变量	描述
LMCACHE_LOG_LEVEL	LMCache 的日志级别（DEBUG、INFO、WARNING、ERROR）。设置为 DEBUG 以查看 L2 存储活动、预取结果等。
PYTHONHASHSEED	设置为固定值以实现跨进程的可重复哈希（在使用 --hash-algorithm builtin 时相关）。
LMCACHE_TRACK_USAGE	设置为 false 以禁用匿名使用统计（见下文）。
DO_NOT_TRACK	设置为 1 以禁用匿名使用统计信息（跨工具约定）。
LMCACHE_USAGE_TRACK_INTERVAL	Seconds between continuous usage-telemetry flushes (default 600). See 使用统计收集.

完整示例

lmcache server \
    --host 0.0.0.0 \
    --port 6555 \
    --chunk-size 512 \
    --max-workers 4 \
    --max-gpu-workers 2 \
    --hash-algorithm blake3 \
    --engine-type default \
    --lookup-hash-log-dir /data/lmcache/lookup_hashes \
    --lookup-hash-log-rotation-interval 21600 \
    --lookup-hash-log-rotation-max-size 104857600 \
    --lookup-hash-log-max-files 100 \
    --l1-size-gb 100 \
    --l1-use-lazy \
    --l1-init-size-gb 20 \
    --l1-align-bytes 4096 \
    --l1-write-ttl-seconds 600 \
    --l1-read-ttl-seconds 300 \
    --eviction-policy noop \
    --l2-store-policy skip_l1 \
    --eviction-trigger-watermark 0.9 \
    --eviction-ratio 0.1 \
    --l2-prefetch-policy default \
    --l2-prefetch-max-in-flight 8 \
    --periodic-notifier-interval-ms 5 \
    --l2-adapter '{"type": "nixl_store", "backend": "POSIX", "backend_params": {"file_path": "/data/lmcache/l2", "use_direct_io": "false"}, "pool_size": 64}' \
    --prometheus-port 9090 \
    --metrics-sample-rate 0.01 \
    --enable-tracing \
    --otlp-endpoint http://localhost:4317

匿名使用统计

MP 服务器报告匿名使用统计信息：在启动时进行一次环境/配置快照，并每 LMCACHE_USAGE_TRACK_INTERVAL 秒报告一次间隔计数器（检索/存储的令牌、存储的字节、正常运行时间）。绝不会发送提示、密钥、KV Cache 数据、模型名称或 --instance-id，并且报告绝不会影响服务。可以通过 LMCACHE_TRACK_USAGE=false 或 DO_NOT_TRACK=1 选择退出；有关详细信息，请参见 使用统计收集。