lmcache trace

The lmcache trace command inspects and replays LMCache trace files (.lct). A file holds one trace level: storage records StorageManager calls; events records the cache-event stream a server emits for the MP coordinator. It has three sub-commands:

lmcache trace {info,replay,replay-events} FILE [options]

备注

lmcache trace 需要完整的 lmcache 包（StorageManager、trace 编解码器、TraceReader）。在轻量级的 lmcache-cli 安装中不可用，如果缺少这些模块，则会以状态 2 退出。

Trace capture is not a trace sub-command — recording is bound to a live server via lmcache server --trace-level {storage,events} [--trace-output ...] (see lmcache server).

信息

Print a one-screen summary of a trace file: header metadata plus per-qualname record counts. Works for both levels.

lmcache trace info path/to/trace.lct

A storage file:

Trace file: path/to/trace.lct
  level:                      storage
  format_version:             1
  trace_schema_version:       1
  duration:                   12.345s
  sm_config_digest:           a1b2c3d4
  total_records:              2048
  ops:
    StorageManager.store: 1024
    StorageManager.retrieve: 1024

An events file adds the header metadata the level records and a summary of the stream: which server wrote it and how many times it restarted (one incarnation per start), batches by event type, tier, backend and whether the backend is shared, entries per event type, bytes stored, and how many store entries carry token ids.

Trace file: events-node-a.lct
  level:                      events
  format_version:             1
  trace_schema_version:       1
  duration:                   612.480s
  sm_config_digest:           7a10b9a5c8e8344d32b5f8b7a530327a19f189439f5d2d5151065fd5d19ecff8
  cache_event_schema_version: 1
  instance_id:                node-a
  lmcache_version:            0.5.6
  total_records:              8
  ops:
    events.batch: 6
    events.lifecycle: 2
  events:
    instances:                  1
      node-a: incarnations=[1758400000] restarts=0
    batches (type/tier/backend/shared):
      access/l1/-/local: 1
      delete/l1/dram/local: 1
      store/l1/dram/local: 2
      store/l2/fs/local: 1
      store/l2/s3/shared: 1
    entries:
      access: 30
      delete: 12
      store: 128
    store_bytes:                113246208
    entries_with_tokens:        64
    lifecycle:                  start=1, stop=1

lifecycle: start=1, stop=1 says the server started once and shut down cleanly; a missing stop means the process was killed and the tail may be truncated. restarts counts incarnations beyond the first.

唯一的参数是位置参数 FILE（指向 .lct 跟踪文件的路径）。

重放

Reissue every recorded call against a fresh StorageManager, honoring the recorded inter-call timings. replay accepts storage files only; an events file is refused with a message saying so, since its records are coordinator input rather than storage calls (see replay-events).

lmcache trace replay path/to/trace.lct \
    --l1-size-gb 10 --eviction-policy LRU

replay 接受标准的存储管理器配置标志（--l1-size-gb, --eviction-policy, --l2-...）；有关完整列表，请参见 lmcache server --help。重放侧的配置可能与记录在追踪中的配置不同，这可能会导致合法的查找未命中。

警告

重放环境不匹配可能导致查找未命中。重放使用重放端的 StorageManager 配置（可能与记录的配置不同），在性能可能与记录主机不同的主机上运行，并且 StorageManager 的读/写是异步的。将查找未命中计数视为重放环境的信号，而不是记录中的缺陷。

选项

标志	描述
FILE	`.lct` 跟踪文件的路径（位置参数，必填）。
--verbose	每个重放记录打印一行。
--jsonl-out PATH	将每个重放记录写入一个 JSON 对象到 PATH（qualname，延迟_ms，失败）。
--output-dir DIR	聚合的 CSV / JSON 摘要输出目录（默认：当前目录）。
--no-csv	跳过聚合的 CSV 摘要导出。
--json	还可以导出一个汇总的 JSON。
-q / --quiet	抑制终端指标表（文件仍然会被写入）。

终端摘要报告整体重放统计信息（重放/跳过/失败的记录、持续时间、配置摘要匹配）和每个操作的延迟百分位数。如果任何记录失败，replay 将以状态 1 退出。

replay-events

Deliver the cache-event stream an events file holds to a coordinator, the way the server that wrote it would have. Several files, one per server, replay as one fleet, merged by wall-clock time:

lmcache trace replay-events node-a.lct node-b.lct \
    --coordinator-url http://coordinator:9300

Each start mark registers its server with the coordinator (at the address the server recorded, or loopback) and begins heartbeating it every --heartbeat-interval seconds, each batch is one POST /events carrying the record unchanged, and each stop mark deregisters. A file that ends without a stop is a server that died or a recording that was cut: its heartbeats end with its last record, and the coordinator retires it after its instance timeout, as it would a real crash. By default the stream goes as fast as the coordinator takes it; --speed 1 replays at the recorded rate. The servers are not there to be called back, so a replay exercises ingest and the views (directory, usage, declared capacity), not a controller's deletes or prefetches; run the coordinator with its eviction loop off to keep it from addressing them. The summary says how the coordinator counted the batches: applied, duplicate (a sequence it had already admitted) and stale (an incarnation it had already moved past).

警告

Replay into a coordinator that does not have the same fleet live. Registering a recorded server replaces a live one's registration, and the coordinator would address the live server at the recorded address.

选项

标志	描述
FILE	One or more events-level .lct files (positional, required).
--coordinator-url URL	The coordinator to deliver the stream to (required).
--speed N	Pacing relative to the recording; 1 is the recorded rate, 0 (default) as fast as the coordinator takes it.
--heartbeat-interval SECONDS	Heartbeat each registered server this often (default 5, below the coordinator's 30 s instance timeout); 0 sends none.
-q / --quiet	Suppress the terminal summary.

replay-events exits with status 2 if a file is not an events trace this build can replay, and with status 1 if the coordinator cannot be reached or refuses a call; the replay stops there rather than skip records, since a partly delivered stream would leave views that look complete and are not. The per-record log says how far it got.