Raw Block (Rust)#

A built-in L2 adapter that stores KV objects in fixed-size slots on a raw block device or pre-sized file using the Rust raw-device I/O bindings. It reuses the existing raw-block metadata checkpoint model and writes directly into the caller-provided load buffers during prefetch.

Required fields:

  • device_path: Raw device path or pre-sized file path.

  • slot_bytes: Fixed slot size in bytes. Must be aligned to block_align.

Optional fields:

  • capacity_bytes: Optional cap on the usable device bytes. Default 0 means use the full device/file size.

  • use_odirect: true or false (default true).

  • block_align: Device alignment in bytes (default 4096). Must be a power of two.

  • header_bytes: Per-slot header reservation (default 4096).

  • meta_total_bytes: Reserved metadata checkpoint region (default 256MiB).

  • meta_magic / meta_version: Metadata checkpoint identity/version knobs.

  • meta_checkpoint_interval_sec / meta_idle_quiet_ms / meta_enable_periodic / meta_verify_on_load: Checkpoint and recovery controls carried over from the legacy raw-block backend.

  • load_checkpoint_on_init: Load an existing on-device metadata checkpoint during startup (default true). Set to false to start with an empty in-memory index instead.

  • enable_zero_copy: Try aligned direct-buffer I/O when possible. With io_engine="io_uring", also try to register eligible L1 memory as fixed buffers.

  • io_engine: Rust raw-block I/O engine. Valid values are "posix" (default synchronous pread/pwrite path), "io_uring" (direct Rust io_uring syscall path).

  • use_uring_cmd: Enable NVMe passthrough via io_uring command interface for direct device access. Requires io_engine="io_uring" and NVMe character device node (e.g., /dev/ng0n1).

  • iouring_queue_depth: Queue depth for io_engine="io_uring".

  • max_data_transfer_size: Maximum data transfer size for use_uring_cmd=true. Large transfers are split into smaller chunks that fit within device limits. When left unset (or <= 0), it is auto-detected from the device’s sysfs queue limits, bounded by both max_hw_sectors_kb and max_segments * page_size (the NVMe passthrough path uses one scatter-gather segment per page).

  • fdp_enabled: Enables NVMe Flexible Data Placement (FDP) discovery and non-zero placement identifier registration. cache_salt values with ":" bucket prefixes use FDP placement by default. Requires io_engine="io_uring" and use_uring_cmd=true.

  • fdp_placement_ids: Optional non-zero placement identifier list for KV data placement. If omitted, the adapter uses all device-reported non-zero identifiers except meta_checkpoint_placement_id.

  • fdp_data_placement_policy: KV data placement policy. "none" omits FDP directives for KV data writes. "cache_salt_prefix" assigns case-insensitive cache_salt prefixes to FDP placement identifiers. "cache_salt_rank" keeps that bucket isolation and further separates rank-local write streams within each bucket when identifiers are available. The default is "cache_salt_prefix" when fdp_enabled=true.

  • fdp_slot_reuse_policy: Free-slot reuse policy. "pid_affinity" prefers a free slot last assigned to the same FDP placement identifier before falling back to any free slot. "none" disables affinity-based reuse. The default is "pid_affinity" when fdp_enabled=true and "none" otherwise.

  • meta_checkpoint_placement_id: Optional non-zero placement identifier for metadata checkpoint payload/header writes. Omit it to keep checkpoint writes on default NVMe placement.

  • num_store_workers / num_lookup_workers / num_load_workers: Worker-thread counts for each operation type.

Notes:

  • raw_block is a server-owned MP adapter. It does not support per-TP device-path mappings in MP mode.

  • raw_block remains "type": "raw_block" for all supported engines.

  • raw_block owns on-device slot allocation, checkpointing, and recovery through RawBlockCore. Slot reclamation is driven by the shared/global L2 eviction controller or explicit delete() calls.

  • POSIX restart recovery validates slot headers with an internal pool of 8 reader threads. Both regular io_uring and io_uring_cmd batch header reads up to iouring_queue_depth.

  • slot_bytes, header_bytes, and meta_total_bytes must be multiples of block_align.

  • If use_odirect or use_uring_cmd is enabled, the server’s --l1-align-bytes must be at least block_align.

  • With O_DIRECT, raw-block I/O rejects offsets and total I/O lengths that are not multiples of block_align. Misaligned write buffers use an aligned bounce buffer.

  • persist_enabled must remain true for this adapter.

  • For use_uring_cmd=true, device_path must use the NVMe character device node (e.g., /dev/ng0n1) instead of the block device node (/dev/nvme0n1). The character device provides direct NVMe command passthrough.

  • For use_uring_cmd=true, block_align must be a multiple of the NVMe namespace LBA size. An incompatible value is rejected when the device opens.

  • use_uring_cmd requires io_engine="io_uring" to be set.

  • When use_uring_cmd=true, use_odirect is ignored for NVMe namespace character devices. FDP examples set use_odirect=false because io_uring_cmd uses NVMe passthrough rather than the POSIX write path.

  • FDP registers only non-zero placement identifiers. fdp_placement_ids is the KV data placement pool: if omitted, all discovered non-zero identifiers except meta_checkpoint_placement_id are used; if provided, every identifier must be reported by the device and must not contain 0.

  • meta_checkpoint_placement_id must not overlap with fdp_placement_ids. Keeping metadata checkpoints and KV data on separate placement identifiers avoids mixing long-lived raw-block metadata with cache data buckets.

  • With fdp_data_placement_policy="cache_salt_prefix", cache_salt values containing ":" are bucketed by the prefix before ":". The bucket name is case-insensitive, so RAG:app1, rag:app2, and rag: share the same bucket. Values without ":" and values with an empty prefix use no FDP directive. Buckets are assigned exclusive FDP placement identifiers in first-seen order while identifiers are available.

  • If the number of discovered cache-salt buckets exceeds the number of registered FDP placement identifiers, additional buckets continue to store with no FDP directive and the adapter emits one warning. report_status() exposes a fallback count and a bounded bucket sample instead of retaining the complete fallback bucket set. Empty cache_salt values also use default NVMe placement with no directive.

  • With fdp_data_placement_policy="cache_salt_rank", placement is keyed by the case-insensitive cache_salt bucket prefix and the local rank encoded in kv_rank. The suffix after ":" remains part of the object key but does not affect FDP placement. For example, batch:app-a and batch:app-b share the batch bucket: local rank 0 from both applications shares one placement identifier, local rank 1 shares another, and so on. Each bucket can use at most the detected visible GPU count worth of placement identifiers. Extra ranks, or buckets that cannot obtain an identifier, store with no FDP directive.

  • Sharing placement identifiers within a bucket can be intentional because the available FDP placement-identifier pool is finite. Operators can assign the same prefix to applications with similar cache lifetimes, or use a distinct prefix for each application when placement isolation is required. For example, app01:<suffix> through app16:<suffix> can provide 16 application buckets with eight rank streams each, consuming up to 128 placement identifiers when at least 128 identifiers are available and the adapter detects at least eight visible GPUs. Applications using the same prefix share the placement identifiers for equal local ranks instead of receiving separate sets.

  • Bucket-to-placement assignments are process-local. Restart recovery does not need them for correctness because cache_salt is part of the object key, but first-seen FDP placement assignments may change after adapter restart.

  • Slot affinity is process-local and is not stored in metadata checkpoints. After recovery, free slots have no recorded affinity until they are reused.

  • Store and retrieve requests must use the same cache_salt to address the same object. This is the normal LMCache key identity rule; FDP placement is a write directive and is not used to locate data on reads.

  • Metadata checkpoint writes use meta_checkpoint_placement_id when configured, otherwise they use default NVMe placement with no directive.

Fixed-buffer registration and memlock:

With io_engine="io_uring" and enable_zero_copy=true, the adapter registers the stable L1 memory range during initialization. An anonymous MixedMemoryAllocator exposes its full arena; LazyMemoryAllocator exposes only the currently pinned prefix. POSIX shared-memory arenas do not expose an eligible range. Registration uses regions no larger than 1 GiB and does not grow when the lazy allocator expands.

The kernel charges registered buffers against RLIMIT_MEMLOCK unless the process has CAP_IPC_LOCK. The default lazy initial prefix is 20 GiB, so allow at least that much locked memory, plus headroom for other registrations and locked memory. Splitting the prefix into 1 GiB regions does not reduce the total memlock requirement. See the Linux io_uring_register(2) manual.

For a Bash-launched server, inspect the soft and hard limits in KiB and set the soft limit before starting the server from the same shell:

ulimit -Sl
ulimit -Hl
# 20 GiB in KiB; requires a hard limit of at least this value.
ulimit -Sl 20971520

If the hard limit is too low, raise it through the service, container, or login configuration. A systemd service can use LimitMEMLOCK=infinity; a Docker container can use --ulimit memlock=-1:-1. Apply the setting to the process running the LMCache server, then restart it. Registration is attempted once during adapter initialization.

If registration fails, the adapter logs a warning and continues with ordinary io_uring. The adapter can therefore remain healthy even when fixed buffers are inactive. Check RawBlockL2Adapter.report_status()["core"] for:

  • fixed_buffers_registered: true only after successful kernel registration.

  • fixed_buffer_registered_bytes: Total bytes in the registered regions; 0 when no buffers are registered. This is the registered range size, not the final L1 capacity or the number of bytes transferred.

Both fields also appear under each raw-block adapter in GET /status:

curl -fsS http://localhost:8080/status | jq '
  .storage_manager.l2_adapters[]
  | select(.type == "RawBlockL2Adapter")
  | .core
  | {fixed_buffers_registered, fixed_buffer_registered_bytes}'

For a successfully registered 20 GiB prefix, the result is:

{
  "fixed_buffers_registered": true,
  "fixed_buffer_registered_bytes": 21474836480
}

Failed or skipped registration reports false and 0, as does a closed adapter. Successful registration does not mean every request uses fixed I/O: requests outside the registered prefix or crossing a region boundary still use ordinary io_uring. When comparing performance, verify these fields and keep enable_zero_copy enabled in both cases to isolate registration from direct-buffer I/O.

Configuration examples:

# Basic raw_block with posix I/O
--l2-adapter '{"type": "raw_block", "device_path": "/dev/nvme0n1", "slot_bytes": 1048576, "block_align": 4096, "header_bytes": 4096, "meta_total_bytes": 268435456, "use_odirect": true, "num_store_workers": 2, "num_lookup_workers": 1, "num_load_workers": 4}'

# With io_uring
--l2-adapter '{"type": "raw_block", "device_path": "/dev/nvme0n1", "slot_bytes": 1048576, "io_engine": "io_uring", "iouring_queue_depth": 256, "use_odirect": true}'

# With io_uring_cmd (NVMe passthrough)
--l2-adapter '{"type": "raw_block", "device_path": "/dev/ng0n1", "slot_bytes": 1048576, "io_engine": "io_uring", "use_uring_cmd": true, "iouring_queue_depth": 256, "max_data_transfer_size": 131072, "use_odirect": false}'

# With FDP discovery and cache_salt prefix placement enabled
--l2-adapter '{"type": "raw_block", "device_path": "/dev/ng0n1", "slot_bytes": 1048576, "io_engine": "io_uring", "use_uring_cmd": true, "fdp_enabled": true, "use_odirect": false}'

# With FDP discovery and cache_salt/rank placement enabled
--l2-adapter '{"type": "raw_block", "device_path": "/dev/ng0n1", "slot_bytes": 1048576, "io_engine": "io_uring", "use_uring_cmd": true, "fdp_enabled": true, "fdp_data_placement_policy": "cache_salt_rank", "use_odirect": false}'

# With FDP discovery only, keeping KV data writes on default NVMe placement
--l2-adapter '{"type": "raw_block", "device_path": "/dev/ng0n1", "slot_bytes": 1048576, "io_engine": "io_uring", "use_uring_cmd": true, "fdp_enabled": true, "fdp_data_placement_policy": "none", "use_odirect": false}'

# With eviction
--l2-adapter '{"type": "raw_block", "device_path": "/dev/nvme0n1", "slot_bytes": 1048576, "load_checkpoint_on_init": false, "eviction": {"eviction_policy": "LRU", "trigger_watermark": 0.9, "eviction_ratio": 0.1}}'

Failure handling and shutdown:

Recoverable io_uring submission errors (EAGAIN, EINTR, EBUSY) retry after completion progress or 1–100 ms exponential backoff. Permanent errors, or 30 seconds without submission or completion progress, mark the worker unhealthy and stop new native I/O. The core status exposes the failure reason as worker_error.

After a known worker failure, new prefetch tasks report failure through their normal task results and new stores are skipped. The adapter still publishes completion results and notifications so controllers can release locks and buffers; the serving engine can recompute missing data instead of receiving a synchronous storage exception. Valid hits from L1 or other L2 adapters remain usable. The legacy non-MP raw-block backend also skips new stores after worker failure.

Already-accepted I/O retains its buffers until completion or successful cancellation. Close and destructor cleanup release the Python GIL while waiting, but can still block indefinitely if accepted I/O cannot finish or be cancelled. The 30-second submission retry budget is not a shutdown timeout.

Hardware-gated FDP status validation:

FDP live-device validation is opt-in because it requires an FDP-capable NVMe namespace character device. The status probe opens the character device through the Rust raw-block binding and calls fetch_fdp_status() with a read-only file descriptor. It does not issue writes, initialize the MP adapter layout, write KV data, or verify the MP adapter’s cache-salt placement policy.

LMCACHE_TEST_FDP_CHAR_DEVICE=/dev/ng0n1 \
  pytest -q tests/v1/storage_backend/test_raw_block_fdp_status_probe.py

When the variable is not set, the test skips. If the configured device, kernel, or controller does not support the FDP status query, the test skips with the underlying capability error. A passing status probe only confirms that the live device can answer the FDP status query; full adapter initialization and KV write placement on hardware are separate validations.