安装#

前提条件： Linux · Python 3.9–3.13 · NVIDIA GPU（算力 7.0+）· CUDA 12.1+ · uv

安装 LMCache#

For guidance on choosing a vLLM release, LMCache release channel, and native runtime, see vLLM and LMCache compatibility before installing.

uv venv --python 3.12
source .venv/bin/activate
uv pip install lmcache

重要

一切准备就绪！您现在可以开始使用 LMCache。有关实践指南和更多使用示例，请参阅 更多示例 部分。

备注

NIXL 支持（例如用于分离式 Prefill 和 P2P KV 共享）是一个可选的附加功能：

uv pip install lmcache[nixl]

CUDA 12.9 的 wheel 发布在专用的 GitHub Release 上，而不是 PyPI。

uv venv --python 3.12
source .venv/bin/activate
VERSION=0.4.3  # replace with target release
uv pip install lmcache==${VERSION} \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/v${VERSION}-cu129 \
    --index-strategy unsafe-best-match

备注

--extra-index-url https://download.pytorch.org/whl/cu129 确保解析 PyTorch 的 CUDA 12.9 构建。没有它，pip 可能会选择不匹配的 CUDA 变体。

The ROCm wheel targets AMD Instinct gfx942 (MI300X / MI325X) and gfx950 (MI350X / MI355X) in one fat binary, and is ABI-matched to the upstream vllm/vllm-openai-rocm image (torch 2.11, ROCm 7.2, Python 3.12). It is published to a dedicated GitHub Release rather than PyPI.

Install directly inside an upstream vLLM ROCm container — torch and the ROCm runtime are already present, so --no-deps binds against them:

docker run -it --device /dev/kfd --device /dev/dri \
    --group-add video --security-opt seccomp=unconfined \
    --entrypoint bash vllm/vllm-openai-rocm:v0.25.0

VERSION=0.5.3  # replace with target release
pip install lmcache==${VERSION}+rocm7.2 --no-deps \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/v${VERSION}-rocm

备注

The wheel excludes torch and the ROCm runtime libraries (they bind to the host image at runtime). Match the wheel's minor torch/ROCm version to your container; for other bases, use the From Source tab.

备注

The ROCm wheel carries a +rocm7.2 PEP 440 local version, so pip show lmcache reports which build is installed and the ROCm build can be requested explicitly. A bare lmcache==${VERSION} also resolves it, since == ignores the local segment.

The Intel XPU wheel is ABI-matched to the upstream vllm/vllm-openai-xpu:v0.26.0 image (torch 2.12.0+xpu and oneAPI/SYCL). It is published to a dedicated GitHub Release rather than PyPI.

Install directly inside the matching upstream vLLM XPU container. Torch and the oneAPI/SYCL runtime are already present, so --no-deps preserves that runtime stack:

docker run -it --device /dev/dri --shm-size=4g \
    --entrypoint bash vllm/vllm-openai-xpu:v0.26.0

VERSION=0.5.3  # replace with target release
pip install lmcache==${VERSION}+xpu --no-deps \
    --no-index \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/v${VERSION}-xpu

备注

The wheel excludes torch and oneAPI/SYCL runtime libraries, which bind to the host image at runtime. Match the wheel's torch and oneAPI versions to the container; for other bases, use the From Source tab.

--no-index restricts pip to the GitHub Release asset, preventing it from selecting a same-version CUDA wheel from PyPI.

备注

The XPU wheel carries a +xpu PEP 440 local version, so pip show lmcache reports which build is installed and the XPU build can be requested explicitly. A bare lmcache==${VERSION} also resolves it, since == ignores the local segment.

This wheel targets AMD Instinct gfx942 (MI300X / MI325X) and gfx950 (MI350X / MI355X). It is built and smoke-tested in the public AMD PyTorch image rocm/pytorch:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0 pinned at digest sha256:4449f856653602317e4101a76fce599c7fcd58ccec2e539951fce5f73083179e. It does not require the ATOM image.

The supported ABI tuple is exact:

Install the matching wheel inside that pinned image:

docker run -it --device /dev/kfd --device /dev/dri \
    --group-add video --security-opt seccomp=unconfined \
    --entrypoint bash \
    rocm/pytorch:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0@sha256:4449f856653602317e4101a76fce599c7fcd58ccec2e539951fce5f73083179e

VERSION=0.5.4  # replace with target release
pip install \
    lmcache==${VERSION}+rocm7.2.4.torch2.10.git3d3aa833.cxx11abi1 \
    --no-deps \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/v${VERSION}-rocm-torch210

The wheel links torch and ROCm libraries from the container at runtime. Other torch 2.10, ROCm 7.2.x, Python, or C++ ABI combinations are not covered by this artifact; build from source for those environments.

每日构建版 wheel 包每天 UTC 时间 07:30 从最新的 dev 分支构建并发布到 GitHub Releases。无需锁定版本 — --pre 会自动选取最新的每日构建版。

uv venv --python 3.12
source .venv/bin/activate
uv pip install lmcache --pre \
    --extra-index-url https://download.pytorch.org/whl/cu130 \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/nightly \
    --index-strategy unsafe-best-match

uv venv --python 3.12
source .venv/bin/activate
uv pip install lmcache --pre \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/nightly-cu129 \
    --index-strategy unsafe-best-match

Run inside an upstream vLLM ROCm container so torch and the ROCm runtime are already present, then install with --no-deps:

docker run -it --device /dev/kfd --device /dev/dri \
    --group-add video --security-opt seccomp=unconfined \
    --entrypoint bash vllm/vllm-openai-rocm:v0.26.0

pip install lmcache --pre --no-deps --no-index \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/nightly-rocm

Nightly ROCm wheels are versioned like the CUDA nightlies with the ROCm local segment appended, e.g. 0.5.4.dev15+rocm7.2.

备注

--no-index is required here. --find-links only adds a source, so without it pip also considers PyPI — and under PEP 440 a pre-release such as 0.5.4rc4 outranks 0.5.4.dev15+rocm7.2, so --pre would install the CUDA wheel instead. The stable tab does not need it because lmcache==${VERSION}+rocm7.2 is an exact pin that only the ROCm release can satisfy. --no-deps is what makes --no-index safe here: torch and the ROCm runtime come from the container, so nothing else needs resolving.

--no-build-isolation 确保内核与您环境中已安装的相同 torch 进行编译，从而防止运行时出现未定义符号错误。

git clone https://github.com/LMCache/LMCache.git
cd LMCache

uv venv --python 3.12
source .venv/bin/activate

uv pip install -r requirements/build.txt
uv pip install vllm  # pulls in required torch version (cu13)
uv pip install -e . --no-build-isolation

git clone https://github.com/LMCache/LMCache.git
cd LMCache

uv venv --python 3.12
source .venv/bin/activate

uv pip install -r requirements/build.txt
# Pin vLLM (and torch) to the cu12.9 wheel index so the local
# CUDA 12 toolchain matches what the extensions are built against.
uv pip install vllm \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --index-strategy unsafe-best-match
# LMCACHE_CUDA_MAJOR=12 makes setup.py pick cupy-cuda12x
# for install_requires instead of the cu13 default.
LMCACHE_CUDA_MAJOR=12 \
    uv pip install -e . --no-build-isolation

git clone https://github.com/LMCache/LMCache.git
cd LMCache

uv venv --python 3.12
source .venv/bin/activate

# Need to install these packages manually to avoid build isolation
uv pip install -r requirements/build.txt

# Install torch from the ROCm wheel index. Use the rocm7.2 index to
# match the upstream vllm/vllm-openai-rocm image (torch 2.11, ROCm 7.2).
uv pip install torch torchvision --index-url https://download.pytorch.org/whl/rocm7.2

# Build LMCache. BUILD_WITH_HIP=1 makes setup.py pick cupy-rocm-7-0 automatically.
# PYTORCH_ROCM_ARCH selects the target GPU(s):
#   gfx942  -> MI300X / MI325X
#   gfx950  -> MI350X / MI355X
# Comma-separate to build a fat binary for multiple archs.
PYTORCH_ROCM_ARCH="gfx942,gfx950" \
TORCH_DONT_CHECK_COMPILER_ABI=1 \
CXX=hipcc \
BUILD_WITH_HIP=1 \
uv pip install -e . --no-build-isolation

git clone https://github.com/LMCache/LMCache.git
cd LMCache

uv venv --python 3.12
source .venv/bin/activate

# Need to install these packages manually to avoid build isolation
uv pip install -r requirements/build.txt

# Build LMCache with SYCL backend.
BUILD_WITH_SYCL=1 uv pip install --no-build-isolation -e .

MACA is CUDA-compatible: a MACA-enabled torch build reports device.type == "cuda", so LMCache's existing CUDA-compatible connector path works without a separate device backend. There is no prebuilt MACA wheel or CI build -- this is a self-compile-only path, built with a MACA-enabled torch already installed via MetaX's own toolchain (not from PyPI).

# Puts the MACA SDK's cu-bridge nvcc-compatible compiler on PATH
# and its runtime libs on LD_LIBRARY_PATH -- required before the
# build step below, so torch.utils.cpp_extension can locate it.
# Adjust MACA_PATH to your actual MACA SDK install root.
export MACA_PATH=/opt/maca
export CUCC_PATH=${MACA_PATH}/tools/cu-bridge
export PATH=${CUCC_PATH}/bin:${CUCC_PATH}/tools:${MACA_PATH}/mxgpu_llvm/bin:${MACA_PATH}/bin:${PATH}
export LD_LIBRARY_PATH=${MACA_PATH}/lib:${MACA_PATH}/mxgpu_llvm/lib:${MACA_PATH}/ompi/lib:${LD_LIBRARY_PATH}

git clone https://github.com/LMCache/LMCache.git
cd LMCache

# Assumes a MACA-enabled torch is already installed/active in this
# environment (e.g. inside a vllm-metax container).

# Need to install these packages manually to avoid build isolation
pip install -r requirements/build.txt

# --no-deps skips install_requires entirely, including
# requirements/common.txt's cufile-python/nvtx (NVIDIA-only,
# not needed on MACA) and the unpinned generic "torch" entry
# (which would otherwise risk resolving over the MACA build).
# Install any other runtime deps you actually need yourself first --
# MP mode needs mcpy (MetaX's cupy equivalent), published on MetaX's
# own pip index rather than PyPI:
#   pip install mcpy -i https://repos.metax-tech.com/r/maca-pypi/simple \
#       --trusted-host repos.metax-tech.com

# SETUPTOOLS_SCM_PRETEND_VERSION makes the wheel filename and
# lmcache.__version__ carry the MACA build identity (mirrors how
# torch's own ROCm wheels are named e.g. torch-2.11.0+rocm7.2-...).
# Derives the base version from this checkout's own git tag (so
# it never needs manual updates across releases) and appends a
# +maca<build> local segment -- set MACA_AI_VERSION to your MACA
# SDK/build number, or leave it at the default below.
BASE_VERSION=$(python -m setuptools_scm)
SETUPTOOLS_SCM_PRETEND_VERSION="${BASE_VERSION}+maca${MACA_AI_VERSION:-0.0.0.0}" \
BUILD_WITH_MACA=1 pip install --no-deps --no-build-isolation -e .

docker pull lmcache/vllm-openai

docker pull lmcache/vllm-openai:latest-cu129

docker pull lmcache/vllm-openai:latest-nightly

docker pull lmcache/vllm-openai:latest-nightly-cu129

docker pull rocm/vllm-dev:nightly_0624_rc2_0624_rc2_20250620

docker pull vllm/vllm-openai-xpu:v0.26.0

请参阅 Docker 部署 以获取运行容器和 ROCm 镜像的信息。

轻量级纯 CLI 软件包，用于查询或对远程 LMCache 服务器进行基准测试。无需 CUDA，支持任意操作系统。

pip install lmcache-cli

备注

lmcache-cli 和 lmcache 提供相同的 lmcache CLI 命令。请勿在同一环境中同时安装两者。

构建 Docker 镜像#

您也可以不拉取预构建镜像，而是直接使用仓库中提供的 Dockerfile 自行构建 LMCache（集成 vLLM）镜像，Dockerfile 位于 docker/。

在 LMCache 仓库的根目录下：

docker build --tag <IMAGE_NAME>:<TAG> --target image-build --file docker/Dockerfile .

将 <IMAGE_NAME> 和 <TAG> 替换为所需的镜像名称和标签。各构建参数的说明请参阅 docker/ 中的示例构建文件。

验证安装#

python -c "import lmcache.cuda_ops"