Installation#

Prerequisites: Linux · Python 3.9–3.13 · NVIDIA GPU (compute 7.0+) · CUDA 12.1+ · uv

Install LMCache#

uv venv --python 3.12
source .venv/bin/activate
uv pip install lmcache

Important

You’re all set! You can now start using LMCache. For hands-on guides and more usage examples, see the More Examples section.

The CUDA 12.9 wheel is published to a dedicated GitHub Release rather than PyPI.

uv venv --python 3.12
source .venv/bin/activate
VERSION=0.4.3  # replace with target release
uv pip install lmcache==${VERSION} \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/v${VERSION}-cu129 \
    --index-strategy unsafe-best-match

Note

--extra-index-url https://download.pytorch.org/whl/cu129 ensures the CUDA 12.9 build of PyTorch is resolved. Without it, pip may select a mismatched CUDA variant.

Nightly wheels are built from the latest dev branch each day at 07:30 UTC and published to GitHub Releases. No version pinning required — --pre picks the latest nightly automatically.

uv venv --python 3.12
source .venv/bin/activate
uv pip install lmcache --pre \
    --extra-index-url https://download.pytorch.org/whl/cu130 \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/nightly \
    --index-strategy unsafe-best-match

uv venv --python 3.12
source .venv/bin/activate
uv pip install lmcache --pre \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --find-links https://github.com/LMCache/LMCache/releases/expanded_assets/nightly-cu129 \
    --index-strategy unsafe-best-match

--no-build-isolation ensures the kernels are compiled against the same torch already installed in your environment, preventing undefined symbol errors at runtime.

git clone https://github.com/LMCache/LMCache.git
cd LMCache

uv venv --python 3.12
source .venv/bin/activate

uv pip install -r requirements/build.txt
uv pip install vllm  # pulls in required torch version (cu13)
uv pip install -e . --no-build-isolation

git clone https://github.com/LMCache/LMCache.git
cd LMCache

uv venv --python 3.12
source .venv/bin/activate

uv pip install -r requirements/build.txt
# Pin vLLM (and torch) to the cu12.9 wheel index so the local
# CUDA 12 toolchain matches what the extensions are built against.
uv pip install vllm \
    --extra-index-url https://download.pytorch.org/whl/cu129 \
    --index-strategy unsafe-best-match
# LMCACHE_CUDA_MAJOR=12 makes setup.py pick cupy-cuda12x / nixl-cu12
# for install_requires instead of the cu13 defaults.
LMCACHE_CUDA_MAJOR=12 \
    uv pip install -e . --no-build-isolation

git clone https://github.com/LMCache/LMCache.git
cd LMCache

uv venv --python 3.12
source .venv/bin/activate

# Need to install these packages manually to avoid build isolation
uv pip install -r requirements/build.txt

# Install torch from the ROCm wheel index
uv pip install torch torchvision --index-url https://download.pytorch.org/whl/rocm7.0

# Build LMCache. BUILD_WITH_HIP=1 makes setup.py pick cupy-rocm-7-0 automatically.
PYTORCH_ROCM_ARCH="gfx942" \
TORCH_DONT_CHECK_COMPILER_ABI=1 \
CXX=hipcc \
BUILD_WITH_HIP=1 \
uv pip install -e . --no-build-isolation

git clone https://github.com/LMCache/LMCache.git
cd LMCache

uv venv --python 3.12
source .venv/bin/activate

# Need to install these packages manually to avoid build isolation
uv pip install -r requirements/build.txt

# Build LMCache with SYCL backend.
BUILD_WITH_SYCL=1 uv pip install --no-build-isolation -e .

docker pull lmcache/vllm-openai

docker pull lmcache/vllm-openai:latest-cu129

docker pull lmcache/vllm-openai:latest-nightly

docker pull lmcache/vllm-openai:latest-nightly-cu129

docker pull rocm/vllm-dev:nightly_0624_rc2_0624_rc2_20250620

docker pull docker pull intel/vllm:0.17.0-xpu

See Docker deployment for running the container and ROCm images.

Lightweight CLI-only package for querying or benchmarking a remote LMCache server. No CUDA required, works on any OS.

pip install lmcache-cli

Note

lmcache-cli and lmcache ship the same lmcache CLI command. Do not install both in the same environment.

Verify Installation#

python -c "import lmcache.c_ops"

Compatibility Matrix#

✅ compatible · ❌ API incompatible · 🕯 torch mismatch (use --no-build-isolation)

LMCache 0.4.2

LMCache 0.4.1

LMCache 0.3.15

LMCache 0.3.14

LMCache 0.3.13

LMCache 0.3.12

LMCache 0.3.11

LMCache 0.3.10

vLLM 0.18.0.x

✅

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️

vLLM 0.17.1.x

✅

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️

vLLM 0.17.0.x

✅

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️

vLLM 0.16.0.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.15.1.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.15.0.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.14.1.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.14.0.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.13.0.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.12.0.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.11.2.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.11.1.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.11.0.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.10.2.x

🕯️

✅

✅

✅

✅

✅

✅

✅

vLLM 0.10.1.x

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️

🕯️