Qwen2.5-VL (multimodal)#

Validated models#

Engine documentation: Qwen2.5-VL in vLLM supported models (architecture Qwen2_5_VLForConditionalGeneration).

Status: Validated with LMCache (image inputs, MP mode).

Start the LMCache MP server:

lmcache server --l1-size-gb 20 --eviction-policy LRU

Start vLLM with the LMCache MP connector:

vllm serve Qwen/Qwen2.5-VL-3B-Instruct \
    --tensor-parallel-size 1 \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both"}'

No multimodal-specific LMCache configuration is needed. Image identity is folded into the cache keys automatically: vLLM emits identical placeholder token ids for every image, so LMCache overwrites each placeholder span with a value derived from the image’s content hash (mm_hash) before hashing. Two requests with the same text but different images therefore get distinct cache entries, while repeating the same image and prompt hits the cache.

If there are any issues with vLLM setup, please refer to the vLLM Recipes for more details.

Status: Not validated with LMCache.

Status: Not validated with LMCache.

CacheBlend support#

Not validated.

Compression support#

Method

Status

Notes

CacheGen

Not validated

Caveats#

  • KV cache only. LMCache caches the decoder KV for multimodal requests. Vision-encoder outputs are a separate cache: see Encoder caching (in-process mode only; not yet available in MP mode).

  • Cross-user sharing. Cache entries are content-addressed: two users sending the byte-identical image and prompt share an entry. For hard per-user or per-tenant isolation, set cache_salt on the request – salted requests never share entries, even for identical content.

  • Cache hits require identical images. The key derives from the image content hash, so a re-encoded or resized variant of the same picture is a different image and misses the cache.