Qwen3.5 / Qwen3.6 / Qwen3.8 series#

A hybrid architecture interleaving Mamba / Gated-DeltaNet (GDN) linear-attention layers with full-attention layers, shared by the Qwen3.5, Qwen3.6 and Qwen3.8 series (all use the Qwen3_5ForConditionalGeneration architecture). LMCache reinterprets the recurrent state caches as opaque pages at registration time; see Hybrid Attention 模型 for the general handling of Mamba / linear-attention models.

验证过的模型#

引擎文档： vLLM 支持的模型中的 Qwen3.5（架构 Qwen3_5ForConditionalGeneration）。

状态: 已通过 LMCache 验证。

Every model in this family needs the same settings: the align Mamba cache mode, prefix caching, and --separate-object-groups on the LMCache server. The chunk size is negotiated from vLLM's unified block size, which is model-specific — vLLM logs Setting attention block size to N tokens at startup:

模型

统一块大小 N

GPU

Qwen/Qwen3.8-27B

784

1

Qwen/Qwen3.6-27B

784

1

Qwen/Qwen3.5-0.8B

544

1

Qwen3.6-27B (1 GPU, N = 784):

lmcache server --chunk-size 784 --separate-object-groups \
    --l1-size-gb 100 --eviction-policy LRU

vllm serve Qwen/Qwen3.6-27B \
    --enable-prefix-caching \
    --mamba-cache-mode align \
    --kv-transfer-config \
    '{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both"}'

Qwen3.8-27B (1 GPU, N = 784): same block size as Qwen3.6-27B, so the commands above apply unchanged apart from the model id.


Qwen3.5-0.8B (1 GPU, N = 544): identical to the above, with --chunk-size 544. A lower configured chunk-size minimum, including the default 256, negotiates to 544; see Chunk-Size Negotiation.

--mamba-cache-mode align is required (GDN does not support the all mode). --separate-object-groups (server) is required for hybrid models so the Mamba layers get their own cache objects. If vLLM reports "max_num_seqs exceeds available Mamba cache blocks", lower --max-num-seqs to ≤ that count (each decode sequence needs one Mamba block) or raise --gpu-memory-utilization.

有关通用 LMCache + vLLM 连接（端口、远程主机），请参见 快速入门。

状态： 未通过 LMCache 验证。

状态： 支持。TRT-LLM + LMCache 的配置方式，请参阅 快速入门。

CacheBlend 支持#

不支持：混合组的缓存页面是字节不透明的（请参见注意事项）。

压缩支持#

方法

状态

备注

CacheGen

不支持

混合组的缓存页面是字节不透明的。

注意事项#

  • 缓存运行与全新运行的生成结果不保证逐位一致：GDN 后端不支持 vLLM 的批不变模式。预期结果在得分层面等价，而非 token 层面严格一致（CI 验证门为 hma_lm_eval_qwen3_5 gsm8k 存储与检索对比）。

  • Mamba 和 full-attention 组的缓存页面是字节不透明视图，因此不适用内容感知处理，缓存条目不得在具有不同注意力后端或内核块大小的引擎之间共享。

  • vLLM 的 Mamba 前缀缓存在 align 模式下属于实验性功能。

  • Qwen/Qwen3.6-27B and Qwen/Qwen3.8-27B are vision-language models (they load a vision tower); the LMCache validation covers text generation (the hma_lm_eval_qwen3_5 gsm8k store-vs-retrieve gate runs on Qwen3.5 / Qwen3.6). Caching of image/video KV is not validated.