Qwen/Qwen3.5-27B
Qwen3.5 dense multimodal model (27B) with gated delta networks hybrid attention, MTP, and 262K context
Qwen3.5 flagship dense — single-GPU FP8 or 2x GPU or 2-4x Intel Arc Pro B60/B70 BF16
export VLLM_USE_RUST_FRONTEND=1 vllm serve Qwen/Qwen3.5-27B \ --trust-remote-code \ --tensor-parallel-size 1 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3 \ --mm-encoder-tp-mode data
The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /Qwen/Qwen3.5-27B.md.
Overview
Qwen3.5-27B is the flagship dense model of the Qwen3.5 family. It uses the same gated delta networks hybrid attention as its MoE siblings, supports vision+text input, and natively serves 262K context. MTP (multi-token prediction) is supported out of the box for low-latency decoding.
Prerequisites
- vLLM version: >= 0.17.0
- Hardware (BF16): 1x H200, 2x H100, 4x Intel Arc Pro B60, 2x Intel Arc Pro B70
- Hardware (FP8): single 40 GB GPU (H100/H200/L40S)
- Hardware (Int4): single 24 GB GPU
Client Usage
from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
# The server's model name is the checkpoint it runs: the selected variant's.
model = client.models.list().data[0].id
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Write a haiku about gated delta networks."}],
max_tokens=256,
)
print(resp.choices[0].message.content)
Qwen3.5 thinks before it answers. To turn thinking off for one request, pass
extra_body={"chat_template_kwargs": {"enable_thinking": False}}; for every
request, start the server with
--default-chat-template-kwargs '{"enable_thinking": false}'.
References
assert num_cache_lines >= batchNVIDIAAMD
The CUDA graph capture size is larger than the Mamba cache. Reduce
--max-cudagraph-capture-size (default 512); see
vllm#34571.
Prefix caching is experimental
Prefix caching for the Mamba cache runs in the experimental "align" mode.