vLLM/Recipes
Search
Qwen

Qwen/Qwen3.5-27B

Qwen3.5 dense multimodal model (27B) with gated delta networks hybrid attention, MTP, and 262K context

Qwen3.5 flagship dense — single-GPU FP8 or 2x GPU or 2-4x Intel Arc Pro B60/B70 BF16

dense27B262,144 ctxvLLM 0.17.0+Multimodal
Default configuration: 8× H200 · Tensor Parallel · BF16
export VLLM_USE_RUST_FRONTEND=1
vllm serve Qwen/Qwen3.5-27B \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --mm-encoder-tp-mode data

The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /Qwen/Qwen3.5-27B.md.

Overview

Qwen3.5-27B is the flagship dense model of the Qwen3.5 family. It uses the same gated delta networks hybrid attention as its MoE siblings, supports vision+text input, and natively serves 262K context. MTP (multi-token prediction) is supported out of the box for low-latency decoding.

Prerequisites

  • vLLM version: >= 0.17.0
  • Hardware (BF16): 1x H200, 2x H100, 4x Intel Arc Pro B60, 2x Intel Arc Pro B70
  • Hardware (FP8): single 40 GB GPU (H100/H200/L40S)
  • Hardware (Int4): single 24 GB GPU

Client Usage

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
# The server's model name is the checkpoint it runs: the selected variant's.
model = client.models.list().data[0].id
resp = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "Write a haiku about gated delta networks."}],
    max_tokens=256,
)
print(resp.choices[0].message.content)

Qwen3.5 thinks before it answers. To turn thinking off for one request, pass extra_body={"chat_template_kwargs": {"enable_thinking": False}}; for every request, start the server with --default-chat-template-kwargs '{"enable_thinking": false}'.

References