autotrust/JEV-27B-VL
Dense 27B vision-language decision model on a Qwen3.8-27B backbone, with a rank-32 LoRA (jev-decision) for calibrated choices over text and images.
Qwen3.8-27B VL backbone; System 1 decisions through the jev-decision LoRA
export VLLM_ROCM_USE_AITER=1
vllm serve autotrust/JEV-27B-VL \
--served-model-name autotrust/JEV-27B-VL \
--enable-lora \
--max-lora-rank 32 \
--lora-modules jev-decision=JEV-27B-VL/adapter_vllm \
--logprobs-mode processed_logprobs \
--max-model-len 32768 \
--enable-prefix-caching \
--mamba-cache-mode align \
--limit-mm-per-prompt '{"image": 8}' \
--max-num-seqs 8 \
--trust-request-chat-template \
--tensor-parallel-size 1 \
--attention-backend ROCM_AITER_FAThe interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /autotrust/JEV-27B-VL.md.
Overview
JEV-27B-VL is AutoTrust's
vision-language decision model. The checkpoint is a dense BF16
Qwen3_5ForConditionalGeneration model (about 27.8B parameters, vision
tower included) continued from
Qwen/Qwen3.8-27B. System 2 is
the served checkpoint, named autotrust/JEV-27B-VL. System 1 is the
rank-32 LoRA jev-decision (adapter_vllm/, r 32, lora_alpha 64),
used for calibrated choice probabilities over text or images.
MI355X is verified; see the AMD section below.
Prerequisites
- vLLM: >= 0.32.0. Earlier releases are not supported for this checkpoint.
- Weights:
hf download autotrust/JEV-27B-VL --local-dir JEV-27B-VLso the LoRA path below exists. The sameadapter_vllmdirectory is in the ModelScope checkout; if that folder is not namedJEV-27B-VL, point the module at<checkout>/adapter_vllm(the HF snapshot path ends insnapshots/<rev>/adapter_vllm). - Docker: the image only bind-mounts the Hugging Face cache. Mount the
checkout into the container and keep
jev-decisionpointed at thatadapter_vllmdirectory. - Context:
text_config.max_position_embeddingsis 262144. The validated command uses--max-model-len 32768(theserve.shdefault). The card says a full 256K prompt needs about 17 GB of KV cache on top of the weights, and suggests--max-model-len 131072on an 80 GB GPU.
Decision LoRA and serve flags
These flags are the model card's validated vllm serve line. They are
base arguments, not optional toggles.
jev-decision—--enable-lora,--max-lora-rank 32, and--lora-modules jev-decision=JEV-27B-VL/adapter_vllm. The module name staysjev-decision. The path is the local checkoutJEV-27B-VL/adapter_vllm, or theadapter_vllmdirectory inside an HF or ModelScope checkout.--mamba-cache-mode align— the Qwen3.8 hybrid backbone mixes full attention with Mamba-style layers; the card serves it inalignmode.--limit-mm-per-prompt '{"image": 8}'— at most 8 images per request.--max-num-seqs 8— required. The card says that with more than 8 sequences in one batch, vLLM's LoRA path for this multimodal class returns wrong System 1 probabilities. Extra requests queue.--logprobs-mode processed_logprobs— System 1 reads post-processor logprobs. On/v1/chat/completionsor/v1/completionsforjev-decision, the card also says to sendtop_k: 0andtop_p: 1.0:generation_config.jsonsetstop_k20 andtop_p0.95, and those defaults would truncate the returned distribution.--trust-request-chat-template— image decisions send a raw decision chat template on the request. System 2 keeps the checkpoint's own template.--served-model-name autotrust/JEV-27B-VL— public name for System 2.--enable-prefix-cachingmatches the card's serve line.
POST /v1/decide is not part of stock vllm serve. The checkpoint ships
serve_decide.py, which starts the OpenAI server with these same flags
and adds that route. Stock vllm serve still exposes System 2 as
autotrust/JEV-27B-VL and System 1 as the LoRA model jev-decision.
AMD (1x MI355X, BF16)
One MI355X at TP1. VLLM_ROCM_USE_AITER=1 and --attention-backend ROCM_AITER_FA select AITER FlyDSL for the 48 gated-delta-net layers and
ROCM_AITER_FA for the 16 full-attention layers.
docker run --device=/dev/kfd --device=/dev/dri --group-add video \
--ipc=host --shm-size 16g \
-e VLLM_ROCM_USE_AITER=1 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-v /path/to/JEV-27B-VL:/workspace/JEV-27B-VL \
-p 8000:8000 \
vllm/vllm-openai-rocm:latest \
autotrust/JEV-27B-VL \
--served-model-name autotrust/JEV-27B-VL \
--tensor-parallel-size 1 \
--attention-backend ROCM_AITER_FA \
--enable-lora --max-lora-rank 32 \
--lora-modules jev-decision=/workspace/JEV-27B-VL/adapter_vllm \
--logprobs-mode processed_logprobs \
--max-model-len 32768 \
--enable-prefix-caching \
--mamba-cache-mode align \
--limit-mm-per-prompt '{"image": 8}' \
--max-num-seqs 8 \
--trust-request-chat-template
Verified on 1x MI355X with :latest (vLLM 0.33.0); :nightly gives the
same results:
- System 1 on the card's example (
billing/shipping/tech support) returns 0.9970 / 0.0000 / 0.0030 with 49 prompt tokens; the card reports 0.9969 / 0.0000 / 0.0031 and 49. - Images work on System 2 and System 1.
text_onlyrejects image requests;encoder_parallelserves them. - At
--max-model-len 262144, a needle was retrieved from a 250,391-token prompt. vllm bench serve, 1000 in / 256 out, concurrency 32: about 550 output tok/s and 2,700 total tok/s. Median TTFT is about 11.4 s because--max-num-seqs 8keeps only 8 requests running.
Client usage
System 2 (the base checkpoint), with an image:
client.chat.completions.create(
model="autotrust/JEV-27B-VL",
messages=[{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": image_url}},
{"type": "text", "text": "What does this image show?"},
],
}],
)
System 1 through the checkpoint's /v1/decide route (serve_decide.py):
curl localhost:8000/v1/decide -H 'Content-Type: application/json' -d '{
"kind": "choice",
"state": "Customer: my card was charged twice for one coffee.",
"question": "Which team should handle this?",
"options": ["billing", "shipping", "tech support"]}'
Text-only System 1 calls the LoRA directly (model jev-decision). The
card says those text decisions match
autotrust/JEV-27B.