Yuki
November 16, 2026

How I run Qwen 3.8 27B

AI Linux

I use Arch Linux, so the guide will be written with that in mind.

Following this guide, you will be able to run Qwen 3.8 27B Q4 on a 3090 with 150k context size including vision. As of writing this guide, llama-cpp is version 0.9.0-1. The image model will be offloaded to CPU so the context can be bigger. You can change this if you want of course.

My computer specs are:

CPU: AMD Ryzen 7 5800X3D (16) @ 4.450GHz
GPU: NVIDIA GeForce RTX 3090 24GB
Memory: 64GB

Requirements

How to run

llama-server \
-m /mnt/980Pro/AI/Qwen/unsloth/Qwen3.8-27B-UD-Q4_K_M.gguf \
--mmproj /mnt/980Pro/AI/Qwen/unsloth/mmproj-F16.gguf \
--no-mmproj-offload \
--image-min-tokens 1024 \
--port 1234 \
--host 0.0.0.0 \
--cache-type-k q8_0 \
--cache-type-v q4_0 \
--temp 1 \
--top-p 0.95 \
--top-k 20 \
--presence-penalty 0.0 \
--min-p 0.00 \
--repeat-penalty 1.0 \
--spec-type draft-mtp,ngram-mod \
--spec-draft-n-max 2 \
--spec-draft-threads 8 \
--flash-attn on \
--jinja \
--reasoning-effort xhigh \
--reasoning-format deepseek \
--reasoning-preserve \
--ctx-size 150000 \
--batch-size 2048 \
--ubatch-size 512 \
--threads 16 \
-ngl 999 \
--chat-template-kwargs '{"reasoning_effort":"xhigh"}'

feed previous