Qwen3.8-Flash-Next

​

Deployment

pip install -U uv
uv venv --python 3.12 && source .venv/bin/activate

# Qwen3.8-Flash-Next model support:
# https://github.com/sgl-project/sglang/pull/36497
git clone https://github.com/sgl-project/sglang.git
cd sglang
git fetch origin pull/36497/head && git checkout FETCH_HEAD
uv pip install -e python

docker pull lmsysorg/sglang:qwen38flashnext

docker pull lmsysorg/sglang:dev-qwen38-next-local

docker pull lmsysorg/sglang-rocm:qwen38flashnext

​

Playground

​

1. Model Introduction

• Attention — GDN + QSA hybrid. Three of every four layers use Gated DeltaNet, which compresses history into a fixed-size recurrent state; the fourth is global attention running Qwen Sparse Attention (QSA). A lightweight indexer aggregates the sequence into micro-blocks, scores importance at block level, and selects the relevant regions — so the indexing overhead shrinks along with the attention itself. Unlike approaches that reuse indices across layers, QSA compresses independently per layer, which suits an architecture that interleaves GDN and attention. Qwen measures up to 10.2× prefill and 6.6× decode speedups for the QSA attention kernel at 1M tokens.
• Residual — Gated Residual (GR). The single residual stream widens into four parallel branches, with an element-wise dynamic gate controlling how much each layer reads from and writes back to each branch. Qwen reports one branch naturally becoming a long-range bus. The gate also suppresses activation outliers, and the residual state can be held in FP8.
• Embedding — N-gram Embedding. Lookups keyed on the local context (current token plus a few preceding ones) rather than a single token, adding 51B parameters at almost no extra per-token compute. Because lookup addresses are known in advance, the table can live in host memory and be prefetched asynchronously alongside model compute. The final model uses a single such layer near the start of the network.
• Optimization — Muon. Muon for the genuine 2-D linear maps (attention, GDN and MoE expert weights), AdamW for embeddings, the MoE router and GR’s low-rank parameters, with fused QKV / SwiGLU / GDN projections split before orthogonalization. The scaling law was refit for the new architecture, and batch-size warmup was dropped — it cost 18.8% more optimizer steps without improving the result.

​

2. Advanced Usage

​

2.1 Reasoning

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="Qwen/Qwen3.8-Flash-Next",
    messages=[{"role": "user", "content": "What is 15% of 240?"}],
    reasoning_effort="xhigh",  # xhigh (default) | medium | low
)
msg = resp.choices[0].message
print("Reasoning:", getattr(msg, "reasoning_content", None))
print("Answer:", msg.content)

Pending update — a sample transcript will be added here once the weights are public.

​

2.2 Tool Calling

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "The city name"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

resp = client.chat.completions.create(
    model="Qwen/Qwen3.8-Flash-Next",
    messages=[{"role": "user", "content": "What's the weather in Beijing?"}],
    tools=tools,
)
msg = resp.choices[0].message
print("Reasoning:", getattr(msg, "reasoning_content", None))
print("Content:", msg.content)
print("Tool calls:", msg.tool_calls)

Pending update — a sample transcript will be added here once the weights are public.