Intern-S2-Mobius - SGLang Documentation

Deployment

Install SGLang

For all methods and hardware platforms, see the official SGLang installation guide. The two paths below match the Python / Docker toggle in the command panel.

Python (pip / uv)
Docker

Command

pip install --upgrade pip
pip install uv
uv pip install --prerelease=allow sglang

Then run the Python output of the command panel below in that environment.

Command

docker pull lmsysorg/sglang:dev

Intern-S2-Mobius was upstreamed in PR #33691 (merged 2026-08-08) — it lives on lmsysorg/sglang:dev (nightly) until the next release cut. For how to launch the image, see Install → Method 3: Using Docker. Substitute the inner sglang serve ... with what the command generator below produces.

Pick your hardware + recipe to generate the launch command. The two serving strategies cover the main operating points:

Low-Latency — MTP (NEXTN) speculative decoding on. Fastest reply for a single user.
High-Throughput — spec off, more tokens per second when many users share the server.

Speed and accuracy numbers apply to the BF16 checkpoint. They were measured with --random-range-ratio 1.0, --flush-cache, on 2×H200 TP=2 against main @ e0828ee3 + PR #33691 head (since merged 2026-08-08 — lmsysorg/sglang:dev is the live equivalent). GSM8K is the full 1319-example test split; GPQA is Diamond 198 problems × 8 repeats (pass@1 avg-of-8). Both ran with no server-side sampling override, so the checkpoint’s generation_config.json defaults applied (temperature 1.0, top_p 0.95, top_k 20). The FP8 H200 recipes have launch, text, vision, and stop-reason verification but no benchmark numbers yet. All B200 recipes are inferred and unverified.

Playground

The Playground is where you experiment with SGLang features beyond the verified matrix. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing.

1. Model Introduction

Intern-S2-Mobius is InternLM’s 35B scientific multimodal foundation model built on the Mobius-v0 architecture (continually pre-trained from Qwen3.5-35B, then SFT and RL post-trained). Instead of binding knowledge storage and reasoning computation layer by layer as conventional Transformers do, Mobius organizes knowledge into a globally shared Memory that multiple Reasoners iteratively query against, yielding two native capabilities:

Backward Residual Connection — shallow and deep reasoning stages can reach knowledge across the model rather than relying only on forward layer-wise flow.
Dynamic Latent Reasoning — recurrent latent iteration refines hidden states before decoding, internalizing part of the deliberation process and shrinking visible chain-of-thought. The reported result is roughly a 4× end-to-end inference speedup over the Qwen3.5-35B baseline while holding comparable scores on general reasoning benchmarks and improving on scientific tasks (Biology-Instructions, Mol-Instructions, MolecularIQ).

On the serving side the model is a hybrid: 30 of 40 transformer layers use GDN (Gated Delta Net) linear attention (kimi-linear-family), with a full-attention layer every 4th layer (full_attention_interval: 4 → 10 full-attention layers), and the bottom of the stack is MoE-routed (2,560 routed experts × 512 intermediate, 8 active per token); a separate MoE-256 / top-8 MTP (NEXTN) layer feeds speculative decoding. It takes images via a vision tower and recognizes the standard <|vision_start|>…<|vision_end|> + <|image_pad|> markers. Context length is 262,144 tokens.

Variant	Architecture	Context	License
Intern-S2-Mobius	Mobius-v0 · GDN ×30 + full ×10 · MoE-2560 / top-8 · MTP · BF16	262,144	Apache-2.0
Intern-S2-Mobius-FP8	Mobius-v0 · GDN ×30 + full ×10 · MoE-2560 / top-8 · MTP · FP8 E4M3	262,144	Apache-2.0

Recommended generation: temperature=0.8, top_p=1.0, top_k=50, min_p=0.0 — the values the model card recommends. Both checkpoints ship generation_config.json with different values (temperature=1.0, top_p=0.95, top_k=20), and SGLang applies that file by default (--sampling-defaults model) — so send the recommended values explicitly per request if you want them. Resources: HuggingFace (BF16) · HuggingFace (FP8) · GitHub (InternLM/Intern-S2-Mobius).

2. Configuration Tips

Trust remote code is required. Intern-S2-Mobius ships a custom configuration_interns2_mobius.py / modeling_interns2_mobius.py on its HF repo; every recipe adds --trust-remote-code.
FP8 uses one H200. The FP8 checkpoint stores E4M3 weights with 128×128 block scales and dynamic activation quantization. The verified H200 recipes use TP=1, deep_gemm, --mem-fraction-static 0.6, disabled prefill CUDA graphs, and decode graphs up to batch size 16. Higher memory fractions or the default graph range exhausted H200 memory during DeepGEMM warmup; the running MTP server used about 99 GB.
Speculative decoding schedule. The checkpoint ships one MTP layer. Enable MTP for the lowest latency (--speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4). We measured accept-length ~3.9/4 draft tokens at 8K-in / 1K-out, roughly tripling single-stream decode speed (median TPOT 9.79 ms → 3.13 ms at conc=1, 14.26 ms → 6.84 ms at conc=16) and roughly doubling mid-concurrency total throughput (9358 → 18029 tokens/s at conc=16, 21395 → 26033 tokens/s at conc=64). The high-throughput recipe stays spec-off because once you can batch wide, its saturation point is higher (34786 tokens/s at conc=256 vs the spec recipe’s peak at conc=64).
Mamba pool sizing. GDN layers live in a separate Mamba state pool; the --mamba-full-memory-ratio (defaults to 0.9) controls the split between the 10 full-attention layers’ KV pool and the 30 GDN layers’ conv+SSM state pool. Default split comfortably handles conc=64 on a 2×H200 node; if you need higher concurrency than --max-running-requests allows for your workload, raise --mamba-full-memory-ratio slightly (each +1% mamba ratio costs full-attn KV).
Vision input. Images are accepted via the standard image_url chat message type. Vision tokens are counted into the prompt (prompt_tokens_details.image_tokens shows the count), and the model honors <|vision_start|> / <|vision_end|> boundaries exactly.
B200 sizing. B200 (192 GB HBM) fits the BF16 weights + KV + Mamba pool on a single GPU with --tp 1. The FP8 B200 cells inherit the verified H200 TP=1 recipe but remain unverified; treat them as a starting point until a Blackwell measurement is available.

3. Advanced Usage

The outputs below are verbatim captures from a live server (sampling per the checkpoint’s generation_config.json, temperature 1.0). Re-running the same request yields a semantically equivalent but textually different trace — treat them as representative, not deterministic.

3.1 Reasoning

InternS2-Mobius is a hybrid-reasoning model — thinking traces start with “Thinking Process:” before the final answer. Enable the qwen3 reasoning parser (toggle Reasoning Parser in the Parsers card of the Playground above) to split thinking into message.reasoning_content and the answer into message.content.

Reasoning Example (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="internlm/Intern-S2-Mobius",
    messages=[{"role": "user", "content": "What is 15% of 240?"}],
)
msg = resp.choices[0].message
print("Reasoning:", getattr(msg, "reasoning_content", None))
print("Answer:", msg.content)

Example Output

Output

Reasoning: Thinking Process:

1.  **Identify the core question:** The user is asking for 15% of 240.
2.  **Determine the calculation method:** To find a percentage of a
    number, multiply the number by the percentage expressed as a decimal
    or fraction.
    *   Percentage: 15%
    *   Decimal: 0.15
    *   Fraction: 15/100
3.  **Perform the calculation:** $240 \times 0.15$
    *   Method 1: $240 \times 0.10 = 24$ (10%) and
        $240 \times 0.05 = 12$ (5%). Then add them: $24 + 12 = 36$.
    *   Method 2: $240 \times 15 = 3600$. Divide by 100 -> 36.
4.  **Verify the result:** The calculation is correct.
5.  **Formulate the answer:** State the final number clearly.cw

Answer:

15% of 240 is **36**.

Here is the math:
$240 \times 0.15 = 36$

3.2 Tool Calling

Enable the qwen3_coder tool-call parser (toggle Tool Call Parser in the Parsers card of the Playground above) to surface structured tool calls via message.tool_calls. Intern-S2-Mobius emits <tool_call>…<function=name>…<parameter=key>…value…</parameter>… — this is exactly the format qwen3_coder parses; without the parser the call is left as raw text in content. On this thinking-mode model the turn also fills reasoning_content, so print both fields. Auto-resolution works out of the box. Intern-S2-Mobius’s chat template contains the <function= / <parameter= markers the auto-detector keys on, so --reasoning-parser auto --tool-call-parser auto resolves to qwen3 / qwen3_coder without any extra config (verified on this build by tailing the server log’s “Auto-detected …” lines + a live tools request). You can pass the literal qwen3_coder slug, but you don’t have to.

Tool Calling Example (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "The city name"},
            },
            "required": ["location"],
        },
    },
}]
resp = client.chat.completions.create(
    model="internlm/Intern-S2-Mobius",
    messages=[{"role": "user", "content": "What is the weather in Beijing?"}],
    tools=tools,
)
msg = resp.choices[0].message
print("Reasoning:", getattr(msg, "reasoning_content", None))
print("Tool calls:", msg.tool_calls)

Example Output

Output

Reasoning: The user is asking for the weather in Beijing. I have access
to a get_weather function that can get the current weather for a
location. The function requires a "location" parameter which should be
the city name. In this case, the user specified "Beijing", so I should
use that as the location parameter.

Tool calls: [
  {
    "id": "call_545b5956b4c3457286261490",
    "index": 0,
    "type": "function",
    "function": {"name": "get_weather", "arguments": "{\"location\": \"Beijing\"}"}
  }
]

finish_reason: tool_calls

3.3 Vision Input

Intern-S2-Mobius takes images via the OpenAI-compatible image_url content type. Vision input works with the same server the Deploy panel produces — no extra model-specific flags needed.

Vision Example (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="internlm/Intern-S2-Mobius",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url",
             "image_url": {"url": "https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg"}},
            {"type": "text", "text": "Describe this image in one sentence."},
        ],
    }],
)
msg = resp.choices[0].message
print("Reasoning:", getattr(msg, "reasoning_content", None))
print("Answer:", msg.content)

Example Output

Output

Reasoning: The user wants a one-sentence description of the image.
Key elements: Tiger, lying down, grass, looking at camera.
Drafting: A tiger is lying in the green grass looking directly at the
camera.
Refining for flow and detail: A majestic tiger with striking orange and
black stripes rests calmly on a bed of lush green grass, staring
intently directly at the viewer.

Answer:

A majestic tiger with striking orange and black stripes rests calmly on a
bed of lush green grass, staring intently directly at the viewer.

​Deployment

​Playground

​1. Model Introduction

​2. Configuration Tips

​3. Advanced Usage

​3.1 Reasoning

​3.2 Tool Calling

​3.3 Vision Input

Deployment

Playground

1. Model Introduction

2. Configuration Tips

3. Advanced Usage

3.1 Reasoning

3.2 Tool Calling

3.3 Vision Input