KEMBAR78
Qwen3-Next Running Command · Issue #10306 · sgl-project/sglang · GitHub
Skip to content

Qwen3-Next Running Command #10306

@yizhang2077

Description

@yizhang2077

Qwen/Qwen3-Next-80B-A3B-Thinking

# TP 2
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Thinking --tp 2 --reasoning-parser deepseek-r1
# TP 4
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Thinking --tp 4 --reasoning-parser deepseek-r1
# TP 8
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Thinking --tp 8 --reasoning-parser deepseek-r1
# TP 4 DP 4
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Thinking --tp 4  --dp 4 --enable-dp-attention
# TP 4 DP 4 EP 4 --reasoning-parser deepseek-r1
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Thinking --tp 4  --dp 4 --enable-dp-attention --enable-ep-moe --reasoning-parser deepseek-r1
# TP 4 + NEXTN 
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Thinking --tp 4 --speculative-num-steps 3  --speculative-eagle-topk 1  --speculative-num-draft-tokens 4 --speculative-algo NEXTN --reasoning-parser deepseek-r1
# TP 4 DP 4 + NEXTN
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Thinking --tp 4 --dp 4 --enable-dp-attention --speculative-num-steps 3  --speculative-eagle-topk 1  --speculative-num-draft-tokens 4 --speculative-algo NEXTN --reasoning-parser deepseek-r1

Qwen/Qwen3-Next-80B-A3B-Instruct

# TP 2
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Instruct --tp 2
# TP 4
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Instruct --tp 4
# TP 8
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Instruct --tp 8
# TP 4 DP 4
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Instruct --tp 4  --dp 4 --enable-dp-attention
# TP 4 DP 4 EP 4
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Instruct --tp 4  --dp 4 --enable-dp-attention --enable-ep-moe
# TP 4 + NEXTN
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Instruct --tp 4 --speculative-num-steps 3  --speculative-eagle-topk 1  --speculative-num-draft-tokens 4 --speculative-algo NEXTN
# TP 4 DP 4 + NEXTN
python3 -m sglang.launch_server --model Qwen/Qwen3-Next-80B-A3B-Instruct --tp 4 --dp 4 --enable-dp-attention --speculative-num-steps 3  --speculative-eagle-topk 1  --speculative-num-draft-tokens 4 --speculative-algo NEXTN

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions