docs/source/deployment-guide/quick-start-recipe-for-gpt-oss-on-trtllm.md (5)

68-77: YAML: align indentation under moe_config and document new top-level keys

• Indentation under moe_config is inconsistent (4 spaces vs. 2 under cuda_graph_config). While valid YAML, it’s brittle and easy to drift.
• stream_interval and num_postprocess_workers are introduced but not documented in “Extra LLM API Options.” Add brief descriptions for both.
• Sanity check: confirm both keys are supported by TorchLlmArgs for the pytorch backend path.

Apply the indentation tweak:

 enable_attention_dp: false
 cuda_graph_config:
   enable_padding: true
   max_batch_size: 720
 moe_config:
-    backend: TRTLLM
+  backend: TRTLLM
 stream_interval: 10
 num_postprocess_workers: 4

And add brief docs for the new keys in the “Extra LLM API Options” section (see suggested insertion around Line 175):

 #### `moe_config`
 
 * **Description**: Configuration for Mixture-of-Experts (MoE) models.
 
 * **Options**:
 
   * `backend`: The backend to use for MoE operations.
     **Default**: `CUTLASS`
 
+#### `stream_interval`
+
+* **Description**: Frequency (in tokens) to flush streamed partial responses to the client. Lower values improve “liveness” at modest overhead. Tune based on latency vs. throughput needs.
+
+#### `num_postprocess_workers`
+
+* **Description**: Number of worker threads for postprocessing (e.g., detokenization/formatting). Increase to reduce CPU postprocess bottlenecks when GPU throughput is high.

---

85-95: Mirror the YAML cleanup for the CUTLASS profile and verify key support

• Same indentation nit under moe_config.
• Confirm stream_interval and num_postprocess_workers are honored for the CUTLASS MoE backend profile as well.

 enable_attention_dp: true
 cuda_graph_config:
   enable_padding: true
   max_batch_size: 720
 moe_config:
-    backend: CUTLASS
+  backend: CUTLASS
 stream_interval: 10
 num_postprocess_workers: 4

---

101-113: Keep CLI max_batch_size and YAML cuda_graph_config.max_batch_size in lockstep; add a guardrail note

--max_batch_size is now 720; good. Ensure it always matches cuda_graph_config.max_batch_size to avoid falling off the captured graph path. Consider adding a one-line note near the command reminding readers to keep the two in sync.

---

139-141: Nice clarification on batch-size vs. total sequence length

Clear and accurate. Consider adding a tiny example (e.g., “1024 in + 1024 out → total 2048”) for concreteness.

---

272-273: Clarify the concurrency heuristic and fix “through-put” typo

• Wording: “throughput” (no hyphen).
• The rule of thumb “concurrency = max_batch_size * num_gpus” can be misleading with attention DP, TP/EP mixes, or heterogeneous participation in attention. Prefer “number of GPUs participating in attention” and call out that EP shards may not multiply attention capacity.

-To achieve max through-put, with attention DP on, one needs to sweep up to `concurrency = max_batch_size * num_gpus`.
+To achieve max throughput with attention DP enabled, sweep concurrency up to:
+`concurrency = max_batch_size * num_attention_gpus`,
+where `num_attention_gpus` is the number of GPUs participating in attention (often the TP world size). If EP is used, note that expert shards do not necessarily increase attention capacity.

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

docs/source/deployment-guide/quick-start-recipe-for-gpt-oss-on-trtllm.md (2)

146-149: Double‑check behavior when --max_seq_len is omitted

The note that max_seq_len will be inferred from the model config is plausible, but this can affect memory planning and scheduling. Please verify for trtllm-serve with the pytorch backend that omitting this flag behaves as intended for GPT‑OSS‑120B, and call out how users can override it if needed.

---

232-236: Tighten punctuation and verify evals compatibility claim

– Move the comma outside the inline code span for clarity.
– Confirm that gpt_oss.evals can indeed target TRT-LLM Serve’s Chat Completions/Responses API without any shim code in this release.

Suggested diff:

- With the added support of Chat Completions and Responses API in `trtllm-serve,` `gpt_oss.evals` works directly without any modifications.
+ With the added support of Chat Completions and Responses API in `trtllm-serve`, `gpt_oss.evals` works directly without any modifications.