docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (1)

277-346: Replace unrealistic example response with a concise, valid OpenAI-style JSON

The current example includes non-API tokens (“<|channel|>analysis…”) and an excessively long, markdown-heavy response that may confuse users. Provide a minimal, realistic response that mirrors the server’s schema.

Proposed replacement:

-```bash
-{
-  "id": "chatcmpl-c440e2a3e7e14cd699295afc3739bf42",
-  "object": "chat.completion",
-  "created": 1754358426,
-  "model": "openai/gpt-oss-120b",
-  "choices": [
-    {
-      "index": 0,
-      "message": {
-        "role": "assistant",
-        "content": "<|channel|>analysis<|message|>The user asks: \"What is NVIDIA's advantage for inference?\" ...
-        ... (omitted for brevity) ...
-      },
-      "logprobs": null,
-      "finish_reason": "length",
-      "stop_reason": null,
-      "disaggregated_params": null
-    }
-  ],
-  "usage": {
-    "prompt_tokens": 17,
-    "total_tokens": 1041,
-    "completion_tokens": 1024
-  },
-  "prompt_token_ids": null
-}
-```
+```bash
+{
+  "id": "chatcmpl-1234567890",
+  "object": "chat.completion",
+  "created": 1721000000,
+  "model": "openai/gpt-oss-120b",
+  "choices": [
+    {
+      "index": 0,
+      "message": {
+        "role": "assistant",
+        "content": "NVIDIA’s inference advantage comes from specialized Tensor Cores, the TensorRT compiler/runtime, and an optimized software stack (CUDA, cuDNN, Triton) that together deliver high throughput at low latency."
+      },
+      "finish_reason": "stop"
+    }
+  ],
+  "usage": {
+    "prompt_tokens": 17,
+    "completion_tokens": 42,
+    "total_tokens": 59
+  }
+}
+```

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (8)

54-55: Grammar: “is has been” → “has been”

Minor but visible grammar issue.

-The support for gpt-oss is has been [merged](https://github.com/NVIDIA/TensorRT-LLM/pull/6645) into the **main branch** of TensorRT-LLM.
+Support for gpt-oss has been [merged](https://github.com/NVIDIA/TensorRT-LLM/pull/6645) into the **main branch** of TensorRT-LLM.

---

27-28: Grammar: “follow docker command” → “following docker command”

-Run the follow docker command to start the TensorRT-LLM container in interactive mode:
+Run the following docker command to start the TensorRT-LLM container in interactive mode:

---

23-23: Markdown style: remove extra space after heading marker

-###  NGC Docker Image of dev branch
+### NGC Docker Image of dev branch

---

140-141: Typo: “requeests” → “requests”; tighten the wording

Also reads better as “a sufficient number of requests.”

-`--max_batch_size` controls the maximum batch size that the inference engine could serve, while `--concurrency` is the number of concurrent requests that the benchmarking client is sending. `--num_requests` is set to 10 times of `--concurrency` to run enough number of requeests.
+`--max_batch_size` controls the maximum batch size the inference engine can serve, while `--concurrency` is the number of concurrent requests the benchmarking client sends. `--num_requests` is set to 10× `--concurrency` to run a sufficient number of requests.

---

174-178: Grammar and tone: bullet intros should be parallel and capitalized

“Compare to” → “Compared to”; start each bullet with “Set …” for consistency.

-Compare to the low-latency configuration, we:
-- set `enable_attention_dp` to `true` to use attention DP which is better for high throughput.
-- set `stream_interval` to 10 to stream the results to the client every 10 tokens. At high concurrency the detokenization overhead of the streaming mode cannot be hidden under GPU execution time, `stream_interval` is a workaround to reduce the overhead.
-- set `moe_config.backend` to `CUTLASS` to use the `CUTLASS` MoE kernels which are optimized for high throughput.
+Compared to the low-latency configuration, we:
+- Set `enable_attention_dp` to `true` to use attention DP, which is better for high throughput.
+- Set `stream_interval` to 10 to stream the results to the client every 10 tokens. At high concurrency, the detokenization overhead of streaming cannot be hidden under GPU execution time; `stream_interval` mitigates the overhead.
+- Set `moe_config.backend` to `CUTLASS` to use MoE kernels optimized for high throughput.

---

269-269: Typo in sample request payload: missing apostrophe

JSON example should read “NVIDIA's advantage”.

-            "content": "What is NVIDIAs advantage for inference?"
+            "content": "What is NVIDIA's advantage for inference?"

---

50-51: Minor readability: add commas and possessive

Nit, but this sentence reads more naturally with punctuation and possessive.

-Lastly the container mounts your user `.cache` directory to save the downloaded model checkpoints which are saved to `~/.cache/huggingface/hub/` by default.
+Lastly, the container mounts your user's `.cache` directory to save downloaded model checkpoints, which are stored in `~/.cache/huggingface/hub/` by default.

---

367-374: Minor grammar in Troubleshooting bullets

A few small tweaks improve clarity.

-- Add `print_iter_log: true` to extra LLM API options YAML file to inspect the per-iteration log.
+- Add `print_iter_log: true` to the extra LLM API options YAML file to inspect the per-iteration log.
-- For performance issues, check GPU utilization with `nvidia-smi` while the server is running
+- For performance issues, check GPU utilization with `nvidia-smi` while the server is running.
-- If the container fails to start, verify that the NVIDIA Container Toolkit is properly installed
+- If the container fails to start, verify that the NVIDIA Container Toolkit is properly installed.
-- For connection issues, make sure port 8000 is not being used by another application
+- For connection issues, make sure port 8000 is not being used by another application.

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (2)

217-227: Fix broken multi-line command (line-continuation) and incorrect KV-cache flag in low-latency serve example

• Inline comments after a trailing backslash break shell line-continuation (Lines 217 and 225).
• Flag name should be --kv_cache_free_gpu_mem_fraction to match CLI and earlier examples (Line 224).

Apply:

-  gpt-oss-120b \  # Or ${local_model_path}
+  gpt-oss-120b \
+  # Or, alternatively, pass a local path instead of the model name:
+  # ${local_model_path}
@@
-  --kv_cache_free_gpu_memory_fraction 0.9 \
+  --kv_cache_free_gpu_mem_fraction 0.9 \
@@
-  --max_batch_size ${max_batch_size} \  # E.g., 1
+  --max_batch_size ${max_batch_size} \
+  # E.g., 1

---

234-245: Repeat: fix line-continuation and KV-cache flag in max-throughput serve example

The same two issues appear here (Lines 235, 242, and 243).

-  gpt-oss-120b \  # Or ${local_model_path}
+  gpt-oss-120b \
+  # Or, alternatively, pass a local path instead of the model name:
+  # ${local_model_path}
@@
-  --kv_cache_free_gpu_memory_fraction 0.9 \
+  --kv_cache_free_gpu_mem_fraction 0.9 \
@@
-  --max_batch_size ${max_batch_size} \  # E.g., 640 
+  --max_batch_size ${max_batch_size} \
+  # E.g., 640

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (5)

23-23: Fix markdown heading spacing (lint: MD019)

Remove the extra space after the ATX heading marker.

-###  NGC Docker Image of dev branch
+### NGC Docker Image of dev branch

---

54-54: Grammar fix and clarity improvements

-The support for gpt-oss is has been [merged](https://github.com/NVIDIA/TensorRT-LLM/pull/6645) into the **main branch** of TensorRT-LLM. We are continuing to optimize the performance of gpt-oss, you can build the TensorRT-LLM from source to get the latest features and support. Please refer to the [doc](https://nvidia.github.io/TensorRT-LLM/latest/installation/build-from-source-linux.html) if you want to build from source by yourself.
+The support for gpt-oss has been [merged](https://github.com/NVIDIA/TensorRT-LLM/pull/6645) into the **main branch** of TensorRT-LLM. We are continuing to optimize the performance of gpt-oss; you can build TensorRT-LLM from source to get the latest features and support. Please refer to the [doc](https://nvidia.github.io/TensorRT-LLM/latest/installation/build-from-source-linux.html) if you want to build from source yourself.

---

140-140: Typos and concise phrasing in benchmark description

-`--max_batch_size` controls the maximum batch size that the inference engine could serve, while `--concurrency` is the number of concurrent requests that the benchmarking client is sending. `--num_requests` is set to 10 times of `--concurrency` to run enough number of requeests.
+`--max_batch_size` controls the maximum batch size that the inference engine can serve, while `--concurrency` is the number of concurrent requests that the benchmarking client is sending. `--num_requests` is set to 10× `--concurrency` to run enough requests.

---

174-176: Grammar: “Compared to”

-Compare to the low-latency configuration, we:
+Compared to the low-latency configuration, we:

---

355-360: Clarify Triton install vs preinstalled kernels in NGC image

The text first states the NGC container already includes Triton MoE kernels (no install needed), then immediately instructs building/installing Triton. Clarify this to avoid confusion.

-OpenAI ships a set of Triton kernels optimized for its MoE models. TensorRT-LLM can leverage these kernels for Hopper based GPUs like NVIDIA's H200 for best performance. The NGC TensorRT-LLM container image mentioned above already includes the required kernels so you do not need to build or install them. It is highly recommended to enable them with the steps below:
+OpenAI ships a set of Triton kernels optimized for its MoE models. TensorRT-LLM can leverage these kernels for Hopper-based GPUs like NVIDIA's H200 for best performance. The NGC TensorRT-LLM container image mentioned above already includes the required kernels, so you typically do not need to build or install them. If you are building from source or using a base image without these kernels, follow the steps below:
@@
-### Installing OpenAI Triton
+### Triton installation (only if building from source)
@@
-You will need to build and install a specific version of Triton. Please follow the instructions in this [link](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/models/core/gpt_oss#using-openai-triton-kernels-for-moe).
+If you are building TensorRT-LLM from source or using a base image without the prebuilt kernels, build and install the required Triton version by following this [guide](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/models/core/gpt_oss#using-openai-triton-kernels-for-moe). Otherwise (when using the NGC container above), you can skip this step.

Please confirm the NGC dev image tag indeed includes the Triton MoE kernels for the gpt-oss path so the “skip” guidance is accurate for all users of that image.

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (3)

215-226: Great serve section; a couple of blocking shell issues to fix (line-continuation + wrong flag).

• The inline comments after backslashes on Line 216 and Line 224 break the multi-line command. In bash, the backslash must be the last character on the line.
• The flag on Line 223 should be --kv_cache_free_gpu_mem_fraction (to match bench commands and the CLI).

Apply this diff:

 trtllm-serve \
-  gpt-oss-120b \  # Or ${local_model_path}
+  gpt-oss-120b \
   --host 0.0.0.0 \
   --port 8000 \
   --backend pytorch \
   --tp_size ${num_gpus} \
   --ep_size 1  \
   --extra_llm_api_options low_latency.yaml \
-  --kv_cache_free_gpu_memory_fraction 0.9 \
-  --max_batch_size ${max_batch_size} \  # E.g., 1
+  --kv_cache_free_gpu_mem_fraction 0.9 \
+  --max_batch_size ${max_batch_size} \
   --trust_remote_code

---

233-244: Repeat the same fixes in the max-throughput serve command.

• Remove the inline comment after the backslash (Line 234).
• Use the canonical kv-cache flag name (Line 241).
• Remove the inline comment after the backslash (Line 242).

 trtllm-serve \
-  gpt-oss-120b \  # Or ${local_model_path}
+  gpt-oss-120b \
   --host 0.0.0.0 \
   --port 8000 \
   --backend pytorch \
   --tp_size ${num_gpus} \
   --ep_size ${num_gpus} \
   --extra_llm_api_options max_throughput.yaml \
-  --kv_cache_free_gpu_memory_fraction 0.9 \
-  --max_batch_size ${max_batch_size} \  # E.g., 640 
+  --kv_cache_free_gpu_mem_fraction 0.9 \
+  --max_batch_size ${max_batch_size} \
   --trust_remote_code

---

114-114: Fix the broken intra-doc anchor to match the section title.

The linked section is “(H200/H100 Only) Using OpenAI Triton Kernels for MoE”, so the anchor should be #h200h100-only-using-openai-triton-kernels-for-moe.

-> Note: If you are using NVIDIA H200 GPUs it is highly recommended to set the `moe_config.backend` to TRITON to use the OpenAI Triton MoE kernel. See the section [(H200 Only) Using OpenAI Triton Kernels for MoE](#h200-only-using-openai-triton-kernels-for-moe) for more details.
+> Note: If you are using NVIDIA H200 GPUs it is highly recommended to set the `moe_config.backend` to TRITON to use the OpenAI Triton MoE kernel. See the section [(H200/H100 Only) Using OpenAI Triton Kernels for MoE](#h200h100-only-using-openai-triton-kernels-for-moe) for more details.

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (5)

22-22: Markdown lint: remove extra space in the heading.

-###  NGC Docker Image of dev branch
+### NGC Docker Image of dev branch

---

206-206: Minor readability: add spaces/clarify “>4GPUs”.

-... implementation for >4GPUs is suboptimal ...
+... implementation for > 4 GPUs is suboptimal ...

---

28-38: Optional: run container as the host user to avoid root-owned files in ~/.cache.

Mounting ~/.cache into /root/.cache as root can leave root-owned artifacts on the host. Consider running the container with the host UID/GID.

-docker run --rm --ipc=host -it \
+docker run --rm --ipc=host -it \
+  --user $(id -u):$(id -g) \
   --ulimit stack=67108864 \
   --ulimit memlock=-1 \
   --gpus all \
   -p 8000:8000 \
   -e TRTLLM_ENABLE_PDL=1 \
   -v ~/.cache:/root/.cache:rw \
   nvcr.io/nvidia/tensorrt-llm/release:gpt-oss-dev \
   /bin/bash

---

215-226: Security note: only set --trust_remote_code if needed.

Recommend adding a short caution that --trust_remote_code executes arbitrary model code; users should enable it only when necessary and with trusted sources.

---

353-359: Nice clarification on Triton install; small wording tweak for clarity.

Consider rephrasing slightly for flow.

-The `nvcr.io/nvidia/tensorrt-llm/release:gpt-oss-dev` has Triton installed already (`echo $TRITON_ROOT` could reveal the path). In other situations, you will need to build and install a specific version of Triton. Please follow the instructions in this [link](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/models/core/gpt_oss#using-openai-triton-kernels-for-moe).
+The `nvcr.io/nvidia/tensorrt-llm/release:gpt-oss-dev` image includes Triton already (you can verify with `echo $TRITON_ROOT`). If you’re not using the dev container, you’ll need to build/install a specific Triton version; follow the instructions here: [Using OpenAI Triton Kernels for MoE](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/models/core/gpt_oss#using-openai-triton-kernels-for-moe).

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (3)

216-216: Fix bash line-continuation: inline comments after backslashes break the command

The inline comment after the trailing backslash ends the continuation and will cause the next flags to be parsed as a separate command. Move comments to their own lines.

 trtllm-serve \
-  gpt-oss-120b \  # Or ${local_model_path}
+  gpt-oss-120b \
+  # Or, alternatively, pass a local path instead of the model name:
+  # ${local_model_path}
   --host 0.0.0.0 \
   --port 8000 \
   --backend pytorch \
   --tp_size ${num_gpus} \
   --ep_size 1  \
   --extra_llm_api_options low_latency.yaml \
-  --kv_cache_free_gpu_memory_fraction 0.9 \
-  --max_batch_size ${max_batch_size} \  # E.g., 1
+  --kv_cache_free_gpu_mem_fraction 0.9 \
+  --max_batch_size ${max_batch_size} \
+  # E.g., 1
   --trust_remote_code

Also applies to: 224-224

---

223-223: Use the correct kv-cache flag: --kv_cache_free_gpu_mem_fraction

Serve command uses an invalid flag spelling; bench uses the correct one. Align for consistency and to avoid runtime errors.

-  --kv_cache_free_gpu_memory_fraction 0.9 \
+  --kv_cache_free_gpu_mem_fraction 0.9 \

---

234-234: Fix same line-continuation issue in max-throughput serve command

Inline comments after backslashes break the multi-line command. Move them to separate lines.

 trtllm-serve \
-  gpt-oss-120b \  # Or ${local_model_path}
+  gpt-oss-120b \
+  # Or, alternatively, pass a local path instead of the model name:
+  # ${local_model_path}
   --host 0.0.0.0 \
   --port 8000 \
   --backend pytorch \
   --tp_size ${num_gpus} \
   --ep_size ${num_gpus} \
   --extra_llm_api_options max_throughput.yaml \
-  --kv_cache_free_gpu_memory_fraction 0.9 \
-  --max_batch_size ${max_batch_size} \  # E.g., 640 
+  --kv_cache_free_gpu_mem_fraction 0.9 \
+  --max_batch_size ${max_batch_size} \
+  # E.g., 640
   --trust_remote_code

Also applies to: 242-242

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (4)

210-214: Make env var dependencies explicit before serve commands

The serve snippets rely on num_gpus and max_batch_size defined earlier. Add a short preamble or reminder to set them to avoid copy/paste failures.

 ## Launch the TensorRT-LLM Server

 We can use `trtllm-serve` to serve the model by translating the benchmark commands above. For low-latency configuration, run:
+
+```bash
+# Ensure these are set (reuse values from the benchmarking section or adjust as needed)
+num_gpus=8
+max_batch_size=1
+```

---

22-22: Fix markdownlint MD019: remove double space after heading hashes

Single space after the hashes avoids markdownlint warnings and renders consistently.

-###  NGC Docker Image of dev branch
+### NGC Docker Image of dev branch

---

49-50: Clarify local_model_path usage with a concrete example

A brief example reduces confusion when users choose a local path over downloading weights.

 Additionally, the container mounts your user `.cache` directory to save the downloaded model checkpoints, which are stored in `~/.cache/huggingface/hub/` by default. This prevents having to redownload the weights each time you rerun the container. You can also download the weights to a custom location (we assume `${local_model_path}` is the path to the local model weights).
+
+For example:
+```bash
+local_model_path=/data/models/openai/gpt-oss-120b
+```

---

206-206: Minor wording nit: spacing around numeric comparator

“>4GPUs” reads better as “> 4 GPUs”.

-... implementation for >4GPUs is suboptimal ...
+... implementation for > 4 GPUs is suboptimal ...

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (2)

215-226: Two serve-command bugs: broken line continuation and wrong kv-cache flag; also unify model identifier.

• Trailing backslash followed by an inline comment breaks the shell command (Line 216). The same issue occurs on the --max_batch_size line (Line 224).
• Flag name should be --kv_cache_free_gpu_mem_fraction (not ...memory...) for consistency with bench and CLI.
• Use the same model identifier as benchmarks (openai/gpt-oss-120b) for consistency, or clearly show a local path alternative on its own lines.

 trtllm-serve \
-  gpt-oss-120b \  # Or ${local_model_path}
+  openai/gpt-oss-120b \
+  # Or, alternatively, pass a local path instead of the model name:
+  # ${local_model_path}
   --host 0.0.0.0 \
   --port 8000 \
   --backend pytorch \
   --tp_size ${num_gpus} \
   --ep_size 1  \
   --extra_llm_api_options low_latency.yaml \
-  --kv_cache_free_gpu_memory_fraction 0.9 \
-  --max_batch_size ${max_batch_size} \  # E.g., 1
+  --kv_cache_free_gpu_mem_fraction 0.9 \
+  # E.g., 1
+  --max_batch_size ${max_batch_size} \
   --trust_remote_code

---

233-244: Repeat fixes in max-throughput serve command: line continuation + kv-cache flag + model id.

Same issues as the low‑latency block: trailing backslashes with inline comments (Lines 234, 242) and the kv-cache flag name (Line 241). Also align the model identifier.

 trtllm-serve \
-  gpt-oss-120b \  # Or ${local_model_path}
+  openai/gpt-oss-120b \
+  # Or, alternatively, pass a local path instead of the model name:
+  # ${local_model_path}
   --host 0.0.0.0 \
   --port 8000 \
   --backend pytorch \
   --tp_size ${num_gpus} \
   --ep_size ${num_gpus} \
   --extra_llm_api_options max_throughput.yaml \
-  --kv_cache_free_gpu_memory_fraction 0.9 \
-  --max_batch_size ${max_batch_size} \  # E.g., 640 
+  --kv_cache_free_gpu_mem_fraction 0.9 \
+  # E.g., 640
+  --max_batch_size ${max_batch_size} \
   --trust_remote_code

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (5)

22-22: Fix markdownlint: remove extra space after heading marker.

Two spaces after the hashes violate MD019 and render oddly in some viewers.

-###  NGC Docker Image of dev branch
+### NGC Docker Image of dev branch

---

352-359: Good: Triton section title and install guidance are now clear and linked. Minor grammar polish.

• Title and anchor alignment look correct, and the installation link is helpful.
• Minor wording improvement to tighten phrasing.

-The `nvcr.io/nvidia/tensorrt-llm/release:gpt-oss-dev` has prepared Triton already (`echo $TRITON_ROOT` could reveal the path). In other situations, you will need to build and install a specific version of Triton.
+The `nvcr.io/nvidia/tensorrt-llm/release:gpt-oss-dev` container already includes Triton (`echo $TRITON_ROOT` can reveal the path). In other environments, you will need to build and install a specific version of Triton.

---

12-17: Clarify H100 positioning vs section below.

Prerequisites exclude H100 but later sections include “(H200/H100 Only)” and recommend Triton on Hopper (which includes H100). To avoid confusion, add a brief note here that while detailed H100 performance guidance is forthcoming, the MoE backend selection guidance applies to H100 too.

 - 1x NVIDIA B200/GB200/H200 GPU (more GPUs could be used for lower latency and higher throughput)
+ - 1x NVIDIA B200/GB200/H200 GPU (more GPUs could be used for lower latency and higher throughput)
+   Note: H100 guidance for peak performance is forthcoming; however, the MoE backend selection in this guide also applies to H100.
@@
-We have a forthcoming guide for achieving great performance on H100; however, this guide focuses on the GPUs listed above.
+We have a forthcoming guide for achieving great performance on H100; however, this guide otherwise focuses on the GPUs listed above.

---

206-206: Typo/spacing: “>4GPUs” → “> 4 GPUs”.

Improves readability.

-... implementation for >4GPUs is suboptimal ...
+... implementation for > 4 GPUs is suboptimal ...

---

95-112: Style: reduce repetitive phrasing in “Key takeaways.”

Three bullets start with “is set to …”. Consider tightening to avoid repetition.

-- `enable_attention_dp` is set to `false` to use TP instead of DP for attention.
-- `use_torch_sampler` is set to `true` to use the PyTorch sampler. While the `TRTLLM` sampler is the default, it currently has performance issues, so we use the PyTorch sampler instead.
-- `cuda_graph_config.max_batch_size` is the maximum batch size for CUDA graph.
-- `cuda_graph_config.enable_padding` is set to `true` to enable CUDA graph padding.
-- `moe_config.backend` is set to `TRTLLM` to use the `trtllm-gen` MoE kernels which are optimized for low concurrency.
+- `enable_attention_dp: false` uses TP instead of DP for attention.
+- `use_torch_sampler: true` selects the PyTorch sampler. While `TRTLLM` is the default, it currently has performance issues.
+- `cuda_graph_config.max_batch_size` defines the maximum batch size for CUDA graph; `cuda_graph_config.enable_padding: true` turns on CUDA graph padding.
+- `moe_config.backend: TRTLLM` uses the `trtllm-gen` MoE kernels, optimized for low concurrency.

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (6)

213-213: Bash line-continuation bug: trailing backslash followed by an inline comment breaks the command

Move the comment to its own line so the backslash is the last character.

-  gpt-oss-120b \  # Or ${local_model_path}
+  gpt-oss-120b \
+  # Or, alternatively, pass a local path instead of the model name:
+  # ${local_model_path}

---

221-221: Same line-continuation issue on --max_batch_size line

Inline comment after a trailing backslash breaks the command.

-  --max_batch_size ${max_batch_size} \  # E.g., 1
+  --max_batch_size ${max_batch_size} \
+  # E.g., 1

---

220-220: CLI flag mismatch: use --kv_cache_free_gpu_mem_fraction (not ...memory...)

Keep this consistent with bench commands and the CLI.

-  --kv_cache_free_gpu_memory_fraction 0.9 \
+  --kv_cache_free_gpu_mem_fraction 0.9 \

---

231-231: Repeat line-continuation fix in max-throughput serve command

The inline comment after the backslash breaks the multi-line command.

-  gpt-oss-120b \  # Or ${local_model_path}
+  gpt-oss-120b \
+  # Or, alternatively, pass a local path instead of the model name:
+  # ${local_model_path}

---

239-239: Same line-continuation issue on --max_batch_size (throughput variant)

Move the example to a separate line.

-  --max_batch_size ${max_batch_size} \  # E.g., 640 
+  --max_batch_size ${max_batch_size} \
+  # E.g., 640

---

238-238: Fix kv-cache flag name in max-throughput serve command

Align with bench usage and correct flag spelling.

-  --kv_cache_free_gpu_memory_fraction 0.9 \
+  --kv_cache_free_gpu_mem_fraction 0.9 \

docs/source/blogs/tech_blog/blog9_Deploying_GPT_OSS_on_TRTLLM.md (4)

22-22: Fix markdownlint MD019: remove extra space in heading

Atx headings should have a single space after the hashes.

-###  NGC Docker Image of dev branch
+### NGC Docker Image of dev branch

---

49-50: Define local_model_path before use to avoid confusion

You reference ${local_model_path} in later commands but never show how to set it. Add a short example to make copy/paste flows work.

 Additionally, the container mounts your user `.cache` directory to save the downloaded model checkpoints, which are stored in `~/.cache/huggingface/hub/` by default. This prevents having to redownload the weights each time you rerun the container. You can also download the weights to a custom location (we assume `${local_model_path}` is the path to the local model weights).
+
+For example:
+
+```bash
+# If you've pre-downloaded weights to a local directory:
+export local_model_path=/data/models/gpt-oss-120b
+```

---

107-107: Typo in bullet: stray "s-" prefix

Remove the stray character to keep list formatting consistent.

-s- `cuda_graph_config.max_batch_size` is the maximum batch size for CUDA graph.
+- `cuda_graph_config.max_batch_size` is the maximum batch size for CUDA graph.

---

119-123: Clarify model vs model_path precedence in trtllm-bench

You pass both --model and --model_path. If both are supported, a brief note on precedence helps avoid user confusion; otherwise, show two variants (remote vs local).

Example clarity:

• Remote weights:
  trtllm-bench ... --model openai/gpt-oss-120b ...
• Local weights:
  trtllm-bench ... --model_path ${local_model_path} ...

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.