docs/source/features/speculative-decoding.md (3)

188-189: Grammar: add article and tighten phrasing.
“Two-model based speculation implementations do not support overlap scheduler. It will be disabled automatically.” → improve clarity.

-Two-model based speculation implementations do not support overlap scheduler. It will be disabled automatically.
+Two‑model–based speculation implementations do not support the overlap scheduler; it is disabled automatically.

---

43-44: Make overlap-scheduler guidance consistent with the “auto-disabled” statement.
Examples still pass disable_overlap_scheduler=True unconditionally. Either remove it for two‑model setups or gate it on eagle3_one_model.

-llm = LLM("/path/to/target_model", speculative_config=speculative_config, disable_overlap_scheduler=True)
+llm = LLM("/path/to/target_model", speculative_config=speculative_config)

-# Only need to disable overlap scheduler if eagle3_one_model is False.
-llm = LLM("/path/to/target_model", speculative_config=speculative_config, disable_overlap_scheduler=True)
+# Disable overlap scheduler only for the two‑model path.
+llm = LLM("/path/to/target_model",
+          speculative_config=speculative_config,
+          disable_overlap_scheduler=not eagle3_one_model)

-llm = LLM("/path/to/target_model", speculative_config=speculative_config, disable_overlap_scheduler=True)
+llm = LLM("/path/to/target_model", speculative_config=speculative_config)

Also consider updating the YAML example (Lines 134–140) to reflect this nuance.

Also applies to: 64-66, 86-87

---

228-233: Typos and grammar in acceptance description.
Fix missing word and “drat” typo.

-Currently, only greedy sampling is supported for speculative decoding. A draft token is accepted if
-matches the previously decoded token exactly.
+Currently, only greedy sampling is supported for speculative decoding. A draft token is accepted if
+it matches the previously decoded token exactly.
@@
-`[t, d1, d2, d3]`, where `d1`, `d2`, and `d3` are drat tokens.
+`[t, d1, d2, d3]`, where `d1`, `d2`, and `d3` are draft tokens.

docs/source/examples/kvcacheconfig.md (2)

3-3: Inline code formatting: don’t include “argument” inside backticks.

-Set KV cache behavior by providing the optional ```kv_cache_config argument``` when you create the LLM engine.
+Set KV cache behavior by providing the optional `kv_cache_config` argument when you create the LLM engine.

---

3-3: Consistency: “quickstart” vs. “quick start”.
Use one form throughout (“quick start” is used elsewhere).

-Consider the quickstart example found in ```examples/pytorch/quickstart.py```:
+Consider the quick start example found in `examples/pytorch/quickstart.py`:

-This code disables block reuse for the quick start example.
+This code disables block reuse for the quick start example.

(Apply the same spelling uniformly across the docs.)

Also applies to: 47-47

docs/source/examples/kvcacheretentionconfig.md (2)

31-31: Clarify token-range bounds (inclusive/exclusive).
You say “first four tokens” and pass (0, 4). If end is exclusive, that’s correct; please state it explicitly to avoid ambiguity.

-Assume you know that the first four tokens of each prompt represent a system prompt ...
+Assume you know that the first four tokens of each prompt (token indices 0–3; end index is exclusive) represent a system prompt ...

---

49-56: Variable naming style in example.
Prefer snake_case for Python variables.

-    tokenRangeRetentionConfig = KvCacheRetentionConfig.TokenRangeRetentionConfig(0, 4, 100, None)
-    kv_cache_retention_config = KvCacheRetentionConfig(
-        token_range_retention_configs=[tokenRangeRetentionConfig],
+    token_range_retention_config = KvCacheRetentionConfig.TokenRangeRetentionConfig(0, 4, 100, None)
+    kv_cache_retention_config = KvCacheRetentionConfig(
+        token_range_retention_configs=[token_range_retention_config],

docs/source/features/kvcache.md (1)

45-46: Class name casing.
Use KvCacheConfig (matching the API and link target), not “KVCacheConfig”.

-Many of the features in the KV cache system are optional or have user defined properties that alter how they work. Users can control KV cache features through class [KVCacheConfig](https://nvidia.github.io/TensorRT-LLM/llm-api/reference.html#tensorrt_llm.llmapi.KvCacheConfig).
+Many of the features in the KV cache system are optional or have user‑defined properties that alter how they work. Users can control KV cache features through class [KvCacheConfig](https://nvidia.github.io/TensorRT-LLM/llm-api/reference.html#tensorrt_llm.llmapi.KvCacheConfig).

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Learnt from: thorjohnsen
PR: NVIDIA/TensorRT-LLM#6910
File: cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp:0-0
Timestamp: 2025-08-14T21:04:50.248Z
Learning: In KV cache onboarding logic during prefill in cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp, when calculating which blocks fall within the attention window, use getTokensPerBlock() to advance token indices rather than block->getUniqueTokens().size(), because the calculation needs to consider the post-prefill state where blocks will be filled to capacity, not their current token count.

Learnt from: thorjohnsen
PR: NVIDIA/TensorRT-LLM#6910
File: cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp:0-0
Timestamp: 2025-08-14T21:04:50.248Z
Learning: In KV cache onboarding logic during prefill in cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp, when calculating which blocks fall within the attention window, use getTokensPerBlock() to advance token indices rather than block->getUniqueTokens().size(), because the calculation needs to consider the post-prefill state where blocks will be filled to capacity, not their current token count.

Learnt from: eopXD
PR: NVIDIA/TensorRT-LLM#6767
File: cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp:0-0
Timestamp: 2025-08-15T06:46:54.897Z
Learning: In cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp addToken function, newly allocated blocks are unshared by design. The beam search path in addToken (when sequence.getNumTokens() > windowSize) is currently broken/non-functional with SWA, so the block allocation doesn't follow a shared-then-unshared pattern.

docs/source/features/speculative-decoding.md (5)

188-188: Polish grammar: add article and hyphenation.

Suggested: “Two-model-based speculation implementations do not support the overlap scheduler; it will be disabled automatically.”

-Two-model based speculation implementations do not support overlap scheduler. It will be disabled automatically.
+Two-model-based speculation implementations do not support the overlap scheduler; it will be disabled automatically.

---

43-44: Align examples with auto-disable behavior: remove redundant flag.

Since two-model setups auto-disable overlap scheduling, drop the explicit disable_overlap_scheduler=True to avoid confusion.

-llm = LLM("/path/to/target_model", speculative_config=speculative_config, disable_overlap_scheduler=True)
+llm = LLM("/path/to/target_model", speculative_config=speculative_config)

---

65-66: Make EAGLE 3 snippet reflect conditional need.

Either omit the flag entirely (recommended) or show it conditionally only when using two-model. Example below removes it for clarity.

-# Only need to disable overlap scheduler if eagle3_one_model is False.
-llm = LLM("/path/to/target_model", speculative_config=speculative_config, disable_overlap_scheduler=True)
+# Two-model setups auto-disable overlap scheduling.
+llm = LLM("/path/to/target_model", speculative_config=speculative_config)

---

86-87: NGram example: drop disable_overlap_scheduler=True.

Auto-disable applies to two-model algorithms; keep the example minimal.

-llm = LLM("/path/to/target_model", speculative_config=speculative_config, disable_overlap_scheduler=True)
+llm = LLM("/path/to/target_model", speculative_config=speculative_config)

---

134-140: YAML sample: remove disable_overlap_scheduler or note it’s auto/ignored.

To reduce user confusion, either delete the key or add a brief comment that it’s auto-disabled for two-model setups.

-disable_overlap_scheduler: true
 speculative_config:
   decoding_type: Eagle
   max_draft_len: 4
   speculative_model: /path/to/draft/model

docs/source/examples/kvcacheconfig.md (3)

3-3: Inline code formatting: remove triple backticks and “argument” from code span.

Use single backticks and keep prose outside the code span.

-Set KV cache behavior by providing the optional ```kv_cache_config argument``` when you create the LLM engine. Consider the quickstart example found in ```examples/pytorch/quickstart.py```:
+Set KV cache behavior by providing the optional `kv_cache_config` argument when you create the LLM engine. Consider the quick start example found in `examples/pytorch/quickstart.py`:

---

31-31: Consistent terminology: “quick start” (two words).

Matches usage elsewhere in the docs.

-You can reduce this value to 0.7 by adding the following lines to the quickstart example:
+You can reduce this value to 0.7 by adding the following lines to the quick start example:

---

39-39: Inline code formatting for class name.

Use single backticks, not triple.

-You can also set properties after you create ```KvCacheConfig```. For example:
+You can also set properties after you create `KvCacheConfig`. For example:

docs/source/examples/kvcacheretentionconfig.md (3)

3-3: Inline code formatting: use single backticks.

Applies to both occurrences on this line.

-You can change block priority by providing the optional ```kv_cache_retention_config``` argument when you submit a request to the LLM engine. Consider the quick start example found in ```examples/pytorch/quickstart.py```:
+You can change block priority by providing the optional `kv_cache_retention_config` argument when you submit a request to the LLM engine. Consider the quick start example found in `examples/pytorch/quickstart.py`:

---

49-56: PEP 8 naming + clarify token range bounds.

• Use snake_case for variables in Python examples.
• Please clarify whether the end index is inclusive or exclusive to prevent off-by-one errors.

-    tokenRangeRetentionConfig = KvCacheRetentionConfig.TokenRangeRetentionConfig(0, 4, 100, None)
-    kv_cache_retention_config = KvCacheRetentionConfig(
-        token_range_retention_configs=[tokenRangeRetentionConfig],
+    token_range_retention_config = KvCacheRetentionConfig.TokenRangeRetentionConfig(0, 4, 100, None)
+    kv_cache_retention_config = KvCacheRetentionConfig(
+        token_range_retention_configs=[token_range_retention_config],
         decode_retention_priority=35, # Set generated tokens to default priority
         decode_duration_ms=None)

Follow-up: If the constructor expects a half-open range [start, end), consider adding a note like “Indices are [start, end) (end exclusive).”

---

68-68: Inline code formatting: single backticks.

-This example uses a single ```kv_cache_retention_config``` object for all the prompts. You can also provide a list that must have the same length as the list of prompts.
+This example uses a single `kv_cache_retention_config` object for all the prompts. You can also provide a list that must have the same length as the list of prompts.

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

Learnt from: thorjohnsen
PR: NVIDIA/TensorRT-LLM#6910
File: cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp:0-0
Timestamp: 2025-08-14T21:04:50.248Z
Learning: In KV cache onboarding logic during prefill in cpp/tensorrt_llm/batch_manager/kvCacheManager.cpp, when calculating which blocks fall within the attention window, use getTokensPerBlock() to advance token indices rather than block->getUniqueTokens().size(), because the calculation needs to consider the post-prefill state where blocks will be filled to capacity, not their current token count.