triton_backend/all_models/inflight_batcher_llm/preprocessing/config.pbtxt (1)

264-269: Consider making the HuggingFace model path configurable.

The hardcoded path /model_hf might not be suitable for all deployment scenarios. Consider using a parameterized value like ${hf_model_path} to allow runtime configuration.

 parameters: {
   key: "hf_model_path"
   value: {
-    string_value: "/model_hf"
+    string_value: "${hf_model_path}"
   }
 }

triton_backend/all_models/multimodal/multimodal_encoders/1/model.py (2)

294-316: Batch size limitation needs addressing as mentioned in PR objectives.

The current implementation restricts pixtral to batch size 1 due to variable image counts per request. This limitation is acknowledged in the PR objectives as needing resolution. The attention mask creation logic is correct for handling variable image sizes.

Consider implementing one of the proposed solutions from the PR objectives:

1. Keep the restriction but document it clearly
2. Allow encoder batches to be broken up for arbitrary image counts
3. Rework batching logic for higher batch sizes (ideal but may impact other models)

---

666-666: Fix line length to comply with style guidelines.

Line 666 exceeds the 120 character limit. Please break the comment into multiple lines.

-                # Check if adding this request to the current batch would exceed the maximum batch size
+                # Check if adding this request to the current batch would exceed
+                # the maximum batch size

triton_backend/all_models/inflight_batcher_llm/preprocessing/1/model.py (4)

804-808: Consider validating required attributes for pixtral models

While using suppress(AttributeError) handles missing attributes gracefully, for pixtral models these attributes are required. Consider adding validation.

with suppress(AttributeError):
    # Only for pixtral models
    self.vocab_size = hf_config.text_config.vocab_size
    self.image_size = hf_config.vision_config.image_size
    self.image_token_index = hf_config.image_token_index

# Validate required attributes for pixtral
if vision_model_type == 'pixtral':
    if not all(hasattr(self, attr) for attr in ['vocab_size', 'image_size', 'image_token_index']):
        raise ValueError("[TensorRT-LLM][ERROR] Failed to extract required config attributes for pixtral model")

---

1065-1065: Fix comment formatting

The comment incorrectly shows "else:" which is redundant.

-        else:  # input_images is not None:
+        else:  # input_images is not None

---

1111-1116: Fix line length violation

Line 1115 exceeds the 120 character limit.

         for key, tensor_list in preprocessor_outputs.items():
             val = self.convert_tensor_list_to_tensor(tensor_list)
             if key in self.output_str_dtypes:
-                val = self.convert_tensor_to_str_dtype(
-                    val, self.output_str_dtypes[key])
+                val = self.convert_tensor_to_str_dtype(val, 
+                                                      self.output_str_dtypes[key])
             vision_processed_tensors[key] = val

---

1118-1129: Document the token replacement logic

The logic for replacing image tokens with unique IDs starting from vocab_size is critical for prompt table lookup. Consider adding a comment to explain this behavior.

         # Replace all image tokens with a unique token_id > vocab_size.
         # This shall be used to lookup the prompt table.
+        # Each image token in a batch gets a unique ID starting from vocab_size,
+        # incrementing sequentially. This ensures proper mapping to visual embeddings.
         for batch_id in range(batch_size):

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

Learnt from: yechank-nvidia
PR: NVIDIA/TensorRT-LLM#6254
File: tensorrt_llm/_torch/pyexecutor/model_engine.py:1201-1204
Timestamp: 2025-07-22T09:22:14.726Z
Learning: In TensorRT-LLM's multimodal processing pipeline, shared tensor recovery using `from_shared_tensor()` is only needed during the context phase. Generation requests reuse the already-recovered tensor data and only need to call `strip_for_generation()` to remove unnecessary multimodal data while preserving the recovered tensors. This avoids redundant tensor recovery operations during generation.

Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

triton_backend/all_models/inflight_batcher_llm/preprocessing/1/model.py (3)

139-142: Fix line length violation and update error message.

The error message doesn't include 'pixtral' in the list of supported models, creating an inconsistency with the validation list. Additionally, the line exceeds the 120-character limit.

 assert self.model_type in [
     'llava', 'blip2-opt', 'pixtral', 'vila', 'mllama',
     'llava_onevision', 'qwen2_vl'
-], f"[TensorRT-LLM][ERROR] Currently supported multi-modal models are llava, blip2-opt, vila, mllama, llava_onevision and qwen2_vl. Got {self.model_type}."
+], (f"[TensorRT-LLM][ERROR] Currently supported multi-modal models are "
+    f"llava, blip2-opt, pixtral, vila, mllama, llava_onevision and qwen2_vl. "
+    f"Got {self.model_type}.")

---

154-170: Add robust validation for hf_model_path parameter.

The code assumes hf_model_path exists without validation, which could lead to runtime errors. The parameter access is not safely handled.

 if self.model_type in [
         'mllama', 'llava_onevision', 'qwen2_vl', 'pixtral'
 ]:
-    # Read the correct path from the config
-    hf_model_path = model_config['parameters']['hf_model_path'][
-        'string_value']
+    # Read the correct path from the config
+    hf_model_path = model_config['parameters'].get('hf_model_path')
+    if not hf_model_path:
+        raise ValueError(f"[TensorRT-LLM][ERROR] 'hf_model_path' parameter is required for {self.model_type} model but not found in config")
+    hf_model_path = hf_model_path['string_value']
+    if not hf_model_path or hf_model_path == "${hf_model_path}":
+        raise ValueError(f"[TensorRT-LLM][ERROR] 'hf_model_path' parameter is not properly configured for {self.model_type} model")

---

335-346: Fix incorrect array shape for request_input_len.

The current implementation creates an array with shape (1, batch_size) instead of the expected (batch_size, 1) shape.

-                    request_input_len = np.array([[
-                        len(input_ids_for_batch)
-                        for input_ids_for_batch in pixtral_input_id_tensor
-                    ]])
+                    request_input_len = np.array([
+                        [len(input_ids_for_batch)]
+                        for input_ids_for_batch in pixtral_input_id_tensor
+                    ])

triton_backend/all_models/inflight_batcher_llm/preprocessing/1/model.py (1)

1043-1131: Comprehensive pixtral processing with minor improvements needed.

The implementation is well-structured and handles multiple input types correctly. However, there are a few areas for improvement:

1. Line length violation (line 1116): The line exceeds 120 characters
2. Explicit error handling instead of assert (line 1067 equivalent - already handled with ValueError)
3. Input validation could be enhanced

             if key in self.output_str_dtypes:
-                val = self.convert_tensor_to_str_dtype(
-                    val, self.output_str_dtypes[key])
+                val = self.convert_tensor_to_str_dtype(val, 
+                                                      self.output_str_dtypes[key])

The method demonstrates good practices with:

• Proper error handling for invalid input dimensions
• Consistent batch processing logic
• Appropriate tensor manipulation
• Clear separation of concerns between input types

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.

Learnt from: amitz-nv
PR: NVIDIA/TensorRT-LLM#5616
File: tensorrt_llm/executor/worker.py:375-384
Timestamp: 2025-07-17T09:01:27.402Z
Learning: In tensorrt_llm/executor/worker.py, the LoRA adapter cache optimization logic that checks `is_adapter_in_cpu_cache()` and conditionally passes None for weights/config has a known race condition issue that cannot be solved with simple error handling or verification checks. This is a known limitation that requires a more comprehensive solution.

Learnt from: yechank-nvidia
PR: NVIDIA/TensorRT-LLM#6254
File: tensorrt_llm/_torch/pyexecutor/model_engine.py:1201-1204
Timestamp: 2025-07-22T09:22:14.726Z
Learning: In TensorRT-LLM's multimodal processing pipeline, shared tensor recovery using `from_shared_tensor()` is only needed during the context phase. Generation requests reuse the already-recovered tensor data and only need to call `strip_for_generation()` to remove unnecessary multimodal data while preserving the recovered tensors. This avoids redundant tensor recovery operations during generation.

tests/integration/defs/triton_server/test.sh (1)

47-50: Use more idiomatic default assignment

Tiny shell-style nit: the same effect can be written in one line while preventing an accidental later re-assignment:

: "${PROMPT_EMBEDDING_TABLE_DTYPE:=TYPE_FP16}"
readonly PROMPT_EMBEDDING_TABLE_DTYPE

This is POSIX-compliant, shorter, and marks the value immutable.

tests/integration/defs/triton_server/build_engines.py (1)

1768-1781: Add a concise docstring per guidelines.

Improves readability and Sphinx parsing.

 def prepare_mistral3_pixtral_engine(tensorrt_llm_multimodal_example_root,
                                     tensorrt_llm_llama_example_root,
                                     mistral_small_model_root):
+    """Prepare Mistral Small 3.1 (Pixtral) engines.
+
+    Converts the HF checkpoint to local, builds the LLM engine, and builds the
+    Pixtral visual encoder engine.
+
+    Args:
+        tensorrt_llm_multimodal_example_root: Path to multimodal example root.
+        tensorrt_llm_llama_example_root: Path to llama example root for converter.
+        mistral_small_model_root: HF model directory for Mistral Small 3.1.
+
+    Returns:
+        Tuple[str, str]: (llm engine_dir, multimodal_engine_dir).
+    """

tests/integration/defs/triton_server/test_triton_llm.py (2)

3958-3962: Explicitly import prepare_mistral3_pixtral_engine to avoid F405 and improve clarity

Static analysis flags F405 due to star-imports. Prefer explicit imports for used symbols to reduce ambiguity and lint noise.

Outside this hunk, update imports at the top of the file:

-from .build_engines import *
-from .common import *
+from .build_engines import (
+    prepare_llama_v2_7b_engine,
+    prepare_llama_v2_70b_engine,
+    prepare_mistral_v1_7b_engine,
+    prepare_medusa_vicuna_7b_engine,
+    prepare_eagle_vicuna_7b_engine,
+    prepare_gpt_350m_engine,
+    prepare_gpt_175b_engine,
+    prepare_llama_v3_8b_engine,
+    prepare_llama_v3_70b_engine,
+    prepare_gpt_gather_logits_engine,
+    prepare_t5_small_engine,
+    prepare_whisper_large_engine,
+    prepare_gpt_2b_lora_engine,
+    prepare_tiny_llama_1b_engine,
+    prepare_mllama_engine,
+    prepare_llava_engine,
+    prepare_llava_onevision_engine,
+    prepare_mistral3_pixtral_engine,
+)
+from .common import (
+    prepare_ib_model_repo,
+    prepare_multimodal_model_repo,
+    prepare_custom_config,
+    prepare_disaggregated_serving_model_repo,
+    modify_ib_config_pbtxt,
+    modify_disaggregated_serving_config_pbtxt,
+    check_server_ready,
+    run_cpp_backend_tests,
+    run_cpp_streaming_backend_tests,
+    validate_by_keyword,
+    query_gpu_name,
+    get_gpu_full_name,
+    retrieve_latency_value,
+)

---

3909-3911: Document rationale for TRITON_MAX_BATCH_SIZE=1

Given current batching limitations with variable image counts, locking max batch size to 1 is intentional. Add a short comment here to prevent future “optimizations” that might reintroduce batching regressions.

Example:

NOTE: Pixtral currently supports variable #images per request; set max_batch_size=1 until batching is reworked.

tests/integration/defs/triton_server/common.py (3)

250-251: Document and validate the new dtype parameter (keep backward compatible).

Add a brief docstring entry for PROMPT_EMBEDDING_TABLE_DTYPE (allowed Triton types, default is TYPE_FP16). Optional: validate input early to catch typos.

---

278-278: Consistently propagate prompt_embedding_table_data_type (OK), consider DRY + quoting.

Propagation looks correct. To reduce duplication and avoid shell word-splitting, consider:

• Define once: prompt_dtype_kv = f"prompt_embedding_table_data_type:{PROMPT_EMBEDDING_TABLE_DTYPE}"
• Inject prompt_dtype_kv into each fill_template call
• Quote the KV list (like other calls) to protect spaces in paths.

Also applies to: 310-310, 335-335, 356-356, 371-371

---

355-356: Fix long line (Ruff E501) by splitting KV assignments.

-        f"tensorrt_llm_model_name:{TENSORRT_LLM_TARGET_MODEL_NAME},tensorrt_llm_draft_model_name:{TENSORRT_LLM_DRAFT_MODEL_NAME},logits_datatype:TYPE_FP32," \
+        f"tensorrt_llm_model_name:{TENSORRT_LLM_TARGET_MODEL_NAME}," \
+        f"tensorrt_llm_draft_model_name:{TENSORRT_LLM_DRAFT_MODEL_NAME},logits_datatype:TYPE_FP32," \

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid and distinct model family from Mistral AI, separate from their regular Mistral models. Ministral 8B is specifically designed for edge computing and on-device applications, released in October 2024. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.

Learnt from: galagam
PR: NVIDIA/TensorRT-LLM#6487
File: tests/unittest/_torch/auto_deploy/unit/singlegpu/test_ad_trtllm_bench.py:1-12
Timestamp: 2025-08-06T13:58:07.506Z
Learning: In TensorRT-LLM, test files (files under tests/ directories) do not require NVIDIA copyright headers, unlike production source code files. Test files typically start directly with imports, docstrings, or code.

Learnt from: yechank-nvidia
PR: NVIDIA/TensorRT-LLM#6254
File: tensorrt_llm/_torch/pyexecutor/model_engine.py:1201-1204
Timestamp: 2025-07-22T09:22:14.726Z
Learning: In TensorRT-LLM's multimodal processing pipeline, shared tensor recovery using `from_shared_tensor()` is only needed during the context phase. Generation requests reuse the already-recovered tensor data and only need to call `strip_for_generation()` to remove unnecessary multimodal data while preserving the recovered tensors. This avoids redundant tensor recovery operations during generation.

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid and distinct model family from Mistral AI, separate from their regular Mistral models. Ministral 8B is specifically designed for edge computing and on-device applications, released in October 2024. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

tests/integration/defs/triton_server/test_triton_llm.py (1)

4025-4031: Quote the --text argument to preserve the embedded newline

TEXT can contain a \n; without quoting, it may be split into multiple argv tokens.
Use the same quoting style applied to the mllama test:

-        f"--text={TEXT}",
+        f"--text='{TEXT}'",

tests/integration/defs/triton_server/test_triton_llm.py (1)

3968-3971: Consider asserting basic correctness

The test only checks that the call succeeds; no assertions are made on the response.
Adding a minimal keyword check (similar to other multimodal tests) would catch silent regressions.

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: galagam
PR: NVIDIA/TensorRT-LLM#6487
File: tests/unittest/_torch/auto_deploy/unit/singlegpu/test_ad_trtllm_bench.py:1-12
Timestamp: 2025-08-06T13:58:07.506Z
Learning: In TensorRT-LLM, test files (files under tests/ directories) do not require NVIDIA copyright headers, unlike production source code files. Test files typically start directly with imports, docstrings, or code.

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid and distinct model family from Mistral AI, separate from their regular Mistral models. Ministral 8B is specifically designed for edge computing and on-device applications, released in October 2024. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.

triton_backend/all_models/inflight_batcher_llm/preprocessing/1/model.py (2)

154-160: hf_model_path still not validated — comment already raised on previous commit.
Please add the missing presence / placeholder checks to avoid runtime failures before the path is dereferenced.

---

343-346: request_input_len shape is transposed

The construction gives shape (1, batch_size); downstream code expects (batch_size, 1) (see earlier review).
Swap the loops as previously suggested.

-request_input_len = np.array([[
-    len(ids) for ids in pixtral_input_id_tensor
-]])
+request_input_len = np.array(
+    [[len(ids)] for ids in pixtral_input_id_tensor]
+)

triton_backend/all_models/inflight_batcher_llm/preprocessing/1/model.py (1)

299-304: Prefer explicit is not None checks for input-tensor presence

Using truthiness on pb_utils.Tensor objects (img_urls or image_bytes …) relies on the object’s __bool__ implementation and becomes brittle.
Consider:

has_vision_inputs = any(
    tensor is not None
    for tensor in (img_urls, image_bytes, input_images, video_bytes)
)
if self.is_multimodal and has_vision_inputs:
    …

This avoids accidental ValueError: ambiguous truth value issues when the underlying object turns into a NumPy array.

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid and distinct model family from Mistral AI, separate from their regular Mistral models. Ministral 8B is specifically designed for edge computing and on-device applications, released in October 2024. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.

Learnt from: amitz-nv
PR: NVIDIA/TensorRT-LLM#5616
File: tensorrt_llm/executor/worker.py:375-384
Timestamp: 2025-07-17T09:01:27.402Z
Learning: In tensorrt_llm/executor/worker.py, the LoRA adapter cache optimization logic that checks `is_adapter_in_cpu_cache()` and conditionally passes None for weights/config has a known race condition issue that cannot be solved with simple error handling or verification checks. This is a known limitation that requires a more comprehensive solution.

Learnt from: yechank-nvidia
PR: NVIDIA/TensorRT-LLM#6254
File: tensorrt_llm/_torch/pyexecutor/model_engine.py:1201-1204
Timestamp: 2025-07-22T09:22:14.726Z
Learning: In TensorRT-LLM's multimodal processing pipeline, shared tensor recovery using `from_shared_tensor()` is only needed during the context phase. Generation requests reuse the already-recovered tensor data and only need to call `strip_for_generation()` to remove unnecessary multimodal data while preserving the recovered tensors. This avoids redundant tensor recovery operations during generation.

tests/integration/defs/triton_server/build_model.sh (1)

716-723: Align trtllm-build flags with existing conventions and set plugin policy explicitly.

Other blocks explicitly configure GEMM/attention plugins and use space-separated flags. Recommend matching that for predictability.

Apply:

-    trtllm-build --checkpoint_dir ./c-model/Mistral-Small-3.1-24B-Instruct-2503/bf16 \
-                --max_batch_size=8 \
-                --max_input_len=4096 \
-                --max_seq_len=4096 \
-                --max_multimodal_len=32768 \
-                --use_paged_context_fmha=enable \
-                --output_dir trt_engines/Mistral-Small-3.1-24B-Instruct-2503/bf16/1-gpu
+    trtllm-build --checkpoint_dir ./c-model/Mistral-Small-3.1-24B-Instruct-2503/bf16 \
+                --gemm_plugin auto \
+                --max_batch_size 8 \
+                --max_input_len 4096 \
+                --max_seq_len 4096 \
+                --max_multimodal_len 32768 \
+                --use_paged_context_fmha enable \
+                --output_dir trt_engines/Mistral-Small-3.1-24B-Instruct-2503/bf16/1-gpu

Note: If you rely on default plugin selection, keep it but still switch to the consistent flag formatting.

tests/integration/defs/triton_server/test_triton.py (1)

476-479: Wrap long lines to satisfy E501 and improve readability.

Ruff flagged these lines (E501). Wrap the f-strings using parentheses.

Apply:

-    # Visual encoder path follows the multimodal tmp engine pattern similar to others
-    visual_encoder_path = f"{llm_root}/examples/models/core/multimodal/tmp/trt_engines/{os.path.basename(model_path)}/multimodal_encoder/"
-    run_shell_command(
-        f"cd {tritonserver_test_root} && ./test.sh {test_name} {engine_dir} {model_path} {tokenizer_type} skip skip skip {visual_encoder_path}",
-        llm_root)
+    # Visual encoder path follows the multimodal tmp engine pattern similar to others
+    visual_encoder_path = (
+        f"{llm_root}/examples/models/core/multimodal/tmp/trt_engines/"
+        f"{os.path.basename(model_path)}/multimodal_encoder/"
+    )
+    run_shell_command(
+        (
+            f"cd {tritonserver_test_root} && ./test.sh {test_name} "
+            f"{engine_dir} {model_path} {tokenizer_type} skip skip skip "
+            f"{visual_encoder_path}"
+        ),
+        llm_root,
+    )

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid model name from Mistral AI, distinct from the regular Mistral models. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: venkywonka
PR: NVIDIA/TensorRT-LLM#6650
File: tests/integration/test_lists/qa/llm_perf_cluster.yml:33-37
Timestamp: 2025-08-06T03:47:16.802Z
Learning: Ministral is a valid and distinct model family from Mistral AI, separate from their regular Mistral models. Ministral 8B is specifically designed for edge computing and on-device applications, released in October 2024. In TensorRT-LLM test configurations, "ministral_8b" and "ministral_8b_fp8" are correct model identifiers and should not be changed to "mistral_8b".

Learnt from: galagam
PR: NVIDIA/TensorRT-LLM#6487
File: tests/unittest/_torch/auto_deploy/unit/singlegpu/test_ad_trtllm_bench.py:1-12
Timestamp: 2025-08-06T13:58:07.506Z
Learning: In TensorRT-LLM, test files (files under tests/ directories) do not require NVIDIA copyright headers, unlike production source code files. Test files typically start directly with imports, docstrings, or code.

Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.

Learnt from: djns99
PR: NVIDIA/TensorRT-LLM#6728
File: cpp/tensorrt_llm/plugins/mixtureOfExperts/mixtureOfExpertsPlugin.cpp:966-966
Timestamp: 2025-08-08T04:10:18.987Z
Learning: TensorRT plugins currently don't support padding functionality, and TensorRT is not getting new features (in maintenance mode). This means that duplicating parameters like mExpertHiddenSize in function calls, even with TODO comments, can be acceptable as pragmatic solutions within these constraints.

Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.

Learnt from: yechank-nvidia
PR: NVIDIA/TensorRT-LLM#6254
File: tensorrt_llm/_torch/pyexecutor/model_engine.py:1201-1204
Timestamp: 2025-07-22T09:22:14.726Z
Learning: In TensorRT-LLM's multimodal processing pipeline, shared tensor recovery using `from_shared_tensor()` is only needed during the context phase. Generation requests reuse the already-recovered tensor data and only need to call `strip_for_generation()` to remove unnecessary multimodal data while preserving the recovered tensors. This avoids redundant tensor recovery operations during generation.

.gitattributes (1)

10-11: Handle .jpeg and uppercase extensions; simplify with character classes

Git attribute patterns are case-sensitive. Current rules won’t catch .JPEG/.PNG nor .jpeg. Recommend replacing these two lines with case-insensitive character-class patterns that also cover .jpeg.

Apply this diff:

- tests/integration/test_input_files/*.png filter=lfs diff=lfs merge=lfs -text
- tests/integration/test_input_files/*.jpg filter=lfs diff=lfs merge=lfs -text
+tests/integration/test_input_files/*.[Pp][Nn][Gg] filter=lfs diff=lfs merge=lfs -text
+tests/integration/test_input_files/*.[Jj][Pp][Ee]?[Gg] filter=lfs diff=lfs merge=lfs -text

Notes:

• The second rule matches both .jpg and .jpeg (the ? covers the optional “e”).
• If you expect nested subfolders under test_input_files, consider adding an additional pair using **/ to cover recursion (supported in modern Git), or confirm you’ll keep a flat directory.

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

• MCP integration is disabled by default for public repositories
• Jira integration is disabled by default for public repositories
• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

.gitattributes (2)

10-11: Scoped LFS rules for test images look good

Limiting LFS to tests/integration/test_input_files prevents repo-wide PNG/JPG from being forced into LFS while still protecting large test assets. Nice, targeted change.

---

10-11: CI/LFS Sanity Check: Manual Verification Needed

The sandbox environment doesn’t support git lfs, so we can’t auto-verify here. Please confirm in an LFS-enabled clone (e.g. your CI agent with git lfs install) that all image assets under tests/integration/test_input_files are indeed tracked as LFS pointers before running your integration tests:

• Count and list all image files:

find tests/integration/test_input_files \
  -type f \( -iname "*.png" -o -iname "*.jpg" -o -iname "*.jpeg" \) | sort

— Expect 5 files.

• Verify they’re LFS-tracked:

git lfs ls-files -- tests/integration/test_input_files | awk '{print $3}' | sort

— Should also list the same 5 paths.

• Alternatively, check pointer content in your CI checkout:

grep -Il "version https://git-lfs.github.com/spec/v1" tests/integration/test_input_files/*

— Only true pointers will match.

If any image is missing:

• Re-add it to pick up the new .gitattributes rule:

  git rm --cached <file> && git add <file>

• Or rewrite history to migrate existing blobs:

  git lfs migrate import \
    --include="tests/integration/test_input_files/*.{png,jpg,jpeg}"

Also ensure your CI pipeline runs:

git lfs install --local
git lfs fetch --all
git lfs checkout

so that pointer files are replaced with real binaries before tests execute.