tensorrt_llm/_torch/pyexecutor/model_engine.py (1)

973-996: Preserve MetaInit path when ModelConfig.clone() is unavailable.

Switching from deepcopy to clone() risks skipping the MetaInit path if clone() isn't implemented (AttributeError) or not universally supported across configs, causing a fallback to full materialization and potential OOM/perf regressions. Prefer clone() but fall back to deepcopy only for the copy step to retain the MetaInit flow.

             try:
-                # config will be modified in-place for some models, like Qwen2
-                config_copy = config.clone()
+                # config will be modified in-place for some models, like Qwen2
+                # Prefer lightweight clone; fall back to deepcopy to preserve MetaInit path if clone is unavailable.
+                try:
+                    if hasattr(config, "clone"):
+                        config_copy = config.clone()
+                    else:
+                        import copy as _copy
+                        config_copy = _copy.deepcopy(config)
+                except Exception:
+                    import copy as _copy
+                    config_copy = _copy.deepcopy(config)
                 with MetaInitMode():
                     model = AutoModelForCausalLM.from_config(config_copy)
 
                 memo = dict()

tensorrt_llm/_torch/pyexecutor/py_executor_creator.py (1)

255-256: Fix logger call to avoid formatting error at runtime.

logger.info("ATTENTION RUNTIME FEATURES: ", attn_runtime_features) will be formatted with % under std logging and can raise. Use %s or f-string.

-    logger.info("ATTENTION RUNTIME FEATURES: ", attn_runtime_features)
+    logger.info("ATTENTION RUNTIME FEATURES: %s", attn_runtime_features)

cpp/tensorrt_llm/runtime/utils/mpiUtils.cpp (2)

303-316: Fix incorrect printf specifiers for size_t in log messages.

size is size_t but logged with %d, which is UB on LP64 and truncates on 64-bit. Use %zu (or cast to unsigned long long with %llu).

Apply:

-    TLLM_LOG_DEBUG("start MPI_Isend with dest %d, tag %d, size %d", dest, static_cast<int>(tag), size);
+    TLLM_LOG_DEBUG("start MPI_Isend with dest %d, tag %d, size %zu", dest, static_cast<int>(tag), size);
@@
-    TLLM_LOG_DEBUG("end MPI_Isend with dest %d, tag %d, size %d", dest, static_cast<int>(tag), size);
+    TLLM_LOG_DEBUG("end MPI_Isend with dest %d, tag %d, size %zu", dest, static_cast<int>(tag), size);
@@
-    TLLM_LOG_DEBUG("start MPI_Send with dest %d, tag %d, size %d", dest, tag, size);
+    TLLM_LOG_DEBUG("start MPI_Send with dest %d, tag %d, size %zu", dest, tag, size);
@@
-    TLLM_LOG_DEBUG("end MPI_Send with dest %d, tag %d, size %d", dest, tag, size);
+    TLLM_LOG_DEBUG("end MPI_Send with dest %d, tag %d, size %zu", dest, tag, size);
@@
-    TLLM_LOG_DEBUG("start MPI_Recv with source %d, tag %d, size %d", source, tag, size);
+    TLLM_LOG_DEBUG("start MPI_Recv with source %d, tag %d, size %zu", source, tag, size);
@@
-    TLLM_LOG_DEBUG("end MPI_Recv with source %d, tag %d, size %d", source, tag, size);
+    TLLM_LOG_DEBUG("end MPI_Recv with source %d, tag %d, size %zu", source, tag, size);

Also applies to: 324-334, 348-360

---

429-466: Gate probe APIs with couldUseMPI() for consistent MPI-disable behavior.

mprobe/improbe/iprobe bypass the new runtime MPI guard. They should early-guard like other ops.

 void MpiComm::mprobeRawTag(int source, int tag, MPI_Message* msg, MPI_Status* status) const
 {
+    couldUseMPI();
 #if ENABLE_MULTI_DEVICE
@@
 bool MpiComm::improbe(int source, MpiTag tag, MPI_Message* msg, MPI_Status* status) const
 {
+    couldUseMPI();
 #if ENABLE_MULTI_DEVICE
@@
 bool MpiComm::iprobe(int source, MpiTag tag, MPI_Status* status) const
 {
+    couldUseMPI();
 #if ENABLE_MULTI_DEVICE

setup.py (2)

175-187: Fix undefined exception variable 'e' when raising SetupError.

e is not defined in this scope; this raises UnboundLocalError instead of the intended SetupError.

-            else:
-                raise SetupError(
-                    f"Failed to get wheel file from {precompiled_path}.") from e
+            else:
+                raise SetupError(
+                    f"Failed to get wheel file from {precompiled_path}.")

---

206-211: Align python_requires with actual syntax usage (PEP 604 unions).

The file uses type hints like str | None, which require Python 3.10+. Current python_requires is ">=3.7", which will break installs.

-    python_requires=">=3.7, <4")
+    python_requires=">=3.10, <4")

Also applies to: 264-264

cpp/tensorrt_llm/batch_manager/CMakeLists.txt (1)

112-116: Ensure Python3::Python target exists before linking
In cpp/tensorrt_llm/batch_manager/CMakeLists.txt before line 112, wrap the Python target usage with a guarded find_package:

+if(NOT TARGET Python3::Python)
+  find_package(Python3 REQUIRED COMPONENTS Interpreter Development)
+endif()
 find_library(TORCH_PYTHON_LIB torch_python REQUIRED
              HINTS ${TORCH_INSTALL_PREFIX}/lib)
 target_link_libraries(${BATCH_MANAGER_STATIC_TARGET}
                       PUBLIC ${TORCH_PYTHON_LIB} Python3::Python pg_utils)

tests/unittest/_torch/ray/test_placement.py (1)

54-67: Test cleanup and potential race condition concerns

The test modifies and deletes CUDA_VISIBLE_DEVICES but doesn't ensure proper cleanup if the test fails. Additionally, there's no Ray cleanup.

Apply this diff to ensure proper cleanup:

 @pytest.mark.gpu2
 def test_cuda_visible_device():
     """Placement via cuda_visible_device"""
+    original_cuda_visible = os.environ.get("CUDA_VISIBLE_DEVICES")
     os.environ["CUDA_VISIBLE_DEVICES"] = "1"
-
-    llm = LLM(model="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
-              orchestrator_type="ray")
-
-    infer_actor_uuids = llm.collective_rpc("report_device_id")
-
-    del os.environ["CUDA_VISIBLE_DEVICES"]
-    assert infer_actor_uuids[0] == get_device_uuid(1)
-    print(f"{infer_actor_uuids=}")
+    try:
+        ray.init()
+        llm = LLM(model="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
+                  orchestrator_type="ray")
+        
+        infer_actor_uuids = llm.collective_rpc("report_device_id")
+        assert infer_actor_uuids[0] == get_device_uuid(1)
+        print(f"{infer_actor_uuids=}")
+    finally:
+        # Restore original environment
+        if original_cuda_visible is not None:
+            os.environ["CUDA_VISIBLE_DEVICES"] = original_cuda_visible
+        else:
+            os.environ.pop("CUDA_VISIBLE_DEVICES", None)
+        ray.shutdown()

tensorrt_llm/_torch/pyexecutor/py_executor.py (2)

862-871: Disjoint tag namespaces for different inter-PP messages.

Tokens and logits both use tag=prev_microbatch_id between the same src/dst. Per prior incident, these must not share a tag space to avoid message collisions.

I used the retrieved learning from PR #7455. Suggest using distinct, documented offsets, e.g., kTOKENS_TAG_BASE=0, kLOGITS_TAG_BASE=100000, then tag=k*_BASE+prev_microbatch_id.

Also applies to: 1849-1863

---

518-546: Initialize or remove self.global_rank
tensorrt_llm/_torch/pyexecutor/py_executor.py logs self.global_rank (around lines 552–556) but PyExecutor.__init__ never sets it, causing an AttributeError at runtime. Either add self.global_rank = dist.rank in the constructor or drop it from the log.