[None][chore] Upgrade transformers to 4.56.0 #7523

Wanli-Jiang · 2025-09-04T07:23:57Z

Summary by CodeRabbit

Bug Fixes
- Improved robustness when creating tensors on the meta device, correctly handling empty inputs and preventing errors in edge cases.
Chores
- Upgraded Transformers dependency to 4.56.0 for improved compatibility and maintenance.
Tests
- Updated tests to align with the latest Transformers generation utilities, ensuring consistent behavior and reliability.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]

Launch build/test pipelines. All previously running jobs will be killed.

--reuse-test (optional)pipeline-id (OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.

--disable-reuse-test (OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-PyTorch-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--test-backend "pytorch, cpp" (OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".

--detailed-log (OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.

--debug (OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in the stage-list parameter to access the appropriate container environment. Note: Does NOT update GitHub check status.

For guidance on mapping tests to stage names, see docs/source/reference/ci-overview.md
and the scripts/test_to_stage_mapping.py helper.

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

Wanli-Jiang · 2025-09-04T07:24:06Z

/bot run

tensorrt-cicd · 2025-09-04T07:29:12Z

PR_Github #17644 [ run ] triggered by Bot

tensorrt-cicd · 2025-09-04T12:16:48Z

PR_Github #17644 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #13265 completed with status: 'FAILURE'

Wanli-Jiang · 2025-09-04T12:18:05Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-04T12:23:13Z

PR_Github #17683 [ run ] triggered by Bot

tensorrt-cicd · 2025-09-04T14:06:27Z

PR_Github #17683 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #13296 completed with status: 'ABORTED'

Wanli-Jiang · 2025-09-05T02:29:32Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-05T02:35:16Z

PR_Github #17734 [ run ] triggered by Bot

tensorrt-cicd · 2025-09-05T02:35:17Z

PR_Github #17734 [ run ] completed with state DISABLED
L0 testing is limited to prioritized users. User Wanli-Jiang is not in the prioritized list. L0 testing cannot be triggered.

Wanli-Jiang · 2025-09-05T09:42:37Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-05T09:48:13Z

PR_Github #17773 [ run ] triggered by Bot

tensorrt-cicd · 2025-09-05T12:34:00Z

PR_Github #17773 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #13305 completed with status: 'FAILURE'

syuoni · 2025-09-16T01:47:54Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-16T01:53:25Z

PR_Github #18686 [ run ] triggered by Bot

syuoni · 2025-09-16T03:25:58Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-16T04:53:12Z

PR_Github #18686 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #14036 completed with status: 'FAILURE'

syuoni · 2025-09-16T09:48:29Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-16T09:53:40Z

PR_Github #18776 [ run ] triggered by Bot

tensorrt-cicd · 2025-09-16T13:09:32Z

PR_Github #18776 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #14075 completed with status: 'FAILURE'

Wanli-Jiang · 2025-09-17T10:20:52Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-17T10:25:53Z

PR_Github #18982 [ run ] triggered by Bot

tensorrt-cicd · 2025-09-17T14:02:25Z

PR_Github #18982 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #14232 completed with status: 'FAILURE'

Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>

syuoni · 2025-09-22T02:47:54Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-22T02:53:11Z

PR_Github #19475 [ run ] triggered by Bot

tensorrt-cicd · 2025-09-22T05:41:26Z

PR_Github #19475 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #14642 completed with status: 'FAILURE'

Updated the tensor_meta_device patch to handle both 0.0 and empty data cases on the meta device. Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>

syuoni · 2025-09-22T10:02:10Z

/bot run --disable-fail-fast

tensorrt-cicd · 2025-09-22T10:07:51Z

PR_Github #19566 [ run ] triggered by Bot

tensorrt-cicd · 2025-09-22T13:11:02Z

PR_Github #19566 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #14711 completed with status: 'SUCCESS'

coderabbitai · 2025-09-22T13:32:16Z

📝 Walkthrough

Walkthrough

Updates transformers version in requirements. Modifies torch.tensor patch to handle device="meta" for scalar 0.0 and empty inputs, returning zero tensors accordingly; otherwise delegates to original torch.tensor. Test updated to import transformers and reference NEED_SETUP_CACHE_CLASSES_MAPPING via transformers.generation.utils, initializing it to an empty dict.

Changes

Cohort / File(s)	Change Summary
Dependencies `requirements.txt`	Bumps transformers from 4.55.0 to 4.56.0.
Torch meta device tensor patch `tensorrt_llm/_torch/auto_deploy/export/library/tensor_meta_device.py`	Broadens device=="meta" handling: returns torch.zeros(()) for data==0.0; returns torch.zeros((0,), ...) for empty list/tuple; otherwise defers to original torch.tensor. Keeps patch/revert mechanism.
Tests: transformers cache mapping access `tests/unittest/_torch/modeling/test_modeling_nemotron_nas.py`	Imports transformers; ensures transformers.generation.utils.NEED_SETUP_CACHE_CLASSES_MAPPING exists (empty dict); replaces direct references with fully qualified transformers.generation.utils path.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Caller as Caller (e.g., tests)
  participant Torch as torch.tensor (patched)
  participant Orig as original torch.tensor

  Caller->>Torch: tensor(data, device="meta", dtype, ...)
  alt device == "meta" and data == 0.0
    Note over Torch: Return torch.zeros(())
    Torch-->>Caller: zeros scalar on meta
  else device == "meta" and data is [] or ()
    Note over Torch: Return torch.zeros((0,), ...)
    Torch-->>Caller: empty 1D zeros on meta
  else
    Torch->>Orig: tensor(data, device, dtype, ...)
    Orig-->>Torch: tensor(...)
    Torch-->>Caller: delegated result
  end

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description Check	⚠️ Warning	The PR description contains only the repository template and is missing the required filled "Description" and "Test Coverage" sections; it does not explain the rationale for the transformers 4.56.0 upgrade, nor summarize the actual code changes (requirements.txt bump, tensor_meta_device.py modification, and test updates) or which tests cover them. This omission prevents reviewers from assessing scope, risk, and verification steps.	Please complete the PR description by adding a concise "Description" that lists the files changed and the reason for upgrading to transformers 4.56.0, a "Test Coverage" section naming the relevant/updated tests and CI stages, and notes about license/vulnerability scanning, CODEOWNERS or documentation updates, and any behavioral impacts so reviewers and automation can validate the change.
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The PR title "[None][chore] Upgrade transformers to 4.56.0" concisely and accurately identifies the primary change (the transformers dependency bump in requirements.txt); it is focused and readable even though the changeset also includes supporting code and test updates.

✨ Finishing touches

📝 Generate Docstrings

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9dc7316 and 6a115d5.

📒 Files selected for processing (3)

requirements.txt (1 hunks)
tensorrt_llm/_torch/auto_deploy/export/library/tensor_meta_device.py (2 hunks)
tests/unittest/_torch/modeling/test_modeling_nemotron_nas.py (3 hunks)

🧰 Additional context used

📓 Path-based instructions (3)

**/*.{h,hpp,hh,hxx,cpp,cxx,cc,cu,cuh,py}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Use only spaces, no tabs; indent with 4 spaces.

Files:

tests/unittest/_torch/modeling/test_modeling_nemotron_nas.py
tensorrt_llm/_torch/auto_deploy/export/library/tensor_meta_device.py

**/*.py

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.py: Python code must target Python 3.8+.
Indent Python code with 4 spaces; do not use tabs.
Maintain module namespace when importing; prefer 'from package.subpackage import foo' then 'foo.SomeClass()' instead of importing the class directly.
Python filenames should be snake_case (e.g., some_file.py).
Python classes use PascalCase names.
Functions and methods use snake_case names.
Local variables use snake_case; prefix 'k' for variables that start with a number (e.g., k_99th_percentile).
Global variables use upper SNAKE_CASE prefixed with 'G' (e.g., G_MY_GLOBAL).
Constants use upper SNAKE_CASE (e.g., MY_CONSTANT).
Avoid shadowing variables from an outer scope.
Initialize all externally visible members of a class in the constructor.
Prefer docstrings for interfaces that may be used outside a file; comments for in-function or file-local interfaces.
Use Google-style docstrings for classes and functions (Sphinx-parsable).
Document attributes and variables inline so they render under the class/function docstring.
Avoid reflection when a simpler, explicit approach suffices (e.g., avoid dict(**locals()) patterns).
In try/except, catch the most specific exceptions possible.
For duck-typing try/except, keep the try body minimal and use else for the main logic.

Files:

tests/unittest/_torch/modeling/test_modeling_nemotron_nas.py
tensorrt_llm/_torch/auto_deploy/export/library/tensor_meta_device.py

**/*.{cpp,cxx,cc,h,hpp,hh,hxx,cu,cuh,py}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Prepend the NVIDIA Apache-2.0 copyright header with current year to the top of all source files (e.g., .cpp, .h, .cu, .py).

Files:

tests/unittest/_torch/modeling/test_modeling_nemotron_nas.py
tensorrt_llm/_torch/auto_deploy/export/library/tensor_meta_device.py

🔇 Additional comments (6)

tests/unittest/_torch/modeling/test_modeling_nemotron_nas.py (3)

7-7: LGTM!

Adding the transformers import to access the generation utilities namespace.

23-24: LGTM!

Proper initialization of the transformers namespace mapping to an empty dict, which aligns with the updated approach to access cache classes through the transformers module.

398-399: LGTM!

The reference has been correctly updated to use the transformers namespace path instead of accessing the global mapping directly, which is consistent with the module-level initialization.

tensorrt_llm/_torch/auto_deploy/export/library/tensor_meta_device.py (2)

10-15: LGTM!

The updated docstring correctly describes the extended functionality to handle both scalar 0.0 and empty containers on meta device.

25-30: LGTM!

The extended patch logic correctly handles the additional cases:

Scalar 0.0 returns torch.zeros(())

Empty lists/tuples return torch.zeros((0,))

All other cases delegate to the original function

The implementation maintains backward compatibility while addressing the meta device limitations.

requirements.txt (1)

37-37: Transformers 4.56.0 — security check OK; run compatibility tests

PyPI confirms 4.56.0 exists; GitHub advisories show ReDoS/improper-input issues were patched in 4.50.0–4.53.0, so 4.56.0 is not listed as vulnerable. Run the repo's unit/integration tests and review the 4.55.0→4.56.0 changelog/PRs for breaking changes before merging.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

syuoni mentioned this pull request Sep 4, 2025

[None][feat] Add Tencent HunYuanDenseV1 model support #7081

Merged

syuoni force-pushed the user/williamj/upgrade-transformers-4p56p0 branch from a1522e6 to c3d9bc7 Compare September 16, 2025 01:47

Nekofish-L mentioned this pull request Sep 16, 2025

[None][feat] add model seed-oss #7496

Merged

1 task

Wanli-Jiang force-pushed the user/williamj/upgrade-transformers-4p56p0 branch from 485a640 to aa9dccb Compare September 17, 2025 10:20

[None][chore] Upgrade transformers to 4.56.0

802a780

Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>

syuoni force-pushed the user/williamj/upgrade-transformers-4p56p0 branch from aa9dccb to 802a780 Compare September 22, 2025 02:47

Enhance tensor_meta_device patch for empty data handling

6a115d5

Updated the tensor_meta_device patch to handle both 0.0 and empty data cases on the meta device. Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>

syuoni approved these changes Sep 22, 2025

View reviewed changes

syuoni marked this pull request as ready for review September 22, 2025 13:24

syuoni requested review from a team as code owners September 22, 2025 13:24

syuoni requested review from lucaslie, suyoggupta, tomeras91 and yechank-nvidia and removed request for suyoggupta, tomeras91 and yechank-nvidia September 22, 2025 13:24

lucaslie approved these changes Sep 22, 2025

View reviewed changes

syuoni merged commit 2a30f11 into NVIDIA:main Sep 22, 2025
7 checks passed

[None][chore] Upgrade transformers to 4.56.0 #7523

[None][chore] Upgrade transformers to 4.56.0 #7523

Uh oh!

Conversation

Wanli-Jiang commented Sep 4, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

kill

skip

reuse-pipeline

Uh oh!

Wanli-Jiang commented Sep 4, 2025

Uh oh!

tensorrt-cicd commented Sep 4, 2025

Uh oh!

tensorrt-cicd commented Sep 4, 2025

Uh oh!

Wanli-Jiang commented Sep 4, 2025

Uh oh!

tensorrt-cicd commented Sep 4, 2025

Uh oh!

tensorrt-cicd commented Sep 4, 2025

Uh oh!

Wanli-Jiang commented Sep 5, 2025

Uh oh!

tensorrt-cicd commented Sep 5, 2025

Uh oh!

tensorrt-cicd commented Sep 5, 2025

Uh oh!

Wanli-Jiang commented Sep 5, 2025

Uh oh!

tensorrt-cicd commented Sep 5, 2025

Uh oh!

tensorrt-cicd commented Sep 5, 2025

Uh oh!

syuoni commented Sep 16, 2025

Uh oh!

tensorrt-cicd commented Sep 16, 2025

Uh oh!

syuoni commented Sep 16, 2025

Uh oh!

tensorrt-cicd commented Sep 16, 2025

Uh oh!

syuoni commented Sep 16, 2025

Uh oh!

tensorrt-cicd commented Sep 16, 2025

Uh oh!

tensorrt-cicd commented Sep 16, 2025

Uh oh!

Wanli-Jiang commented Sep 17, 2025

Uh oh!

tensorrt-cicd commented Sep 17, 2025

Uh oh!

tensorrt-cicd commented Sep 17, 2025

Uh oh!

syuoni commented Sep 22, 2025

Uh oh!

tensorrt-cicd commented Sep 22, 2025

Uh oh!

tensorrt-cicd commented Sep 22, 2025

Uh oh!

syuoni commented Sep 22, 2025

Uh oh!

tensorrt-cicd commented Sep 22, 2025

Uh oh!

tensorrt-cicd commented Sep 22, 2025

Uh oh!

coderabbitai bot commented Sep 22, 2025

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Pre-merge checks and finishing touches

Uh oh!

Uh oh!

Reviewers

Wanli-Jiang commented Sep 4, 2025 •

edited by coderabbitai bot

Loading