-
Notifications
You must be signed in to change notification settings - Fork 25.7k
[CI] Add smoke test for NVSHMEM availability #158938
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158938
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 376c425 with merge base fef236d ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
.ci/pytorch/smoke_test/smoke_test.py
Outdated
| print("torch not compiled with NVSHMEM") | ||
| return | ||
| # 2.9 behavior: NVSHMEM is expected to be compiled in current build | ||
| # raise RuntimeError("torch not compiled with NVSHMEM") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we could write this logic in a future proof way by checking for torch.__version__ to raise a runtime error if the version is >= 2.9. Otherwise, I assume someone would need to remember to update this before 2.9 release
https://github.com/pytorch/pytorch/blob/main/torch/torch_version.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think suggestion from @huydhn is good if version is 2.8.0 or less we expect torch not to be compled with NVSHMEM and raise exception if its available for version 2.9.0 raise exception if CUDA is available but not nvshmem
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: New commits were pushed while merging. Please rerun the merge command. Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
|
@pytorchbot merge -r |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Pull Request resolved: #158938 Approved by: https://github.com/huydhn, https://github.com/atalman
Stack from ghstack (oldest at bottom):