[cuDNN][SDPA] Update cuDNN grad output layout check #141147
Conversation
eqy
added
module: cudnn
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/141147
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (3 Unrelated Failures)As of commit 183acff with merge base 78491d6 ( FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
aten/src/ATen/native/cudnn/MHA.cpp
Outdated
| const auto innermost_dO_stride = dO.strides()[dO.strides().size() - 1]; | ||
| if (innermost_dO_stride != 1) { | ||
| TORCH_WARN_ONCE( | ||
| "cuDNN SDPA backward got grad_output with an innermost stride != 1 " |
There was a problem hiding this comment.
I wonder if we should even warn once here since a lot of people still use this for testing and this is kind of just like log spams them and there's nothing more efficient they can do.
There was a problem hiding this comment.
I mean, we can tell them to update to a version of PyTorch that support cudnn 9.5.1
There was a problem hiding this comment.
For this check it's a known limitation due to the way the kernel is architected and is not planned to be fixed
eqy
added
ciflow/trunk
|
@pytorchmergebot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
pytorchmergebot
force-pushed
the
cudnn951dO
branch
from
b99e438 to
6af9575
Compare
|
@pytorchmergebot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 jobs have failed, first few of them are: periodic / linux-focal-cuda11.8-py3.9-gcc9 / test (multigpu, 1, 1, lf.linux.g5.12xlarge.nvidia.gpu, oncall:distributed) Details for Dev Infra teamRaised by workflow job |
|
@pytorchmergebot rebase |
|
@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here |
|
Successfully rebased |
pytorchmergebot
force-pushed
the
cudnn951dO
branch
from
6af9575 to
183acff
Compare
|
@pytorchmergebot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Thanks to pytorch#137978 from @Skylion007 which bumps to cuDNN 9.5.1 the broken assumption of dO strides == O strides is fixed Note that there is still the restriction that the innermost stride of the grad output is 1 (this is almost always guaranteed because this condition is required of the input tensors). The main exception would be in test code that does e.g., `.sum().backward()` which yields grad output tensors with strides `[0, 0, 0, 0]`. CC @drisspg Pull Request resolved: pytorch#141147 Approved by: https://github.com/drisspg
4 participants
Thanks to #137978 from @Skylion007 which bumps to cuDNN 9.5.1 the broken assumption of dO strides == O strides is fixed
Note that there is still the restriction that the innermost stride of the grad output is 1 (this is almost always guaranteed because this condition is required of the input tensors). The main exception would be in test code that does e.g.,
.sum().backward()which yields grad output tensors with strides[0, 0, 0, 0].CC @drisspg
cc @csarofeen @ptrblck @xwang233 @drisspg @mikaylagawarecki