[FSDP2] Added check for contiguous parameters #137000
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/137000
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit b3b3882 with merge base 7ff8e66 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
pytorch-bot
bot
added
oncall: distributed
awgu
added
release notes: distributed (fsdp2)
|
I am going to assume that we are not missing signal from this test Lint / Test run_test.py is usable without boto3/rockset: |
|
@pytorchbot merge -f "unrelated test failure" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Since our implementation currently assumes contiguous strides, let us add an explicit check and raise an error at construction time if the parameter is not contiguous. We can try to support this in the future. Mainly, I want to first learn more about how DTensor support for non-contiguous memory formats works. Pull Request resolved: pytorch#137000 Approved by: https://github.com/weifengpy
3 participants
Stack from ghstack (oldest at bottom):
Since our implementation currently assumes contiguous strides, let us add an explicit check and raise an error at construction time if the parameter is not contiguous.
We can try to support this in the future. Mainly, I want to first learn more about how DTensor support for non-contiguous memory formats works.
cc @XilunWu @H-Huang @kwen2501 @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @c-p-i-o