KEMBAR78
[FSDP2] privateuse1 support fsdp2. by will-core · Pull Request #139539 · pytorch/pytorch · GitHub
Skip to content

[FSDP2] privateuse1 support fsdp2. #139539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[FSDP2] privateuse1 support fsdp2. #139539

wants to merge 1 commit into from

Conversation

@will-core
Copy link
Contributor

@will-core will-core commented Nov 2, 2024 •
edited by pytorch-bot bot
Loading

We are looking forward to supporting FSDP2 with devices other than CUDA. Please give me some coding suggestions. Thank you very much.

cc @H-Huang @awgu @kwen2501 @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @c-p-i-o

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Nov 2, 2024 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: will-core / name: will-core (5f0584d)

@pytorch-bot pytorch-bot bot added oncall: distributed Add this issue/PR to distributed oncall triage queue release notes: distributed (fsdp) release notes category labels Nov 2, 2024
@pytorch-bot
Copy link

pytorch-bot bot commented Nov 2, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139539

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ No Failures

As of commit 5f0584d with merge base ceab24d (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

kwen2501
kwen2501 reviewed Nov 2, 2024
View reviewed changes
Copy link
Contributor

@kwen2501 kwen2501 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@awgu thinking out loud -- how about we just remove the device type check here and let the program fail at the place where it needs to access stream? I believe / hope there would be equivalent error message from core that would let user know the same thing.

@zou3519 zou3519 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Nov 5, 2024
@kwen2501 kwen2501 requested review from awgu and weifengpy November 8, 2024 01:52
kwen2501
kwen2501 approved these changes Nov 8, 2024
View reviewed changes
Copy link
Contributor

@kwen2501 kwen2501 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI looks good. Approving.

@will-core
Copy link
Contributor Author

Please take a look at this PR and see if there are any other suggestions. Thank you. @awgu @weifengpy

@awgu
Copy link
Collaborator

awgu commented Nov 15, 2024

yea it's probably fine
I do not think this is a major concern

@awgu
Copy link
Collaborator

awgu commented Nov 15, 2024

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Nov 15, 2024
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
@will-core
[FSDP2] privateuse1 support fsdp2. (pytorch#139539)
13182d6 
We are looking forward to supporting FSDP2 with devices other than CUDA. Please give me some coding suggestions. Thank you very much.

Pull Request resolved: pytorch#139539
Approved by: https://github.com/kwen2501
@FFFrog FFFrog mentioned this pull request Dec 9, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kwen2501 kwen2501 kwen2501 approved these changes

@awgu awgu Awaiting requested review from awgu

@weifengpy weifengpy Awaiting requested review from weifengpy

Assignees

No one assigned

Labels

ciflow/trunk Trigger trunk jobs on your pull request Merged oncall: distributed Add this issue/PR to distributed oncall triage queue open source release notes: distributed (fsdp) release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@will-core @awgu @pytorchmergebot @kwen2501 @zou3519 @pytorchbot