KEMBAR78
Serialize Remaining Patterns by eellison · Pull Request #108917 · pytorch/pytorch · GitHub
Skip to content

Serialize Remaining Patterns #108917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 16 commits into from
Closed

Serialize Remaining Patterns #108917

wants to merge 16 commits into from

Conversation

eellison
Copy link
Contributor

@eellison eellison commented Sep 8, 2023 •
edited
Loading

@eellison
Serialize 2->8
f66853b 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Sep 8, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/108917

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit edcb9e5 with merge base 518308a (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Sep 8, 2023
Closed
Closed
Closed
@eellison
Update on "Serialize 2->8"
0dc88d4 
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison eellison mentioned this pull request Sep 11, 2023
Closed
@eellison
Update on "Serialize 2->8"
6ceee11 
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison eellison mentioned this pull request Sep 11, 2023
Closed
@eellison
Update on "Serialize 2->8"
05af182 
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Update on "Serialize 2->8"
ad34883 
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison eellison mentioned this pull request Sep 12, 2023
Closed
@eellison eellison changed the title Serialize 2->8 Serialize Remaining Patterns Sep 12, 2023
@eellison
Update on "Serialize Remaining Patterns"
7beeec5 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
This was referenced Sep 12, 2023
Closed
Closed
@eellison
Update on "Serialize Remaining Patterns"
d39e069 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison eellison requested a review from yanboliang September 13, 2023 16:33
@eellison
Update on "Serialize Remaining Patterns"
03001af 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison eellison added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 13, 2023
@eellison
Update on "Serialize Remaining Patterns"
750657d 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Update on "Serialize Remaining Patterns"
0d94704 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Update on "Serialize Remaining Patterns"
01ff2a9 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Update on "Serialize Remaining Patterns"
c9117b9 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Update on "Serialize Remaining Patterns"
f45c6a1 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Copy link
Contributor Author

@pytorchbot merge

@eellison eellison added the topic: not user facing topic category label Sep 19, 2023
@eellison
Copy link
Contributor Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pytorchmergebot pushed a commit that referenced this pull request Sep 19, 2023
@eellison @pytorchmergebot
Generate patterns in fp16 and fp32 (#109142)
14994cc 
aten.softmax will generate a different decomposition for fp16/bf16 and fp32 because when invoked in lower precision it will upcast the inputs to fp32 and then downcast after. This has been causing us to miss bf16 patterns. For example, Camembert improves 20% with this PR (as do I'm sure many other models).

Pull Request resolved: #109142
Approved by: https://github.com/yanboliang
ghstack dependencies: #108894, #108917
@eellison
Copy link
Contributor Author

@pytorchbot revert -m "land race" -c landrace

@eellison eellison reopened this Sep 19, 2023
@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

@pytorchmergebot
Copy link
Collaborator

@eellison your PR has been successfully reverted.

pytorchmergebot added a commit that referenced this pull request Sep 19, 2023
@pytorchmergebot
Revert "Serialize Remaining Patterns (#108917)"
8b4b181 
This reverts commit 7bf08b7.

Reverted #108917 on behalf of https://github.com/eellison due to land race ([comment](#108917 (comment)))
@eellison
Update on "Serialize Remaining Patterns"
e26eb28 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison eellison mentioned this pull request Sep 19, 2023
Closed
@eellison
Update on "Serialize Remaining Patterns"
bf1f047 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Update on "Serialize Remaining Patterns"
edcb9e5 
Serializes the remaining traced patterns.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Copy link
Contributor Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pytorchmergebot pushed a commit that referenced this pull request Sep 20, 2023
@eellison @pytorchmergebot
Generate patterns in fp16 and fp32 (#109142)
ad53b53 
aten.softmax will generate a different decomposition for fp16/bf16 and fp32 because when invoked in lower precision it will upcast the inputs to fp32 and then downcast after. This has been causing us to miss bf16 patterns. For example, Camembert improves 20% with this PR (as do I'm sure many other models).

Pull Request resolved: #109142
Approved by: https://github.com/yanboliang
ghstack dependencies: #109663, #108894, #108917
pytorchmergebot pushed a commit that referenced this pull request Sep 20, 2023
@eellison @pytorchmergebot
Add 3d Attn Pattern to match HF Whisper (#109156)
d24ba7a 
Adds a 3d pattern that improves perf of HF Whisper from 1.3 -> 4.1. We could be matching more generally on 3d, but i'll leave that for another pr.

Thanks to @drisspg for helping me write the pattern.

Pull Request resolved: #109156
Approved by: https://github.com/yanboliang
ghstack dependencies: #109663, #108894, #108917, #109142
pytorchmergebot pushed a commit that referenced this pull request Sep 20, 2023
@eellison @pytorchmergebot
Use pretty print for checking no duplicated pattern (#109066)
db6e9f6 
The pretty print is faster and more concise because it memoizes objects.

Pull Request resolved: #109066
Approved by: https://github.com/yanboliang
ghstack dependencies: #109663, #108894, #108917, #109142, #109156
@facebook-github-bot facebook-github-bot deleted the gh/eellison/532/head branch September 23, 2023 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yanboliang yanboliang Awaiting requested review from yanboliang

1 more reviewer

@davidberard98 davidberard98 davidberard98 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged module: inductor Reverted topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@eellison @pytorchmergebot @davidberard98