KEMBAR78
Improve mem efficiency of constant folding by eellison · Pull Request #108421 · pytorch/pytorch · GitHub
Skip to content

Improve mem efficiency of constant folding #108421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 8 commits into from
Closed

Improve mem efficiency of constant folding #108421

wants to merge 8 commits into from

Conversation

@eellison
Copy link
Contributor

@eellison eellison commented Sep 1, 2023 •
edited
Loading

Stack from ghstack (oldest at bottom):

Couple changes to make it more efficient.

  • Because we replacing nodes that only have a single value, only store a single value instead of the whole tensor for node replacement
  • torch.fx.Interpreter will preserve a Tensor in the env as long as it has more uses. That also applies even to output uses, but we are not going to constant fold that use. Instead of using last use for garbage collection, use last non output use.

If reviewers would prefer I ghstack this bc of code movement let me know.

Fix for #108388

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov

Differential Revision: D49020616

@pytorch-bot
Copy link

pytorch-bot bot commented Sep 1, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/108421

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Merge Blocking SEVs

There is 1 active merge blocking SEVs. Please view them below:

If you must merge, use @pytorchbot merge -f.

✅ No Failures

As of commit fb0988f with merge base bae14b3 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@eellison
Update on "Improve mem efficiency of constant folding"
055f431 
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Update on "Improve mem efficiency of constant folding"
1c202de 
cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
eellison
eellison commented Sep 1, 2023
View reviewed changes
@eellison
Update on "Improve mem efficiency of constant folding"
5ed13a9 
Couple changes to make it more efficient.

- Because we replacing nodes that only have a single value, only store a single value instead of the whole tensor for node replacement
- torch.fx.Interpreter will preserve a Tensor in the env as long as it has more uses. That also applies even to output uses, but we are not going to constant fold that use. Instead of using last use for garbage collection, use last non output use. 

If reviewers would prefer I ghstack this bc of code movement let me know.

Fix for #108388


cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
eellison added a commit that referenced this pull request Sep 1, 2023
@eellison
Improve mem efficiency of constant folding
b45cb6a 
ghstack-source-id: 2ecec4c
Pull Request resolved: #108421
@jansel
Copy link
Contributor

jansel commented Sep 1, 2023

Lints/test errors

It would be easier to review if the code movement were in another PR.

@eellison
Update on "Improve mem efficiency of constant folding"
76a2bad 
Couple changes to make it more efficient.

- Because we replacing nodes that only have a single value, only store a single value instead of the whole tensor for node replacement
- torch.fx.Interpreter will preserve a Tensor in the env as long as it has more uses. That also applies even to output uses, but we are not going to constant fold that use. Instead of using last use for garbage collection, use last non output use. 

If reviewers would prefer I ghstack this bc of code movement let me know.

Fix for #108388


cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison eellison mentioned this pull request Sep 1, 2023
Closed
@eellison
Update on "Improve mem efficiency of constant folding"
39f1713 
Couple changes to make it more efficient.

- Because we replacing nodes that only have a single value, only store a single value instead of the whole tensor for node replacement
- torch.fx.Interpreter will preserve a Tensor in the env as long as it has more uses. That also applies even to output uses, but we are not going to constant fold that use. Instead of using last use for garbage collection, use last non output use. 

If reviewers would prefer I ghstack this bc of code movement let me know.

Fix for #108388


cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
@eellison
Update on "Improve mem efficiency of constant folding"
0700bf9 
Couple changes to make it more efficient.

- Because we replacing nodes that only have a single value, only store a single value instead of the whole tensor for node replacement
- torch.fx.Interpreter will preserve a Tensor in the env as long as it has more uses. That also applies even to output uses, but we are not going to constant fold that use. Instead of using last use for garbage collection, use last non output use. 

If reviewers would prefer I ghstack this bc of code movement let me know.

Fix for #108388


cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
eellison added a commit that referenced this pull request Sep 1, 2023
@eellison
Improve mem efficiency of constant folding
ecf5252 
ghstack-source-id: 9609414
Pull Request resolved: #108421
@eellison
Copy link
Contributor Author

eellison commented Sep 2, 2023

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 2, 2023
@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team Raised by workflow job

@eellison eellison added the topic: not user facing topic category label Sep 2, 2023
@eellison
Copy link
Contributor Author

eellison commented Sep 2, 2023

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: Command git -C /home/runner/work/pytorch/pytorch cherry-pick -x ecf525257d11ae4a5da4f1bebf948d72b58bcc74 returned non-zero exit code 1

Auto-merging test/inductor/test_torchinductor.py
CONFLICT (content): Merge conflict in test/inductor/test_torchinductor.py
CONFLICT (modify/delete): torch/_inductor/constant_folding.py deleted in HEAD and modified in ecf525257d1 (Improve mem efficiency of constant folding).  Version ecf525257d1 (Improve mem efficiency of constant folding) of torch/_inductor/constant_folding.py left in tree.
Auto-merging torch/_inductor/fx_passes/joint_graph.py
CONFLICT (content): Merge conflict in torch/_inductor/fx_passes/joint_graph.py
error: could not apply ecf525257d1... Improve mem efficiency of constant folding
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
Details for Dev Infra team Raised by workflow job

@eellison
Update on "Improve mem efficiency of constant folding"
fb0988f 
Couple changes to make it more efficient.

- Because we replacing nodes that only have a single value, only store a single value instead of the whole tensor for node replacement
- torch.fx.Interpreter will preserve a Tensor in the env as long as it has more uses. That also applies even to output uses, but we are not going to constant fold that use. Instead of using last use for garbage collection, use last non output use. 

If reviewers would prefer I ghstack this bc of code movement let me know.

Fix for #108388


cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov

[ghstack-poisoned]
eellison added a commit that referenced this pull request Sep 5, 2023
@eellison
Improve mem efficiency of constant folding
18bf94b 
ghstack-source-id: cc1e242
Pull Request resolved: #108421
@eellison eellison mentioned this pull request Sep 5, 2023
Closed
@eellison
Copy link
Contributor Author

eellison commented Sep 5, 2023

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@eellison
Copy link
Contributor Author

eellison commented Sep 6, 2023

@eellison has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot facebook-github-bot deleted the gh/eellison/524/head branch September 9, 2023 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jansel jansel jansel approved these changes

@xw285cornell xw285cornell Awaiting requested review from xw285cornell

@voznesenskym voznesenskym Awaiting requested review from voznesenskym

Assignees

No one assigned

Labels

ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged module: inductor topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@eellison @jansel @pytorchmergebot