Profile guided optimization for automatic_dynamic #139001

ezyang · 2024-10-27T03:05:41Z

Stack from ghstack (oldest at bottom):

-> Profile guided optimization for automatic_dynamic #139001

Previously: #138052 but the implementation is done from scratch, so I open a new PR.

This implements the ability to save and load profiles of automatic dynamic decisions, so on subsequent runs we can directly make something automatically dynamic. Unlike the previous implementation, this cache is never enabled by default; instead, you have to specify a "job id" that says it's OK to share results. We will be able to automatically populate this id for internal MAST jobs but for generic OSS users you will have to explicitly opt into it.

Signed-off-by: Edward Z. Yang ezyang@meta.com

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov @rec

Differential Revision: D65065497

[ghstack-poisoned]

pytorch-bot · 2024-10-27T03:05:44Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139001

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 1ae1682 with merge base 92d7f29 ():

NEW FAILURE - The following job has failed:

pull / linux-focal-cuda11.8-py3.10-gcc9 / test (distributed, 3, 3, linux.g4dn.12xlarge.nvidia.gpu) (gh)
distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass2_use_new_runtime_True

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Previously: #138052 but the implementation is done from scratch, so I open a new PR. Signed-off-by: Edward Z. Yang <ezyang@meta.com> ghstack-source-id: e1f0761 Pull Request resolved: #139001

torch/_dynamo/config.py

torch/_dynamo/pgo.py

[ghstack-poisoned]

Previously: #138052 but the implementation is done from scratch, so I open a new PR. Signed-off-by: Edward Z. Yang <ezyang@meta.com> ghstack-source-id: 6dfd835 Pull Request resolved: #139001

ezyang · 2024-10-28T15:32:14Z

@ezyang has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[ghstack-poisoned]

Previously: #138052 but the implementation is done from scratch, so I open a new PR. Signed-off-by: Edward Z. Yang <ezyang@meta.com> ghstack-source-id: 868296a Pull Request resolved: #139001

[ghstack-poisoned]

ezyang · 2024-10-29T17:46:45Z

@ezyang has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[ghstack-poisoned]

Previously: #138052 but the implementation is done from scratch, so I open a new PR. Signed-off-by: Edward Z. Yang <ezyang@meta.com> ghstack-source-id: 79b1b5c Pull Request resolved: #139001

ezyang · 2024-10-30T04:15:42Z

@ezyang has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

This reverts commit f6be44c. Reverted #139001 on behalf of https://github.com/ezyang due to more fbcode errors ([comment](#139001 (comment)))

pytorchmergebot · 2024-11-02T13:11:07Z

@ezyang your PR has been successfully reverted.

[ghstack-poisoned]

Previously: #138052 but the implementation is done from scratch, so I open a new PR. Signed-off-by: Edward Z. Yang <ezyang@meta.com> ghstack-source-id: 78f0975 Pull Request resolved: #139001

ezyang · 2024-11-02T13:15:43Z

@ezyang has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

ezyang · 2024-11-02T19:02:48Z

Landing via fbcode as direct fbcode change needed

ezyang · 2024-11-02T23:04:35Z

@pytorchbot merge -f "already merged in fbcode"

pytorchmergebot · 2024-11-02T23:06:12Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-11-02T23:06:27Z

Merge failed

Reason: This PR has internal changes and must be landed via Phabricator! Please try reimporting/rexporting the PR!

Details for Dev Infra team

Raised by workflow job

ezyang · 2024-11-03T03:26:29Z

@pytorchbot merge -f "already merged in fbcode"

pytorchmergebot · 2024-11-03T03:27:52Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-11-03T03:28:05Z

Merge failed

Reason: This PR has internal changes and must be landed via Phabricator! Please try reimporting/rexporting the PR!

Details for Dev Infra team

Raised by workflow job

huydhn · 2024-11-03T06:28:20Z

@pytorchbot merge -f "already merged in fbcode"

pytorchmergebot · 2024-11-03T06:29:44Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Previously: pytorch#138052 but the implementation is done from scratch, so I open a new PR. This implements the ability to save and load profiles of automatic dynamic decisions, so on subsequent runs we can directly make something automatically dynamic. Unlike the previous implementation, this cache is never enabled by default; instead, you have to specify a "job id" that says it's OK to share results. We will be able to automatically populate this id for internal MAST jobs but for generic OSS users you will have to explicitly opt into it. Signed-off-by: Edward Z. Yang <ezyang@meta.com> Differential Revision: [D65065497](https://our.internmc.facebook.com/intern/diff/D65065497) Pull Request resolved: pytorch#139001 Approved by: https://github.com/oulgen

…9001)" This reverts commit a6630bc. Reverted pytorch#139001 on behalf of https://github.com/ezyang due to internal code triggers import cycle ([comment](pytorch#139001 (comment)))

Previously: pytorch#138052 but the implementation is done from scratch, so I open a new PR. This implements the ability to save and load profiles of automatic dynamic decisions, so on subsequent runs we can directly make something automatically dynamic. Unlike the previous implementation, this cache is never enabled by default; instead, you have to specify a "job id" that says it's OK to share results. We will be able to automatically populate this id for internal MAST jobs but for generic OSS users you will have to explicitly opt into it. Signed-off-by: Edward Z. Yang <ezyang@meta.com> Differential Revision: [D65065497](https://our.internmc.facebook.com/intern/diff/D65065497) Pull Request resolved: pytorch#139001 Approved by: https://github.com/oulgen

…9001)" This reverts commit f6be44c. Reverted pytorch#139001 on behalf of https://github.com/ezyang due to more fbcode errors ([comment](pytorch#139001 (comment)))

Previously: pytorch#138052 but the implementation is done from scratch, so I open a new PR. This implements the ability to save and load profiles of automatic dynamic decisions, so on subsequent runs we can directly make something automatically dynamic. Unlike the previous implementation, this cache is never enabled by default; instead, you have to specify a "job id" that says it's OK to share results. We will be able to automatically populate this id for internal MAST jobs but for generic OSS users you will have to explicitly opt into it. Signed-off-by: Edward Z. Yang <ezyang@meta.com> Pull Request resolved: pytorch#139001 Approved by: https://github.com/oulgen

Update

64b161a

[ghstack-poisoned]

This was referenced Oct 27, 2024

Refactor core algorithm for automatic dynamic shapes #138717

Closed

Make automatic_dynamic state live per CodeId, rather than on code object #138740

Closed

pytorch-bot bot added ciflow/inductor module: dynamo labels Oct 27, 2024

github-actions bot requested review from SherlockNoMad, albanD, antoniojkim, bdhirsh and miladm October 27, 2024 03:05

ezyang mentioned this pull request Oct 27, 2024

[POC] automatic_dynamic_local_pgo #138052

Closed

ezyang commented Oct 27, 2024

View reviewed changes

torch/_dynamo/config.py Outdated Show resolved Hide resolved

Skylion007 reviewed Oct 27, 2024

View reviewed changes

torch/_dynamo/pgo.py Outdated Show resolved Hide resolved

ezyang added the topic: new features topic category label Oct 28, 2024

Update

6e4e78b

[ghstack-poisoned]

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 28, 2024

Update

4eaf596

[ghstack-poisoned]

ezyang added the release notes: dynamo label Oct 28, 2024

Update

53f2953

[ghstack-poisoned]

Update

227db55

[ghstack-poisoned]

pytorch-bot bot added the module: inductor label Oct 30, 2024

pytorchmergebot reopened this Nov 2, 2024

Update

1ae1682

[ghstack-poisoned]

pytorchmergebot added the merging label Nov 2, 2024

pytorchmergebot removed the merging label Nov 2, 2024

pytorchmergebot added the merging label Nov 3, 2024

pytorchmergebot removed the merging label Nov 3, 2024

ezyang mentioned this pull request Nov 3, 2024

Add --skip-internal-checks to allow manual force merge of already landed PRs #139570

Closed

pytorchmergebot added the merging label Nov 3, 2024

pytorchmergebot closed this in 585dbfa Nov 3, 2024

pytorchmergebot removed the merging label Nov 3, 2024

github-actions bot deleted the gh/ezyang/2984/head branch December 5, 2024 02:14

Profile guided optimization for automatic_dynamic #139001

Profile guided optimization for automatic_dynamic #139001

Uh oh!

Conversation

ezyang commented Oct 27, 2024 • edited by huydhn Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139001

❌ 1 New Failure

Uh oh!

Uh oh!

Uh oh!

ezyang commented Oct 28, 2024

Uh oh!

ezyang commented Oct 29, 2024

Uh oh!

ezyang commented Oct 30, 2024

Uh oh!

pytorchmergebot commented Nov 2, 2024

Uh oh!

ezyang commented Nov 2, 2024

Uh oh!

ezyang commented Nov 2, 2024

Uh oh!

ezyang commented Nov 2, 2024

Uh oh!

pytorchmergebot commented Nov 2, 2024

Merge started

Uh oh!

pytorchmergebot commented Nov 2, 2024

Merge failed

Uh oh!

ezyang commented Nov 3, 2024

Uh oh!

pytorchmergebot commented Nov 3, 2024

Merge started

Uh oh!

pytorchmergebot commented Nov 3, 2024

Merge failed

Uh oh!

huydhn commented Nov 3, 2024

Uh oh!

pytorchmergebot commented Nov 3, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ezyang commented Oct 27, 2024 •

edited by huydhn

Loading

pytorch-bot bot commented Oct 27, 2024 •

edited

Loading