KEMBAR78
New export API with dynamic shape specifications instead of constraints by avikchaudhuri · Pull Request #108448 · pytorch/pytorch · GitHub
Skip to content

New export API with dynamic shape specifications instead of constraints #108448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 26 commits into from
Closed

New export API with dynamic shape specifications instead of constraints #108448

wants to merge 26 commits into from

Conversation

@avikchaudhuri
Copy link
Contributor

@avikchaudhuri avikchaudhuri commented Sep 1, 2023 •
edited
Loading

Stack from ghstack (oldest at bottom):

Our experience using constraints / dynamic_dim with the existing export API has found it to be (subjectively) clunky and (objectively) verbose in common cases.

This PR implements a new design for the export API that replaces the use of constraints / dynamic_dim with a new way of specifying dynamic shapes, involving the following concepts:

  • a constructor Dim for first-class named dynamic dimensions with ranges (similar to functorch.dim, and analogous to internal symbolic sizes)
  • a mechanism that uses the above in export calls to associate inputs to their dynamic shape specifications (dynamic_shapes)

Design doc: https://docs.google.com/presentation/d/168U7XK72C_WSsZpGESP6Cho9udh193fi0gfjxCNcJ4E/edit#slide=id.p (Meta-only). Note that we only implement Option 1 in that doc. An older version of this PR also implemented Option 3, which is an alternative way of specifying dynamic shapes using tensor type annotations on the exported callable; but we have moved that to future work for now.

See docs for these new features in torch.export. The existing torch.export.export is modified to use the new API, torch._export.export__RC__, whenever constraints=None. We have not deprecated the existing API yet, but will do in a follow-up.

Constraint violation errors arising through use of the new API will now contain suggested fixes using the new API. No longer do we need to report all specializations for static dimensions and suggest all constraints over dynamic dimensions to fix such errors. Instead, due to the redesign, the suggested fixes are much more concise, only involving modifying the definitions of relevant Dims.

Differential Revision: D48919204

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng

@pytorch-bot
Copy link

pytorch-bot bot commented Sep 1, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/108448

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit f0fd8a1 with merge base 0735f6c (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

avikchaudhuri added a commit that referenced this pull request Sep 1, 2023
@avikchaudhuri
[RFC] alternative designs for export api
87b7470 
Pull Request resolved: #108448


ghstack-source-id: 199529853
@exported-using-ghexport

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)
avikchaudhuri added a commit that referenced this pull request Sep 11, 2023
@avikchaudhuri
[RFC] alternative designs for export api
f1f9f16 
Pull Request resolved: #108448


ghstack-source-id: 200335183
@exported-using-ghexport

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)
avikchaudhuri added a commit that referenced this pull request Sep 12, 2023
@avikchaudhuri
[RFC] alternative designs for export api
7e90c3e 
Pull Request resolved: #108448


ghstack-source-id: 200490523
@exported-using-ghexport

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)
avikchaudhuri added a commit that referenced this pull request Sep 13, 2023
@avikchaudhuri
[RFC] alternative designs for export api
dfbfc4b 
Pull Request resolved: #108448


ghstack-source-id: 200541237
@exported-using-ghexport

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)
@pytorch-bot pytorch-bot bot added the release notes: fx release notes category label Sep 13, 2023
@avikchaudhuri
Update on "[RFC] alternative designs for export api"
7699cfc 
Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng

[ghstack-poisoned]
avikchaudhuri added a commit that referenced this pull request Sep 14, 2023
@avikchaudhuri
[RFC] alternative designs for export api
a813132 
Pull Request resolved: #108448


ghstack-source-id: 200691599
@exported-using-ghexport

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)
@avikchaudhuri
Update on "[RFC] alternative designs for export api"
6b42019 
Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng

[ghstack-poisoned]
@avikchaudhuri
Update on "New export API with dynamic shape specifications instead o…
8d18d20 
…f constraints"


Our experience using `constraints` / `dynamic_dim` with the existing export API has found it to be (subjectively) clunky and (objectively) verbose in common cases.

This PR implements a new design for the export API that replaces the use of `constraints` / `dynamic_dim` with a new way of specifying dynamic shapes, involving the following concepts: 
* a constructor `Dim` for first-class named dynamic dimensions with ranges (similar to `functorch.dim`, and analogous to internal symbolic sizes)
* a special value `_` denoting a static dimension
* specification mechanisms that use the above, for use either at an export call (`dynamic_shapes`) or on the export function itself (`TensorType`).

Design doc: https://docs.google.com/presentation/d/168U7XK72C_WSsZpGESP6Cho9udh193fi0gfjxCNcJ4E/edit#slide=id.p (Meta-only).

See docs for these new features in `torch._export`. The existing `torch.export.export` is modified to use the new API, `torch._export.export__RC__`, whenever `constraints=None`. We have not deprecated the existing API yet, but will do in a follow-up.

Constraint violation errors arising through use of the new API will now contain suggested fixes using the new API. No longer do we need to report all specializations for static dimensions and suggest all constraints over dynamic dimensions to fix such errors. Instead, due to the redesign, the suggested fixes are much more concise, only involving modifying the definitions of relevant `Dim`s.

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng

[ghstack-poisoned]
@avikchaudhuri
Update on "New export API with dynamic shape specifications instead o…
3be455e 
…f constraints"


Our experience using `constraints` / `dynamic_dim` with the existing export API has found it to be (subjectively) clunky and (objectively) verbose in common cases.

This PR implements a new design for the export API that replaces the use of `constraints` / `dynamic_dim` with a new way of specifying dynamic shapes, involving the following concepts: 
* a constructor `Dim` for first-class named dynamic dimensions with ranges (similar to `functorch.dim`, and analogous to internal symbolic sizes)
* a mechanism that uses the above in `export` calls to associate inputs to their dynamic shape specifications (`dynamic_shapes`)

Design doc: https://docs.google.com/presentation/d/168U7XK72C_WSsZpGESP6Cho9udh193fi0gfjxCNcJ4E/edit#slide=id.p (Meta-only). Note that we only implement Option 1 in that doc. An older version of this PR also implemented Option 3, which is an alternative way of specifying dynamic shapes using tensor type annotations on the exported callable; but we have moved that to future work for now.

See docs for these new features in `torch.export`. The existing `torch.export.export` is modified to use the new API, `torch._export.export__RC__`, whenever `constraints=None`. We have not deprecated the existing API yet, but will do in a follow-up.

Constraint violation errors arising through use of the new API will now contain suggested fixes using the new API. No longer do we need to report all specializations for static dimensions and suggest all constraints over dynamic dimensions to fix such errors. Instead, due to the redesign, the suggested fixes are much more concise, only involving modifying the definitions of relevant `Dim`s.

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng

[ghstack-poisoned]
@avikchaudhuri
Update on "New export API with dynamic shape specifications instead o…
9ad8b5a 
…f constraints"


Our experience using `constraints` / `dynamic_dim` with the existing export API has found it to be (subjectively) clunky and (objectively) verbose in common cases.

This PR implements a new design for the export API that replaces the use of `constraints` / `dynamic_dim` with a new way of specifying dynamic shapes, involving the following concepts: 
* a constructor `Dim` for first-class named dynamic dimensions with ranges (similar to `functorch.dim`, and analogous to internal symbolic sizes)
* a mechanism that uses the above in `export` calls to associate inputs to their dynamic shape specifications (`dynamic_shapes`)

Design doc: https://docs.google.com/presentation/d/168U7XK72C_WSsZpGESP6Cho9udh193fi0gfjxCNcJ4E/edit#slide=id.p (Meta-only). Note that we only implement Option 1 in that doc. An older version of this PR also implemented Option 3, which is an alternative way of specifying dynamic shapes using tensor type annotations on the exported callable; but we have moved that to future work for now.

See docs for these new features in `torch.export`. The existing `torch.export.export` is modified to use the new API, `torch._export.export__RC__`, whenever `constraints=None`. We have not deprecated the existing API yet, but will do in a follow-up.

Constraint violation errors arising through use of the new API will now contain suggested fixes using the new API. No longer do we need to report all specializations for static dimensions and suggest all constraints over dynamic dimensions to fix such errors. Instead, due to the redesign, the suggested fixes are much more concise, only involving modifying the definitions of relevant `Dim`s.

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng

[ghstack-poisoned]
@avikchaudhuri
Update on "New export API with dynamic shape specifications instead o…
f0fd8a1 
…f constraints"


Our experience using `constraints` / `dynamic_dim` with the existing export API has found it to be (subjectively) clunky and (objectively) verbose in common cases.

This PR implements a new design for the export API that replaces the use of `constraints` / `dynamic_dim` with a new way of specifying dynamic shapes, involving the following concepts: 
* a constructor `Dim` for first-class named dynamic dimensions with ranges (similar to `functorch.dim`, and analogous to internal symbolic sizes)
* a mechanism that uses the above in `export` calls to associate inputs to their dynamic shape specifications (`dynamic_shapes`)

Design doc: https://docs.google.com/presentation/d/168U7XK72C_WSsZpGESP6Cho9udh193fi0gfjxCNcJ4E/edit#slide=id.p (Meta-only). Note that we only implement Option 1 in that doc. An older version of this PR also implemented Option 3, which is an alternative way of specifying dynamic shapes using tensor type annotations on the exported callable; but we have moved that to future work for now.

See docs for these new features in `torch.export`. The existing `torch.export.export` is modified to use the new API, `torch._export.export__RC__`, whenever `constraints=None`. We have not deprecated the existing API yet, but will do in a follow-up.

Constraint violation errors arising through use of the new API will now contain suggested fixes using the new API. No longer do we need to report all specializations for static dimensions and suggest all constraints over dynamic dimensions to fix such errors. Instead, due to the redesign, the suggested fixes are much more concise, only involving modifying the definitions of relevant `Dim`s.

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx chenyang78 aakhundov kadeng

[ghstack-poisoned]
avikchaudhuri added a commit that referenced this pull request Sep 21, 2023
@avikchaudhuri
New export API with dynamic shape specifications instead of constraints
d26b181 
Pull Request resolved: #108448


ghstack-source-id: 201557160
@exported-using-ghexport

Differential Revision: [D48919204](https://our.internmc.facebook.com/intern/diff/D48919204/)
@avikchaudhuri
Copy link
Contributor Author

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 22, 2023
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@gmagogsfm gmagogsfm mentioned this pull request Sep 22, 2023
Closed
@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: 1 jobs have failed, first few of them are: trunk / linux-focal-rocm5.6-py3.8 / test (default, 1, 3, linux.rocm.gpu)

Details for Dev Infra team Raised by workflow job

@avikchaudhuri
Copy link
Contributor Author

@pytorchbot merge -f "unrelated CI failure"

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@facebook-github-bot facebook-github-bot deleted the gh/avikchaudhuri/43/head branch September 25, 2023 14:24
@angelayi angelayi mentioned this pull request Sep 26, 2023
Closed
angelayi added a commit to angelayi/pytorch that referenced this pull request Sep 26, 2023
@angelayi @facebook-github-bot
[export] Add dynamic_shapes to _export.aot_compile (pytorch#110101)
6997aa5 
Summary:

Following the new dynamic_shapes API (introduced in pytorch#108448), we will also add a dynamic_shapes API to _export.aot_compile

Test Plan: CI

Reviewed By: gmagogsfm

Differential Revision: D49653815
pytorchmergebot pushed a commit that referenced this pull request Sep 27, 2023
@angelayi @pytorchmergebot
[export] Add dynamic_shapes to _export.aot_compile (#110101)
ddbf1aa 
Summary: Following the new dynamic_shapes API (introduced in #108448), we will also add a dynamic_shapes API to _export.aot_compile

Test Plan: CI

Differential Revision: D49653815

Pull Request resolved: #110101
Approved by: https://github.com/gmagogsfm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@suo suo suo approved these changes

@gmagogsfm gmagogsfm gmagogsfm approved these changes

@angelayi angelayi Awaiting requested review from angelayi

@tugsbayasgalan tugsbayasgalan Awaiting requested review from tugsbayasgalan

@ydwu4 ydwu4 Awaiting requested review from ydwu4

@zhxchen17 zhxchen17 Awaiting requested review from zhxchen17

Assignees

No one assigned

Labels

ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged module: dynamo release notes: fx release notes category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@avikchaudhuri @pytorchmergebot @suo @gmagogsfm