Refactor layout constraint selection logic #148104

zou3519 · 2025-02-27T17:27:38Z

Stack from ghstack (oldest at bottom):

This PR:

cleans up some existing comments that don't make sense anymore
hooks up the "custom_op_default_layout_constraint" back (that seems to
have broken)
cleans up the "lazy registration path" which seems to never get hit
anymore
adds dislike_padding to nodes that require exact strides

Test Plan:

tests + CI

disable padding

cc @ezyang @SherlockNoMad @EikanWang @jgong5 @wenzhe-nrv @voznesenskym @penguinwu @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

pytorch-bot · 2025-02-27T17:27:41Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/148104

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

CI workflows being skipped on PR

✅ You can merge normally! (1 Unrelated Failure)

As of commit 938a1a8 with merge base e9e1aac ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

trunk / macos-py3-arm64-mps / test (mps, 1, 1, macos-m1-13) (gh) (matched macos rule in flaky-rules.json)
File doesn't exist

This comment was automatically generated by Dr. CI and updates every 15 minutes.

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI ghstack-source-id: fe56f43 Pull Request resolved: #148104

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

Fix #148356 This is some sort of short term fix to recover the default behavior to apply layout constraint for custom ops when there are no tags. A longer term attempt to make sure Inductor always gets correct eager strides is here: #148104 cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI ghstack-source-id: 897e845 Pull Request resolved: #148104

Fix #148356 This is some sort of short term fix to recover the default behavior to apply layout constraint for custom ops when there are no tags. A longer term attempt to make sure Inductor always gets correct eager strides is here: #148104 Pull Request resolved: #148367 Approved by: https://github.com/eellison, https://github.com/zou3519

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI ghstack-source-id: ca7cb91 Pull Request resolved: #148104

linux-foundation-easycla · 2025-03-06T04:18:07Z

The committers listed above are authorized under a signed CLA.

✅ login: zou3519 / name: Richard Zou (2798d5d, 8bb8a32, 651b783, 5b2e7c7, afabfc4, c3360aa, aa5aaa8, b91c423, 938a1a8, b415282, b084864, c3858d7, e9c66fa, c56c6fd, b6fc13a, d495b0b, 47461ad)

…orch#150511)" This reverts commit a4bb2f1. Reverted pytorch#150511 on behalf of https://github.com/atalman due to [GH job link](https://github.com/pytorch/pytorch/actions/runs/14357056427/job/40251630946) [HUD commit link](https://hud.pytorch.org/pytorch/pytorch/commit/2e7c9d33e7f933ac3b723cb3bb05b9c88432c25c) ([comment](pytorch#148104 (comment)))

This reverts commit 2e7c9d3. Reverted pytorch#148104 on behalf of https://github.com/atalman due to [GH job link](https://github.com/pytorch/pytorch/actions/runs/14357056427/job/40251630946) [HUD commit link](https://hud.pytorch.org/pytorch/pytorch/commit/2e7c9d33e7f933ac3b723cb3bb05b9c88432c25c) ([comment](pytorch#148104 (comment)))

This PR: - cleans up some existing comments that don't make sense anymore - hooks up the "custom_op_default_layout_constraint" back (that seems to have broken) - cleans up the "lazy registration path" which seems to never get hit anymore - adds dislike_padding to nodes that require exact strides Test Plan: - tests + CI ghstack-source-id: d2287b3 Pull Request resolved: pytorch/pytorch#148104 disable padding

This PR: - cleans up some existing comments that don't make sense anymore - hooks up the "custom_op_default_layout_constraint" back (that seems to have broken) - cleans up the "lazy registration path" which seems to never get hit anymore - adds dislike_padding to nodes that require exact strides Test Plan: - tests + CI disable padding cc ezyang SherlockNoMad EikanWang jgong5 wenzhe-nrv voznesenskym penguinwu Guobing-Chen XiaobingSuper zhuhaozhe blzheng jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI Pull Request resolved: #150511 Approved by: https://github.com/eellison, https://github.com/shunting314 ghstack dependencies: #150495, #148104 ghstack-source-id: 7192ed4

zou3519 · 2025-04-30T15:45:11Z

so, as far as I can tell the failures are unrelated:

I don't have a rocm machine but don't see why this PR would impact the "eager" and "aot_eager" backends (it only changes Inductor
some of the CUDA failures (the tts_angular AOTI one) are still failing on main, so these are unrelated.

If all tests pass (aside from the periodic ones, which are still broken on main), then I will re-land this PR. If the rocm failures come back then I'll ask for help from AMD.

This PR: - cleans up some existing comments that don't make sense anymore - hooks up the "custom_op_default_layout_constraint" back (that seems to have broken) - cleans up the "lazy registration path" which seems to never get hit anymore - adds dislike_padding to nodes that require exact strides Test Plan: - tests + CI disable padding cc ezyang SherlockNoMad EikanWang jgong5 wenzhe-nrv voznesenskym penguinwu Guobing-Chen XiaobingSuper zhuhaozhe blzheng jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI Pull Request resolved: #150511 Approved by: https://github.com/eellison, https://github.com/shunting314 ghstack dependencies: #150495, #148104 ghstack-source-id: 923efe0

zou3519 · 2025-05-02T02:47:37Z

@pytorchbot merge

pytorchmergebot · 2025-05-02T02:49:55Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2025-05-02T02:50:50Z

Merge failed

Reason: 1 jobs have failed, first few of them are: Check mergeability of ghstack PR / ghstack-mergeability-check

Details for Dev Infra team

Raised by workflow job

zou3519 · 2025-05-02T13:11:43Z

@pytorchbot merge -f "unrelated failure?"

pytorchmergebot · 2025-05-02T13:13:31Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2025-05-02T13:14:04Z

Merge failed

Reason: Command git -C /home/runner/work/pytorch/pytorch cherry-pick -x 5fc6f80d155b75b13da64673d2bacb310054cfb5 returned non-zero exit code 1

Auto-merging benchmarks/dynamo/pr_time_benchmarks/expected_results.csv
CONFLICT (content): Merge conflict in benchmarks/dynamo/pr_time_benchmarks/expected_results.csv
Auto-merging torch/_inductor/config.py
Auto-merging torch/_inductor/graph.py
Auto-merging torch/_inductor/lowering.py
error: could not apply 5fc6f80d155... Refactor layout constraint selection logic (#148104)
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config set advice.mergeConflict false"

Details for Dev Infra team

Raised by workflow job

This PR: - cleans up some existing comments that don't make sense anymore - hooks up the "custom_op_default_layout_constraint" back (that seems to have broken) - cleans up the "lazy registration path" which seems to never get hit anymore - adds dislike_padding to nodes that require exact strides Test Plan: - tests + CI disable padding cc ezyang SherlockNoMad EikanWang jgong5 wenzhe-nrv voznesenskym penguinwu Guobing-Chen XiaobingSuper zhuhaozhe blzheng jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI Pull Request resolved: #150511 Approved by: https://github.com/eellison, https://github.com/shunting314 ghstack dependencies: #150495, #148104 ghstack-source-id: f095c0a

pytorchmergebot · 2025-05-02T23:56:38Z

Starting merge as part of PR stack under #150511

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI Pull Request resolved: #150511 Approved by: https://github.com/eellison, https://github.com/shunting314 ghstack dependencies: #148104

Inductor respects exact strides on custom ops by default

c3360aa

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

zou3519 mentioned this pull request Feb 27, 2025

Add torch._library.utils.normalize_args_kwargs #148062

Closed

zou3519 mentioned this pull request Feb 27, 2025

Change arg_kwarg_vals propagation strategy #148046

Closed

pytorch-bot bot added ciflow/inductor module: inductor release notes: fx release notes category labels Feb 27, 2025

This was referenced Feb 27, 2025

Add needs_exact_strides operator tag for Inductor to force exact strides #148063

Closed

Implement needs_exact_strides for mutable custom operators #148091

Closed

Rename node.meta["arg_kwarg_vals"] to node.meta["eager_input_vals"] #148092

Closed

facebook-github-bot added the fx label Feb 27, 2025

zou3519 added the keep-going Don't stop on first failure, keep running tests until the end label Feb 27, 2025

Update on "Inductor respects exact strides on custom ops by default"

e9c66fa

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

Update on "Inductor respects exact strides on custom ops by default"

b91c423

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

Update on "Inductor respects exact strides on custom ops by default"

b084864

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

pytorch-bot bot added the module: dynamo label Feb 28, 2025

shunting314 mentioned this pull request Mar 3, 2025

[inductor] use eager stride for custom op if no tags #148367

Closed

Update on "Inductor respects exact strides on custom ops by default"

651b783

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

Update on "Inductor respects exact strides on custom ops by default"

2798d5d

If a tag is not specified on a custom operator, then inductor will assume that it needs exact strides. Test Plan: - tests + CI [ghstack-poisoned]

zou3519 mentioned this pull request Mar 6, 2025

Fix _del_library #148688

Closed

zou3519 added the ciflow/inductor-periodic label Apr 30, 2025

zou3519 added 2 commits May 1, 2025 09:31

pytorchmergebot added the merging label May 2, 2025

pytorchmergebot removed the merging label May 2, 2025

pytorchmergebot added the merging label May 2, 2025

pytorchmergebot removed the merging label May 2, 2025

zou3519 removed the ciflow/inductor-periodic label May 2, 2025

pytorchmergebot closed this in 2b37a72 May 3, 2025

github-actions bot deleted the gh/zou3519/1145/head branch June 15, 2025 02:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Refactor layout constraint selection logic #148104

Refactor layout constraint selection logic #148104

Uh oh!

zou3519 commented Feb 27, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Feb 27, 2025 •

edited

Loading

Uh oh!

linux-foundation-easycla bot commented Mar 6, 2025 •

edited

Loading

Uh oh!

zou3519 commented Apr 30, 2025 •

edited

Loading

Uh oh!

zou3519 commented May 2, 2025

Uh oh!

pytorchmergebot commented May 2, 2025

Uh oh!

pytorchmergebot commented May 2, 2025

Uh oh!

zou3519 commented May 2, 2025

Uh oh!

pytorchmergebot commented May 2, 2025

Uh oh!

pytorchmergebot commented May 2, 2025

Uh oh!

pytorchmergebot commented May 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Refactor layout constraint selection logic #148104

Refactor layout constraint selection logic #148104

Uh oh!

Conversation

zou3519 commented Feb 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Feb 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/148104

❗ 1 Active SEVs

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

linux-foundation-easycla bot commented Mar 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zou3519 commented Apr 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zou3519 commented May 2, 2025

Uh oh!

pytorchmergebot commented May 2, 2025

Merge started

Uh oh!

pytorchmergebot commented May 2, 2025

Merge failed

Uh oh!

zou3519 commented May 2, 2025

Uh oh!

pytorchmergebot commented May 2, 2025

Merge started

Uh oh!

pytorchmergebot commented May 2, 2025

Merge failed

Uh oh!

pytorchmergebot commented May 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

zou3519 commented Feb 27, 2025 •

edited

Loading

pytorch-bot bot commented Feb 27, 2025 •

edited

Loading

linux-foundation-easycla bot commented Mar 6, 2025 •

edited

Loading

zou3519 commented Apr 30, 2025 •

edited

Loading