[inductor] Make sure unfuse_addmm and addmm patterns don't overlap #110235
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/110235
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit fab9c1e with merge base 419ec3b ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Inductor has two opposing patterns, ``` addmm -> add + mm add + mm -> addmm ``` This uses the `extra_check` to disable the addmm fusion pattern when the heuristic to unfuse add is met, for consistency. [ghstack-poisoned]
…verlap" Inductor has two opposing patterns, ``` addmm -> add + mm add + mm -> addmm ``` This uses the `extra_check` to disable the addmm fusion pattern when the heuristic to unfuse add is met, for consistency. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
… overlap" Inductor has two opposing patterns, ``` addmm -> add + mm add + mm -> addmm ``` This uses the `extra_check` to disable the addmm fusion pattern when the heuristic to unfuse add is met, for consistency. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
| torch.testing.assert_close(a2, e2) | ||
| self.assertEqual(counters["inductor"]["pattern_matcher_count"], 2) | ||
| self.assertEqual(counters["inductor"]["pattern_matcher_nodes"], 4) | ||
| count, nodes = (2, 4) if should_fuse else (0, 0) |
There was a problem hiding this comment.
Note that these cases weren't actually fused previously, it's just that the pattern replaced them with a lowering that did add + mm.
… overlap" Inductor has two opposing patterns, ``` addmm -> add + mm add + mm -> addmm ``` This uses the `extra_check` to disable the addmm fusion pattern when the heuristic to unfuse add is met, for consistency. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
|
@pytorchbot merge |
Merge failedReason: This PR needs a If not, please add the To add a label, you can comment to pytorchbot, for example For more information, see Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
| def addmm(match, mat1, mat2, inp): | ||
| if isinstance(inp, ir.TensorBox): | ||
| inp_shape = inp.get_size() | ||
| matched = len(inp_shape) <= 2 | ||
| mm_shape = shape_of_mm(mat1, mat2) | ||
| for i, m in zip(inp_shape, mm_shape): | ||
| matched &= i == 1 or i == m | ||
| else: # inp is a Number | ||
| matched = False |
There was a problem hiding this comment.
Nice to move this away from graph lowering pattern, this was overdue..
| return not should_prefer_unfused_addmm(match) | ||
|
|
||
|
|
||
| @register_graph_pattern( |
There was a problem hiding this comment.
cc @yanboliang @jansel we should have some sort of commutative concept that would avoid this duplication
| if not isinstance(inp, torch.fx.Node): | ||
| return False # Input is a number |
There was a problem hiding this comment.
i've made this check before which was fixed by #108160, you can have a fx.Node input which is a SymInt/SymFloat
…atterns don't overlap" Inductor has two opposing patterns, ``` addmm -> add + mm add + mm -> addmm ``` This uses the `extra_check` to disable the addmm fusion pattern when the heuristic to unfuse add is met, for consistency. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
Merge failedReason: New commits were pushed while merging. Please rerun the merge command. Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
5 participants
Stack from ghstack (oldest at bottom):
cast_to_fp64#110232Inductor has two opposing patterns,
This uses the
extra_checkto disable the addmm fusion pattern when theheuristic to unfuse add is met, for consistency.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler