[inductor][user triton] Check size hints to determine indexing dtype #137234
Conversation
…int64 indexing [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/137234
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 5de8e4d with merge base 63bbf71 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
…nt32 vs tl.int64 indexing" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang Differential Revision: [D63797975](https://our.internmc.facebook.com/intern/diff/D63797975) [ghstack-poisoned]
|
@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
davidberard98
changed the title
|
@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
davidberard98
added
the
module: user triton
| elif isinstance(arg.expr, (float, sympy.Float)): | ||
| return "fp32" | ||
|
|
||
| # if this is a integer |
| seed = torch.randint( | ||
| low=0, high=2**62, size=(1,), dtype=torch.int64 | ||
| ).item() |
There was a problem hiding this comment.
Do we want to explicitly put this in the above int32 range, so that we don't get errors passing silently? Or maybe check both within and outside int32?
There was a problem hiding this comment.
Good point, updated it to [2**32, 2**62).
There was a problem hiding this comment.
Thanks for doing this, although I realized how insanely unlikely it is sample int32 from [2, 2**62] :)
…xing dtype" Previously, all integer inputs to user-defined triton kernels were assumed to be int32. This would result in errors if your input was actually an int64. This PR checks the value to determine which dtype to use for indexing: if it is known to be < int_max, then use int32 (and add guards if relevant); if we can't check (e.g. unbacked symint), then use int64. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang oulgen aakhundov Differential Revision: [D63797975](https://our.internmc.facebook.com/intern/diff/D63797975) [ghstack-poisoned]
…xing dtype" Previously, all integer inputs to user-defined triton kernels were assumed to be int32. This would result in errors if your input was actually an int64. This PR checks the value to determine which dtype to use for indexing: if it is known to be < int_max, then use int32 (and add guards if relevant); if we can't check (e.g. unbacked symint), then use int64. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang oulgen aakhundov Differential Revision: [D63797975](https://our.internmc.facebook.com/intern/diff/D63797975) [ghstack-poisoned]
|
@davidberard98 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
4 participants
Stack from ghstack (oldest at bottom):
Previously, all integer inputs to user-defined triton kernels were assumed to be int32. This would result in errors if your input was actually an int64.
This PR checks the value to determine which dtype to use for indexing: if it is known to be < int_max, then use int32 (and add guards if relevant); if we can't check (e.g. unbacked symint), then use int64.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @oulgen @aakhundov
Differential Revision: D63797975