[EZ][MPS] Extend `arange` to bfloat16 #136754

malfet · 2024-09-26T13:46:40Z

Stack from ghstack (oldest at bottom):

RangeFactories class is the only one that uses AT_DISPATCH_MPS_TYPES

Fixes #136624

RangeFactories class is the only one that uses `AT_DISPATCH_MPS_TYPES` Fixes #136624 [ghstack-poisoned]

pytorch-bot · 2024-09-26T13:46:45Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/136754

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

Long queue for Linux runners

✅ You can merge normally! (1 Unrelated Failure)

As of commit 5f01c05 with merge base 3c542ce ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 4, 5, lf.linux.g5.4xlarge.nvidia.gpu) (gh) (similar failure)
inductor/test_flex_decoding.py::TestFlexDecoding::test_non_sparse_mulitple_block_size

This comment was automatically generated by Dr. CI and updates every 15 minutes.

RangeFactories class is the only one that uses `AT_DISPATCH_MPS_TYPES` Fixes #136624 ghstack-source-id: bd377f8 Pull Request resolved: #136754

malfet · 2024-09-26T15:32:04Z

@pytorchbot merge -f "MPS tests + lint are green"

pytorchmergebot · 2024-09-26T15:33:37Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

As minimal supported OS has been rasied to MacOS 13, some basic complex operations should be supported, and the rest could be `xfailed` Pull Request resolved: #136755 Approved by: https://github.com/Skylion007 ghstack dependencies: #136754

`norm` and `masked.normalize` still have to stay in the list Pull Request resolved: #136821 Approved by: https://github.com/Skylion007 ghstack dependencies: #136754, #136755

This was a stupid cast error that caused MPSGraph to crash with the following exception ``` (mpsFileLoc): /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:233:0: error: 'mps.multiply' op requires the same element type for all operands and results (mpsFileLoc): /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:233:0: note: see current operation: %3 = "mps.multiply"(%2, %arg1) : (tensor<1x3x9x9xf16>, tensor<1xf32>) -> tensor<*xf32> (mpsFileLoc): /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:233:0: error: 'mps.multiply' op requires the same element type for all operands and results (mpsFileLoc): /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:233:0: note: see current operation: %3 = "mps.multiply"(%2, %arg1) : (tensor<1x3x9x9xf16>, tensor<1xf32>) -> tensor<*xf32> /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:953: failed assertion `original module failed verification' ``` Pull Request resolved: #136822 Approved by: https://github.com/Skylion007 ghstack dependencies: #136754, #136755, #136821

Before that attempt to run something like ``` % python -c "import torch;dev,dt='mps',torch.int; print(torch.normal(mean=torch.arange(1., 11., device=dev, dtype=dt), std=torch.arange(10, 0, -1, device=dev, dtype=dt)))" ``` Resulted in hard error ``` (mpsFileLoc): /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:233:0: error: 'mps.multiply' op requires the same element type for all operands and results (mpsFileLoc): /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:233:0: note: see current operation: %5 = "mps.multiply"(%2, %arg1) : (tensor<10xf32>, tensor<10xsi32>) -> tensor<*xf32> (mpsFileLoc): /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:233:0: error: 'mps.multiply' op requires the same element type for all operands and results (mpsFileLoc): /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm:233:0: note: see current operation: %5 = "mps.multiply"(%2, %arg1) : (tensor<10xf32>, tensor<10xsi32>) -> tensor<*xf32> /AppleInternal/Library/BuildRoots/e0873e53-5185-11ef-9a51-9ab6d782fe32/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:953: failed assertion `original module failed verification' ``` After the change, it raises a nice type error Pull Request resolved: #136863 Approved by: https://github.com/Skylion007 ghstack dependencies: #136754, #136755, #136821, #136822

Vargol · 2024-10-12T11:44:12Z

Is there any chance of getting this in 2.5.0 or 2.5.1. if its to late for 2.5.0 ?

Vargol · 2024-10-23T14:56:12Z

Sorry to ask again, but there's a 2.5.1 tracker (#138613) now, any chance of getting this cherry picked for 2.5.1.

malfet · 2024-10-23T15:11:12Z

Sorry to ask again, but there's a 2.5.1 tracker (#138613) now, any chance of getting this cherry picked for 2.5.1.

2.5.1 is an emergency release that accepts only regressions, and despite me wanting to have it in the release, it does not meet the criteria. (as there were no bfloat16 support for arange in 2.4)

[EZ][MPS] Extend arange to bfloat16

5f01c05

RangeFactories class is the only one that uses `AT_DISPATCH_MPS_TYPES` Fixes #136624 [ghstack-poisoned]

malfet requested a review from kulinseth as a code owner September 26, 2024 13:46

pytorch-bot bot added ciflow/mps Run MPS tests (subset of trunk) release notes: mps Release notes category labels Sep 26, 2024

malfet added a commit that referenced this pull request Sep 26, 2024

[EZ][MPS] Extend arange to bfloat16

44d28ad

RangeFactories class is the only one that uses `AT_DISPATCH_MPS_TYPES` Fixes #136624 ghstack-source-id: bd377f8 Pull Request resolved: #136754

malfet requested review from Skylion007 and albanD September 26, 2024 13:47

Skylion007 approved these changes Sep 26, 2024

View reviewed changes

malfet mentioned this pull request Sep 26, 2024

[EZ][BE] Add torch.complex to MPS_DTYPES #136755

Closed

pytorchmergebot added the merging label Sep 26, 2024

pytorchmergebot added the Merged label Sep 26, 2024

pytorchmergebot closed this in 68579ef Sep 26, 2024

pytorchmergebot removed the merging label Sep 26, 2024

This was referenced Sep 27, 2024

[BE][MPS] Delete MacOS12 low-precision ops #136821

Closed

[MPS] Fix AvgPool2d for float16 #136822

Closed

malfet mentioned this pull request Sep 27, 2024

[MPS] Error checking/bf16 support for torch.normal #136863

Closed

Vargol mentioned this pull request Oct 18, 2024

[bug]: MacOSX/M1 Flux workflow 5.0.0: TypeError: BFloat16 is not supported on MPS invoke-ai/InvokeAI#6991

Closed

1 task

github-actions bot deleted the gh/malfet/15/head branch November 23, 2024 02:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[EZ][MPS] Extend `arange` to bfloat16 #136754

[EZ][MPS] Extend `arange` to bfloat16 #136754

Uh oh!

malfet commented Sep 26, 2024 •

edited

Loading

Uh oh!

pytorch-bot bot commented Sep 26, 2024 •

edited

Loading

Uh oh!

malfet commented Sep 26, 2024

Uh oh!

pytorchmergebot commented Sep 26, 2024

Uh oh!

Vargol commented Oct 12, 2024

Uh oh!

Vargol commented Oct 23, 2024

Uh oh!

malfet commented Oct 23, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[EZ][MPS] Extend arange to bfloat16 #136754

[EZ][MPS] Extend arange to bfloat16 #136754

Uh oh!

Conversation

malfet commented Sep 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/136754

❗ 1 Active SEVs

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

malfet commented Sep 26, 2024

Uh oh!

pytorchmergebot commented Sep 26, 2024

Merge started

Uh oh!

Vargol commented Oct 12, 2024

Uh oh!

Vargol commented Oct 23, 2024

Uh oh!

malfet commented Oct 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[EZ][MPS] Extend `arange` to bfloat16 #136754

[EZ][MPS] Extend `arange` to bfloat16 #136754

malfet commented Sep 26, 2024 •

edited

Loading

pytorch-bot bot commented Sep 26, 2024 •

edited

Loading

malfet commented Oct 23, 2024 •

edited

Loading