[CPU] Expand `torch.special.i1` to Half and BF16 #137899

malfet · 2024-10-14T16:06:48Z

To match behavior of torch.special.i0

Noticed while looking at the failures in #137849

Also, add explicit high-precision template specialization for calc_i0 and calc_i1 for BFloat16 and Half

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10

To match behavior for torch.special.i0 Noticed while looking at the failures in #137849

pytorch-bot · 2024-10-14T16:06:52Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/137899

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 030ed3a with merge base f8a5b71 ():

NEW FAILURE - The following job has failed:

pull / linux-jammy-py3.10-clang15-asan / test (default, 6, 6, lf.linux.4xlarge) (gh)
profiler/test_cpp_thread 1/1 failed!

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Skylion007 · 2024-10-14T16:18:55Z

aten/src/ATen/native/Math.h

 }

+// Upcast bfloat16/half input to float for numerical accuracy purposes
+inline c10::BFloat16 calc_i1(c10::BFloat16 a) { return calc_i1(static_cast<float>(a)); }


Why aren't these just template instatiations?

Because

pytorch/aten/src/ATen/native/Math.h

Line 1466 in 929797d

inline typename std::enable_if<std::is_floating_point<T>::value, T>::type

and neither Half nor BF16 are considering floating point types. Which reminds me of another simple BE task (enable_if_t :P )

Skylion007 · 2024-10-14T16:20:33Z

aten/src/ATen/native/Math.h

-// Upcast bfloat16 input to float for numerical accuracy purposes
+// Upcast bfloat16/half input to float for numerical accuracy purposes
 inline c10::BFloat16 calc_i0(c10::BFloat16 a) { return calc_i0(static_cast<float>(a)); }
+inline c10::Half calc_i0(c10::Half a) { return calc_i0(static_cast<float>(a)); }


Same concern

Skylion007 · 2024-10-14T17:31:57Z

@malfet Need to update optests like:

pytorch/torch/testing/_internal/opinfo/definitions/special.py

Line 169 in f8a5b71

"special.i1e",

pytorch/torch/testing/_internal/opinfo/definitions/special.py

Line 139 in f8a5b71

"special.i1",

malfet · 2024-10-14T17:43:23Z

@malfet Need to update optests like:

pytorch/torch/testing/_internal/opinfo/definitions/special.py

Line 169 in f8a5b71

"special.i1e",

pytorch/torch/testing/_internal/opinfo/definitions/special.py

Line 139 in f8a5b71

"special.i1",

I sometimes deliberately delay those to see that our OpInfo is comprehensive enough to fail with unexpected successes...
[Edit] I.e. test_dtypes_special_i1e_cpu should fail, but question is how long will it take to get there :)
[Edit2] And I find out that target-determination determines it's important not to know about the error

malfet · 2024-10-14T21:58:40Z

@pytorchbot merge

pytorchmergebot · 2024-10-14T22:00:47Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-10-14T22:06:00Z

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

pull / linux-jammy-py3.10-clang15-asan / test (default, 3, 6, lf.linux.4xlarge)

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

malfet · 2024-10-15T00:04:49Z

@pytorchbot merge

pytorchmergebot · 2024-10-15T00:06:31Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-10-15T00:16:53Z

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 4, 5, lf.linux.4xlarge.nvidia.gpu)

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

malfet · 2024-10-15T04:20:54Z

@pytorchbot merge

pytorchmergebot · 2024-10-15T04:22:32Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-10-15T06:05:35Z

Merge failed

Reason: 1 mandatory check(s) failed. The first few are:

pull / linux-jammy-py3.10-clang15-asan / test (default, 6, 6, lf.linux.4xlarge)

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

malfet · 2024-10-15T14:54:25Z

@pytorchbot merge -i "cpp_threads failure is clearly unrelated"

pytorch-bot · 2024-10-15T14:54:28Z

❌ 🤖 pytorchbot command failed:

@pytorchbot: error: unrecognized arguments: cpp_threads failure is clearly unrelated

usage: @pytorchbot [-h] {merge,revert,rebase,label,drci,cherry-pick,close} ...

Try @pytorchbot --help for more info.

Skylion007 · 2024-10-15T16:52:43Z

@pytorchbot merge -i

pytorchmergebot · 2024-10-15T16:55:35Z

Merge started

Your change will be merged while ignoring the following 1 checks: pull / linux-jammy-py3.10-clang15-asan / test (default, 6, 6, lf.linux.4xlarge)

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[CPU] Expand torch.special.i1 to Half and BF16

82fb06f

To match behavior for torch.special.i0 Noticed while looking at the failures in #137849

pytorch-bot bot added the module: cpu CPU specific problem (e.g., perf, algorithm) label Oct 14, 2024

malfet requested a review from Skylion007 October 14, 2024 16:06

Update Math.h

beee481

Skylion007 reviewed Oct 14, 2024

View reviewed changes

Skylion007 approved these changes Oct 14, 2024

View reviewed changes

And add i1e

03015dd

malfet added release notes: python_frontend python frontend release notes category topic: improvements topic category labels Oct 14, 2024

malfet added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 14, 2024

malfet mentioned this pull request Oct 14, 2024

TD skips relevant tests #137931

Open

malfet added the ci-no-td Do not run TD on this PR label Oct 14, 2024

And modify opinfo

47217a6

pytorchmergebot added the merging label Oct 14, 2024

pytorchmergebot removed the merging label Oct 14, 2024

Down the rabbithole

1424e22

malfet requested a review from mruberry as a code owner October 14, 2024 23:23

pytorchmergebot added the merging label Oct 15, 2024

pytorchmergebot removed the merging label Oct 15, 2024

And add CUDA

030ed3a

malfet requested review from eqy and syed-ahmed as code owners October 15, 2024 03:41

pytorchmergebot added the merging label Oct 15, 2024

pytorchmergebot removed the merging label Oct 15, 2024

pytorchmergebot added the merging label Oct 15, 2024

pytorchmergebot added the Merged label Oct 15, 2024

pytorchmergebot closed this in e4d7676 Oct 15, 2024

pytorchmergebot removed the merging label Oct 15, 2024

github-actions bot deleted the malfet-patch-2 branch November 15, 2024 02:09

[CPU] Expand torch.special.i1 to Half and BF16 #137899

[CPU] Expand torch.special.i1 to Half and BF16 #137899

Uh oh!

Conversation

malfet commented Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/137899

❌ 1 New Failure

Uh oh!

Skylion007 Oct 14, 2024

Choose a reason for hiding this comment

Uh oh!

malfet Oct 14, 2024

Choose a reason for hiding this comment

Uh oh!

Skylion007 Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Skylion007 commented Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

malfet commented Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

malfet commented Oct 14, 2024

Uh oh!

pytorchmergebot commented Oct 14, 2024

Merge started

Uh oh!

pytorchmergebot commented Oct 14, 2024

Merge failed

Uh oh!

malfet commented Oct 15, 2024

Uh oh!

pytorchmergebot commented Oct 15, 2024

Merge started

Uh oh!

pytorchmergebot commented Oct 15, 2024

Merge failed

Uh oh!

malfet commented Oct 15, 2024

Uh oh!

pytorchmergebot commented Oct 15, 2024

Merge started

Uh oh!

pytorchmergebot commented Oct 15, 2024

Merge failed

Uh oh!

malfet commented Oct 15, 2024

Uh oh!

pytorch-bot bot commented Oct 15, 2024

Uh oh!

Skylion007 commented Oct 15, 2024

Uh oh!

pytorchmergebot commented Oct 15, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[CPU] Expand `torch.special.i1` to Half and BF16 #137899

[CPU] Expand `torch.special.i1` to Half and BF16 #137899

malfet commented Oct 14, 2024 •

edited

Loading

pytorch-bot bot commented Oct 14, 2024 •

edited

Loading

Skylion007 Oct 14, 2024 •

edited

Loading

Skylion007 commented Oct 14, 2024 •

edited

Loading

malfet commented Oct 14, 2024 •

edited

Loading