Reland #50999 (Added pow() on CPU for float16 & bfloat16) #55280

imaginary-person · 2021-04-03T19:19:02Z

Reason for relanding

Line 1607 of torch/testing/_internal/common_methods_invocations.py of #50999 had dtype instead of dtype=torch.bool, so 4 of the 9 sample inputs for bool had incorrect dtype. This bug was caught by #54949.

Summary

Added support for pow() on CPU for float16 (Half) and bfloat16 types.
Both pow(Tensor, Scalar) and pow(Tensor, Tensor) are now supported for the aforementioned types.
However autograd isn't supported for Float16 on CPU yet, as log_vml_cpu can't be enabled for it.
@heitorschueroff added pow_tensor_scalar_optimized_kernel to refactor & simplify PowKernel.cpp.
It provides a common path for all the complex types & floating point types (except Float16, due to lack of complete AVX2 vectorization support for it). It replaced code that had previously been duplicated for (float, double) and complex types,
so PowKernel.cpp looks a lot cleaner now.
Enabled (unskipped) some tests for erf, erfc,erfinv, tan and linalg.vector.norm which were being skipped earlier due to pow() not having been implemented for float16 & bfloat16.
Added an OpInfo for pow() & enabled some test cases for pow().
Extended the coverage of existing tests for pow in test_binary_ufuncs.py in order to enable comparison with numpy, even with discontiguous tensors, and added a test to ensure that a runtime error is raised for pow's inplace variant if resizing the base tensor is required during its invocation.
Added float16 & bfloat16 to square's dtype lists in its UnaryUfuncInfo.
Removed redundant dtypesIfCPU and dtypesIfCUDA from OpInfos where they are equal to dtypes.

facebook-github-bot · 2021-04-03T19:20:10Z

💊 CI failures summary and remediations

As of commit f47c554 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

[skip ci]

heitorschueroff · 2021-04-05T15:19:07Z

@imaginary-person Could you rebase this to get clean signals on the fixed upstream failure.

imaginary-person · 2021-04-05T15:21:02Z

Hello @heitorschueroff, I'll do that asap. Sorry for the inconvenience!

Get changes from master repo

Get 55303

imaginary-person · 2021-04-05T22:14:46Z

Hello @heitorschueroff, the 3 failed CI checks are related to PyTorch docs, and were fixed upstream earlier today. Please let me if any further changes are required.

All other CI checks passed.
I apologize again for the inconvenience of having to reland this PR.
Thank you!

Get fixes for docs CI checks

codecov · 2021-04-06T03:50:20Z

Codecov Report

Merging #55280 (f47c554) into master (c7312f5) will increase coverage by 2.79%.
The diff coverage is 81.64%.

@@            Coverage Diff             @@
##           master   #55280      +/-   ##
==========================================
+ Coverage   74.66%   77.46%   +2.79%     
==========================================
  Files        1896     1896              
  Lines      187857   188005     +148     
==========================================
+ Hits       140268   145631    +5363     
+ Misses      47589    42374    -5215

facebook-github-bot · 2021-04-06T12:57:40Z

@heitorschueroff has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

heitorschueroff

LGTM, landing now.

imaginary-person · 2021-04-06T17:00:34Z

Hello @heitorschueroff, a recently landed PR has introduced a conflict related to test_out, that wasn't present earlier. Please let me know if I should resolve it after the FB internal tests would complete, or now. I suspect that the FB internal tests might end if I add another commit to this PR, so I'm currently hesitant to do so. Thanks!

EDIT: now that I think of it, FB internal tests keep running even after another commit. So, I resolved the conflicts that GitHub was warning me about. The internal FB tests that were running are still running here - https://github.com/pytorch/pytorch/runs/2278377934

facebook-github-bot · 2021-04-06T17:55:55Z

@heitorschueroff has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-04-07T15:44:36Z

@heitorschueroff has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

imaginary-person · 2021-04-09T21:11:44Z

Hello @heitorschueroff, please let me know if any further changes are required on this PR. Thanks!

EDIT: I made the following changes:

Revised sample inputs to have different sample inputs for the inplace variant.
Unskipped test_variant_consistency_eager for float32.
Unskipped the ROCm test failure, as ROCm CI tests now run on ROCm 4.1
In make_tensor, the parameter discontiguous was renamed to noncontiguous in Ports logdet from method_tests() to op_db #55743, so it had to be changed in test_binary_ufuncs.py.

Get changes from master repo

heitorschueroff

LGTM

facebook-github-bot · 2021-04-13T15:19:41Z

@heitorschueroff has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-04-13T20:23:55Z

@heitorschueroff merged this pull request in aceceb3.

…orch#55280) Summary: #### Reason for relanding Line 1607 of `torch/testing/_internal/common_methods_invocations.py` of pytorch#50999 had `dtype` instead of `dtype=torch.bool`, so 4 of the 9 sample inputs for `bool` had incorrect dtype. This bug was caught by pytorch#54949. 1. Added support for pow() on CPU for `float16` (`Half`) and `bfloat16` types. Both `pow(Tensor, Scalar)` and `pow(Tensor, Tensor)` are now supported for the aforementioned types. However autograd isn't supported for `Float16` on CPU yet, as `log_vml_cpu` can't be enabled for it. 2. heitorschueroff added `pow_tensor_scalar_optimized_kernel` to refactor & simplify `PowKernel.cpp`. It provides a common path for all the complex types & floating point types (except Float16, due to lack of complete AVX2 vectorization support for it). It replaced code that had previously been duplicated for (float, double) and complex types, so PowKernel.cpp looks a lot cleaner now. 3. Enabled (unskipped) some tests for `erf`, `erfc`,`erfinv`, `tan` and `linalg.vector.norm` which were being skipped earlier due to `pow()` not having been implemented for `float16` & `bfloat16`. 4. Added an OpInfo for `pow()` & enabled some test cases for `pow()`. 5. Extended the coverage of existing tests for `pow` in `test_binary_ufuncs.py` in order to enable comparison with `numpy`, even with discontiguous tensors, and added a test to ensure that a runtime error is raised for `pow`'s inplace variant if resizing the base tensor is required during its invocation. 6. Added `float16` & `bfloat16` to `square`'s dtype lists in its `UnaryUfuncInfo`. 7. Removed redundant `dtypesIfCPU` and `dtypesIfCUDA` from `OpInfo`s where they are equal to `dtypes`. Pull Request resolved: pytorch#55280 Reviewed By: jbschlosser Differential Revision: D27591772 Pulled By: heitorschueroff fbshipit-source-id: c7420811b32595bb3353149a61e54a73f2eb352b

Add dtype=torch.bool in L1607 of common_methods_invocations.py

da3f533

facebook-github-bot added the cla signed label Apr 3, 2021

imaginary-person changed the title ~~Reland #50999~~ Reland #50999 (Added pow() on CPU for float16 & bfloat16) Apr 3, 2021

imaginary-person marked this pull request as draft April 3, 2021 19:25

[skip ci] Merge changes from main repo

21de7da

[skip ci]

pytorchbot added the open source label Apr 3, 2021

imaginary-person added 4 commits April 3, 2021 14:39

[skip ci] Merge changes from main repo

c21a2bf

Add all changes

f705c40

Fix typo

048cc11

Fix typo

fe020d9

This comment has been minimized.

Sign in to view

imaginary-person marked this pull request as ready for review April 3, 2021 21:10

Edit comment to trigger CI

b75d971

heitorschueroff self-requested a review April 5, 2021 15:19

imaginary-person added 2 commits April 5, 2021 10:25

Get changes from master repo

5534cc5

Get changes from master repo

Get 55303 - backward compatibility fix

e9aa9ea

Get 55303

mthrok mentioned this pull request Apr 5, 2021

Adopt native complex dtype in griffnlim pytorch/audio#1368

Merged

3 tasks

Get fixes for docs CI checks

c55226b

Get fixes for docs CI checks

heitorschueroff approved these changes Apr 6, 2021

View reviewed changes

Merge branch 'master' into pow_reland

b763b0c

mruberry mentioned this pull request Apr 7, 2021

OpInfo Tracker #54261

Closed

Merge branch 'master' into pow_reland

eab165b

This comment has been minimized.

Sign in to view

imaginary-person added 10 commits April 12, 2021 00:26

Revise sample inputs for pow

fbd7d82

Get changes from master repo

5f0bb2f

Get changes from master repo

Fix flake8

3e8b523

Remove space

28e6735

Unskip test_variant_consistency_eager

6667b02

Remove spaces :/

0ed5678

AMD wrote that ROCm 4.1 fixed the int8 pow issue

fc26436

Fix flake8

8061b2a

Flake8 again :(

fe2d0e8

discontiguous has been replaced with noncontiguous

f47c554

imaginary-person requested a review from heitorschueroff April 12, 2021 17:51

heitorschueroff approved these changes Apr 13, 2021

View reviewed changes

facebook-github-bot closed this in aceceb3 Apr 13, 2021

facebook-github-bot added the Merged label Apr 13, 2021

Reland #50999 (Added pow() on CPU for float16 & bfloat16) #55280

Reland #50999 (Added pow() on CPU for float16 & bfloat16) #55280

Uh oh!

Conversation

imaginary-person commented Apr 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reason for relanding

Summary

Uh oh!

facebook-github-bot commented Apr 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

Uh oh!

This comment has been minimized.

heitorschueroff commented Apr 5, 2021

Uh oh!

imaginary-person commented Apr 5, 2021

Uh oh!

imaginary-person commented Apr 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Apr 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

facebook-github-bot commented Apr 6, 2021

Uh oh!

heitorschueroff left a comment

Choose a reason for hiding this comment

Uh oh!

imaginary-person commented Apr 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Apr 6, 2021

Uh oh!

facebook-github-bot commented Apr 7, 2021

Uh oh!

This comment has been minimized.

This comment has been minimized.

imaginary-person commented Apr 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

heitorschueroff left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Apr 13, 2021

Uh oh!

facebook-github-bot commented Apr 13, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

imaginary-person commented Apr 3, 2021 •

edited

Loading

facebook-github-bot commented Apr 3, 2021 •

edited

Loading

imaginary-person commented Apr 5, 2021 •

edited

Loading

codecov bot commented Apr 6, 2021 •

edited

Loading

imaginary-person commented Apr 6, 2021 •

edited

Loading

imaginary-person commented Apr 9, 2021 •

edited

Loading