KEMBAR78
OpInfo for `nn.functional.layer_norm` by krshrimali · Pull Request #63276 · pytorch/pytorch · GitHub
Skip to content

OpInfo for nn.functional.layer_norm #63276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 12 commits into from
Closed

OpInfo for nn.functional.layer_norm #63276

wants to merge 12 commits into from
+65 −26

Conversation

@krshrimali
Copy link
Contributor

Please see pytorch/functorch#78 and #54261.

Note:

  • This PR also adds a reference test inspired by existing tests in test_nn.py.

cc: @mruberry @zou3519

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Aug 14, 2021 •
edited
Loading

🔗 Helpful links

💊 CI failures summary and remediations

As of commit b97c6ec (more details on the Dr. CI page):

  • 3/3 failures possibly* introduced in this PR
    • 1/3 non-scanned failure(s)

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_jit_legacy_test (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Aug 19 10:26:17 RuntimeError: test_jit_legacy failed! 
Aug 19 10:26:16 Generated XML report: test-reports/python-unittest/test_jit_legacy/TEST-jit.test_typing.TestTyping-20210819102416.xml
Aug 19 10:26:16 Generated XML report: test-reports/python-unittest/test_jit_legacy/TEST-jit.test_unsupported_ops.TestUnsupportedOps-20210819102416.xml
Aug 19 10:26:16 Generated XML report: test-reports/python-unittest/test_jit_legacy/TEST-jit.test_warn.TestWarn-20210819102416.xml
Aug 19 10:26:16 Generated XML report: test-reports/python-unittest/test_jit_legacy/TEST-jit.test_with.TestWith-20210819102416.xml
Aug 19 10:26:16 Generated XML report: test-reports/python-unittest/test_jit_legacy/TEST-jit.test_data_parallel.TestDataParallel-20210819102416.xml
Aug 19 10:26:17 Traceback (most recent call last):
Aug 19 10:26:17   File "test/run_test.py", line 1095, in <module>
Aug 19 10:26:17     main()
Aug 19 10:26:17   File "test/run_test.py", line 1074, in main
Aug 19 10:26:17     raise RuntimeError(err_message)
Aug 19 10:26:17 RuntimeError: test_jit_legacy failed!
Aug 19 10:26:17 + cleanup
Aug 19 10:26:17 + retcode=1
Aug 19 10:26:17 + set +x
Aug 19 10:26:17 =================== sccache compilation log ===================
Aug 19 10:26:17 =========== If your build fails, please take a look at the log above for possible reasons ===========
Aug 19 10:26:17 Compile requests                     19
Aug 19 10:26:17 Compile requests executed             0
Aug 19 10:26:17 Cache hits                            0
Aug 19 10:26:17 Cache misses                          0
Aug 19 10:26:17 Cache timeouts                        0

1 failure not recognized by patterns:

Job Step Action
CircleCI pytorch_linux_xenial_py3_clang7_asan_test2 Run tests 🔁 rerun

1 job timed out:

  • pytorch_linux_xenial_py3_clang7_asan_test2

ci.pytorch.org: 1 failed

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@krshrimali krshrimali requested a review from zou3519 August 14, 2021 06:19
@astaff astaff added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Aug 16, 2021
@krshrimali krshrimali added the module: testing Issues related to the torch.testing module (not tests) label Aug 17, 2021
zou3519
zou3519 reviewed Aug 18, 2021
View reviewed changes
@zou3519 zou3519 mentioned this pull request Aug 18, 2021
Closed
23 tasks
@facebook-github-bot
Copy link
Contributor

@zou3519 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@zou3519
Copy link
Contributor

zou3519 commented Aug 24, 2021

NB: some internal tests are failing on this. I will investigate and report back.

@zou3519 zou3519 mentioned this pull request Aug 26, 2021
Closed
@zou3519
Copy link
Contributor

zou3519 commented Aug 31, 2021

I'm still working on merging this! The internal tests were failing due to an internal issue

@facebook-github-bot
Copy link
Contributor

@zou3519 merged this pull request in d5bfdd3.

ngimel
ngimel reviewed Sep 1, 2021
View reviewed changes
test/test_nn.py
self._test_LayerNorm_cuda_half(device)

@onlyOnCPUAndCUDA
def test_LayerNorm_numeric(self, device):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@krshrimali please restore this test. It is testing numerics for large inputs, and it is added on purpose. Added OpInfo does not cover this (and it should not, OpInfos are not intended for testing interesting numerics of individual operations).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, sorry for not noticing this! I'll put this back in (and cc' you and krshrimali as reviewers)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#64385

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for not noticing this, thanks for pointing this @ngimel, and thanks @zou3519 for the quick PR. :)

@zou3519 zou3519 mentioned this pull request Sep 1, 2021
Closed
zou3519 added a commit that referenced this pull request Sep 1, 2021
@zou3519
Restore LayerNorm numerics test
bee6ede 
It was deleted in #63276.

The numerics test was meant to check LayerNorm behavior on large inputs,
but we deleted it without realizing that.

Test Plan:
- wait for tests.

[ghstack-poisoned]
zou3519 added a commit that referenced this pull request Sep 1, 2021
@zou3519
Restore LayerNorm numerics test
96041c2 
It was deleted in #63276.

The numerics test was meant to check LayerNorm behavior on large inputs,
but we deleted it without realizing that.

Test Plan:
- wait for tests.

ghstack-source-id: 54dd01c
Pull Request resolved: #64385
facebook-github-bot pushed a commit that referenced this pull request Sep 1, 2021
@zou3519 @facebook-github-bot
Restore LayerNorm numerics test (#64385)
535526b 
Summary:
Pull Request resolved: #64385

It was deleted in #63276.

The numerics test was meant to check LayerNorm behavior on large inputs,
but we deleted it without realizing that.

Test Plan: - wait for tests.

Reviewed By: ngimel

Differential Revision: D30702950

Pulled By: zou3519

fbshipit-source-id: a480e26c45ec38fb628938b70416cdb22d976a46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zou3519 zou3519 zou3519 approved these changes

@ngimel ngimel ngimel left review comments

Assignees

No one assigned

Labels

cla signed Merged module: testing Issues related to the torch.testing module (not tests) open source triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@krshrimali @facebook-github-bot @zou3519 @ngimel @astaff @pytorchbot