KEMBAR78
introduce TestingErrorMeta for internal use by pmeier · Pull Request #58917 · pytorch/pytorch · GitHub
Skip to content

Conversation

@pmeier
Copy link
Collaborator

@pmeier pmeier commented May 25, 2021

Stack from ghstack:

In #54780 we opted to return Optional[Exception] from all internal
helper functions. Since then multiple PRs added functionality that needs
to amend the error message. For this we recreate the error

E = TypeVar("E", bound=Exception)
def _amend_error_message(exc: E, msg_fmtstr: str) -> E:
"""Amends an exception message.
Args:
exc (E): Exception.
msg_fmtstr: Format string for the amended message.
Returns:
(E): New exception with amended error message.
"""
return type(exc)(msg_fmtstr.format(str(exc)))

To untangle this a little, this PR introduces the _TestingErrorMeta,
which carries the exception type and the message. The idiom

exc = check_foo():
if exc:
    return exc

is still valid although exc should be renamed to error_meta to
reflect the new nature. In the top-level functions
assert_(equal|close)

exc = check_foo():
if exc:
    raise exc

changes to

error_meta = check_foo():
if error_meta:
    raise error_meta.to_error()

Differential Revision: D29259405

In #54780 we opted to return `Optional[Exception]` from all internal
helper functions. Since then multiple PRs added functionality that needs
to amend the error message. For this we recreate the error

https://github.com/pytorch/pytorch/blob/09a1b1cf87fb8724e644693f2c00318ed2637095/torch/testing/_asserts.py#L417-L430

To untangle this a little, this PR introduces the `_TestingErrorMeta`,
which carries the exception type and the message. The idiom

```python
exc = check_foo():
if exc:
    return exc
```

is still valid although `exc` should be renamed to `error_meta` to
reflect the new nature. In the top-level functions
`assert_(equal|close)`

```python
exc = check_foo():
if exc:
    raise exc
```

changes to

```python
error_meta = check_foo():
if error_meta:
    raise error_meta.to_error()
```

[ghstack-poisoned]
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented May 25, 2021

💊 CI failures summary and remediations

As of commit d1cbf1b (more details on the Dr. CI page and at hud.pytorch.org/pr/58917):


  • 1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_macos_10_13_py3_test (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Jun 18 09:38:45 [E request_callback_no_python.c...quest type 267: Unexpected end of pickler archive.
Jun 18 09:38:45 frame #9: torch::distributed::rpc::RRefUserDelete::fromMessage(torch::distributed::rpc::Message const&) + 40 (0x11e80ed48 in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #10: torch::distributed::rpc::deserializeRequest(torch::distributed::rpc::Message const&) + 191 (0x11e818bcf in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #11: torch::distributed::rpc::RequestCallbackNoPython::processMessage(torch::distributed::rpc::Message&, std::__1::vector<c10::Stream, std::__1::allocator<c10::Stream> >) const + 78 (0x11e7e4cee in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #12: torch::distributed::rpc::RequestCallback::operator()(torch::distributed::rpc::Message&, std::__1::vector<c10::Stream, std::__1::allocator<c10::Stream> >) const + 60 (0x11e7e4c0c in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #13: std::__1::__function::__func<torch::distributed::rpc::TensorPipeAgent::respond(std::__1::shared_ptr<tensorpipe::Pipe>&)::$_7::operator()(tensorpipe::Error const&, c10::intrusive_ptr<torch::distributed::rpc::Message, c10::detail::intrusive_target_default_null_type<torch::distributed::rpc::Message> >, std::__1::vector<c10::Stream, std::__1::allocator<c10::Stream> >)::'lambda'(), std::__1::allocator<torch::distributed::rpc::TensorPipeAgent::respond(std::__1::shared_ptr<tensorpipe::Pipe>&)::$_7::operator()(tensorpipe::Error const&, c10::intrusive_ptr<torch::distributed::rpc::Message, c10::detail::intrusive_target_default_null_type<torch::distributed::rpc::Message> >, std::__1::vector<c10::Stream, std::__1::allocator<c10::Stream> >)::'lambda'()>, void ()>::operator()() + 323 (0x10ff80e13 in libtorch_python.dylib)
Jun 18 09:38:45 frame #14: c10::ThreadPool::main_loop(unsigned long) + 569 (0x110a9d5e9 in libc10.dylib)
Jun 18 09:38:45 frame #15: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, c10::ThreadPool::ThreadPool(int, int, std::__1::function<void ()>)::$_0> >(void*) + 67 (0x110a9dc93 in libc10.dylib)
Jun 18 09:38:45 frame #16: _pthread_start + 148 (0x7fff6d99a109 in libsystem_pthread.dylib)
Jun 18 09:38:45 frame #17: thread_start + 15 (0x7fff6d995b8b in libsystem_pthread.dylib)
Jun 18 09:38:45 
Jun 18 09:38:45 [E request_callback_no_python.cpp:552] Received error while processing request type 267: Unexpected end of pickler archive.
Jun 18 09:38:45 Exception raised from readSlowWithBuffer at ../torch/csrc/jit/serialization/unpickler.cpp:756 (most recent call first):
Jun 18 09:38:45 frame #0: c10::Error::Error(c10::SourceLocation, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) + 98 (0x110aab932 in libc10.dylib)
Jun 18 09:38:45 frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, char const*) + 205 (0x110aaa1cd in libc10.dylib)
Jun 18 09:38:45 frame #2: torch::jit::Unpickler::readSlowWithBuffer(char*, unsigned long) + 285 (0x11e44f47d in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #3: torch::jit::Unpickler::run() + 127 (0x11e445b4f in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #4: torch::jit::Unpickler::parse_ivalue() + 29 (0x11e44597d in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #5: torch::jit::unpickle(std::__1::function<unsigned long (char*, unsigned long)>, std::__1::function<c10::StrongTypePtr (c10::QualifiedName const&)>, c10::ArrayRef<at::Tensor>) + 289 (0x11e41b941 in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #6: torch::jit::unpickle(char const*, unsigned long, std::__1::function<c10::StrongTypePtr (c10::QualifiedName const&)>, c10::ArrayRef<at::Tensor>) + 206 (0x11e41baae in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #7: torch::distributed::rpc::(anonymous namespace)::toIValues(torch::distributed::rpc::Message const&, torch::distributed::rpc::MessageType) + 220 (0x11e80d92c in libtorch_cpu.dylib)
Jun 18 09:38:45 frame #8: torch::distributed::rpc::ForkMessageBase::fromMessage(torch::distributed::rpc::Message const&, torch::distributed::rpc::MessageType) + 25 (0x11e80dfd9 in libtorch_cpu.dylib)

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Copy link
Collaborator

@mruberry mruberry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

In #54780 we opted to return `Optional[Exception]` from all internal
helper functions. Since then multiple PRs added functionality that needs
to amend the error message. For this we recreate the error

https://github.com/pytorch/pytorch/blob/09a1b1cf87fb8724e644693f2c00318ed2637095/torch/testing/_asserts.py#L417-L430

To untangle this a little, this PR introduces the `_TestingErrorMeta`,
which carries the exception type and the message. The idiom

```python
exc = check_foo():
if exc:
    return exc
```

is still valid although `exc` should be renamed to `error_meta` to
reflect the new nature. In the top-level functions
`assert_(equal|close)`

```python
exc = check_foo():
if exc:
    raise exc
```

changes to

```python
error_meta = check_foo():
if error_meta:
    raise error_meta.to_error()
```

[ghstack-poisoned]
In #54780 we opted to return `Optional[Exception]` from all internal
helper functions. Since then multiple PRs added functionality that needs
to amend the error message. For this we recreate the error

https://github.com/pytorch/pytorch/blob/09a1b1cf87fb8724e644693f2c00318ed2637095/torch/testing/_asserts.py#L417-L430

To untangle this a little, this PR introduces the `_TestingErrorMeta`,
which carries the exception type and the message. The idiom

```python
exc = check_foo():
if exc:
    return exc
```

is still valid although `exc` should be renamed to `error_meta` to
reflect the new nature. In the top-level functions
`assert_(equal|close)`

```python
exc = check_foo():
if exc:
    raise exc
```

changes to

```python
error_meta = check_foo():
if error_meta:
    raise error_meta.to_error()
```

[ghstack-poisoned]
In #54780 we opted to return `Optional[Exception]` from all internal
helper functions. Since then multiple PRs added functionality that needs
to amend the error message. For this we recreate the error

https://github.com/pytorch/pytorch/blob/09a1b1cf87fb8724e644693f2c00318ed2637095/torch/testing/_asserts.py#L417-L430

To untangle this a little, this PR introduces the `_TestingErrorMeta`,
which carries the exception type and the message. The idiom

```python
exc = check_foo():
if exc:
    return exc
```

is still valid although `exc` should be renamed to `error_meta` to
reflect the new nature. In the top-level functions
`assert_(equal|close)`

```python
exc = check_foo():
if exc:
    raise exc
```

changes to

```python
error_meta = check_foo():
if error_meta:
    raise error_meta.to_error()
```

[ghstack-poisoned]
@mruberry
Copy link
Collaborator

@mruberry has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@mruberry merged this pull request in 583f072.

@facebook-github-bot facebook-github-bot deleted the gh/pmeier/18/head branch June 25, 2021 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed Merged module: testing Issues related to the torch.testing module (not tests) open source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants