-
-
Notifications
You must be signed in to change notification settings - Fork 33.2k
GH-96764: rewrite asyncio.wait_for to use asyncio.timeout
#98518
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
45d0725 to
6c944d9
Compare
|
This is more of a POC at this point. I think it is worth changing cc @gvanrossum |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You got a bit farther than I did when I tried this, but I'm still concerned why you had to modify several tests. (Also, there's one test that fails in CI, but doesn't appear to fail when I run it locally.)
Misc/NEWS.d/next/Library/2022-10-22-09-26-43.gh-issue-96764.Dh9Y5L.rst
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LG (one naming nit).
|
I had hoped to first merge one of the alternatives that can be backported to 3.10, so we can declare this fixed in 3.10. Then we would merge this one on top of that, but only in 3.11 and main. But we could also just do a custom fix for 3.10 based on one of @twisteroidambassador's PRs. (I wish you had linked this PR to the same issue rather than creating a new issue, since it's all related.) |
|
In particular, @twisteroidambassador has this PR: #98607 |
a new task so no sleep is required
Fallback to using async_timeout on older python asyncio.wait_for has some underlying problems that are only fixed in cpython 3.12. See python/cpython#98518
Fallback to using async_timeout on older python asyncio.wait_for has some underlying problems that are only fixed in cpython 3.12. See python/cpython#98518
Fallback to using async_timeout on older python asyncio.wait_for has some underlying problems that are only fixed in cpython 3.12. See python/cpython#98518
Fallback to using async_timeout on older python asyncio.wait_for has some underlying problems that are only fixed in cpython 3.12. See python/cpython#98518
asyncio.wait_for has some underlying problems that are only fixed in cpython 3.12. Use async_timeout instead until the minimum supported version is 3.11+ and it can be replaced with asyncio.timeout See python/cpython#98518
async_timeout does not suffer from the same race problems as asyncio.wait_for see python/cpython#98518 for more details
wasted some time because asyncio.wait_for() was suppressing cancellations. [0][1][2] deja vu... [3] Looks like this is finally getting fixed in cpython 3.12 [4] So far away... In attempt to avoid encountering this again, let's try using asyncio.timeout in 3.11, which is how upstream reimplemented wait_for in 3.12 [4], and aiorpcx.timeout_after in 3.8-3.10. [0] python/cpython#86296 [1] https://bugs.python.org/issue42130 [2] https://bugs.python.org/issue45098 [3] kyuupichan/aiorpcX#44 [4] python/cpython#98518
wasted some time because asyncio.wait_for() was suppressing cancellations. [0][1][2] deja vu... [3] Looks like this is finally getting fixed in cpython 3.12 [4] So far away... In attempt to avoid encountering this again, let's try using asyncio.timeout in 3.11, which is how upstream reimplemented wait_for in 3.12 [4], and aiorpcx.timeout_after in 3.8-3.10. [0] python/cpython#86296 [1] https://bugs.python.org/issue42130 [2] https://bugs.python.org/issue45098 [3] kyuupichan/aiorpcX#44 [4] python/cpython#98518
`asyncio.wait_for` creates another tasks which leads to some race conditions in cancelation and a performance hit cpython 3.12 will change the underlying implementation of `asyncio.wait_for` to use `asyncio.wait` but that is still a long way off for many people: python/cpython#98518
* Switch usage of asyncio.wait_for to async_timeout `asyncio.wait_for` creates another tasks which leads to some race conditions in cancelation and a performance hit cpython 3.12 will change the underlying implementation of `asyncio.wait_for` to use `asyncio.wait` but that is still a long way off for many people: python/cpython#98518 * adjust ci
This is an implementation of Python's `asyncio.timeout()` which was introduced
in 3.11. We strongly encourage the use of the following approach instead of
`asyncio.wait_for()`.
async with timeout(delay):
# Your async code
In Python 3.12, `asyncio.wait_for()` is implemented in terms of
`asyncio.timeout()` which fixed a bunch of issues. However, this was never
backported (because of the lack of `async.timeout()`) and there are still many
remainig issues, specially in Python 3.10, in `async.wait_for()`.
See python/cpython#98518
This is an implementation of Python's `asyncio.timeout()` which was introduced
in 3.11. We strongly encourage the use of the following approach instead of
`asyncio.wait_for()`.
async with timeout(delay):
# Your async code
In Python 3.12, `asyncio.wait_for()` is implemented in terms of
`asyncio.timeout()` which fixed a bunch of issues. However, this was never
backported (because of the lack of `async.timeout()`) and there are still many
remainig issues, specially in Python 3.10, in `async.wait_for()`.
See python/cpython#98518
This patch implements `asyncio.timeout()` which was introduced in 3.11, as well +as `asyncio.wait_for()` on top of it. In Python 3.12, `asyncio.wait_for()` is implemented in terms of `asyncio.timeout()` which fixed a bunch of issues. However, this was never backported (because of the lack of `async.timeout()`) and there are still many remainig issues, specially in Python 3.10, in `async.wait_for()`. See python/cpython#98518
This patch implements `asyncio.timeout()` which was introduced in 3.11, as well +as `asyncio.wait_for()` on top of it. In Python 3.12, `asyncio.wait_for()` is implemented in terms of `asyncio.timeout()` which fixed a bunch of issues. However, this was never backported (because of the lack of `async.timeout()`) and there are still many remainig issues, specially in Python 3.10, in `async.wait_for()`. See python/cpython#98518
This patch implements a custom `asyncio.wait_for()` on top of an also custom `asyncio.timeout()`. `asyncio.timeout()` which was introduced in 3.11. In Python 3.12, `asyncio.wait_for()` is implemented in terms of `asyncio.timeout()` which fixed a bunch of issues. However, this was never backported (because of the lack of `async.timeout()`) and there are still many remainig issues, specially in Python 3.10, in `async.wait_for()`. See python/cpython#98518
This patch implements a custom `asyncio.wait_for()` on top of an also custom `asyncio.timeout()`. `asyncio.timeout()` which was introduced in 3.11. In Python 3.12, `asyncio.wait_for()` is implemented in terms of `asyncio.timeout()` which fixed a bunch of issues. However, this was never backported (because of the lack of `async.timeout()`) and there are still many remainig issues, specially in Python 3.10, in `async.wait_for()`. See python/cpython#98518
This patch uses `wait_for2` package to implement `asyncio.wait_for()` for Python < 3.12. In Python 3.12, `asyncio.wait_for()` is implemented in terms of `asyncio.timeout()` which fixed a bunch of issues. However, this was never backported (because of the lack of `async.timeout()`) and there are still many remainig issues, specially in Python 3.10, in `async.wait_for()`. See python/cpython#98518
This patch uses `wait_for2` package to implement `asyncio.wait_for()` for Python < 3.12. In Python 3.12, `asyncio.wait_for()` is implemented in terms of `asyncio.timeout()` which fixed a bunch of issues. However, this was never backported (because of the lack of `async.timeout()`) and there are still many remainig issues, specially in Python 3.10, in `async.wait_for()`. See python/cpython#98518
This PR changes
asyncio.wait_forto useasyncio.timeoutas its underlying implementation. It simplifies the code and makes it easy to understand the cancellation semantics as bothasyncio.timeoutandasyncio.wait_forbehaves similarly.Fixes #86296
Fixes #81839
Fixes #96764
asyncio.wait_forusingasyncio.timeout#96764