faster shut down #24891
Merged
faster shut down #24891
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
… into snnn/shutdown
adrianlizarraga
previously approved these changes
Jun 2, 2025
skottmckay
reviewed
Jun 2, 2025
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
skottmckay
approved these changes
Jun 5, 2025
khoover
added a commit
to khoover/onnxruntime
that referenced
this pull request
Jun 9, 2025
javier-intel
pushed a commit
to intel/onnxruntime
that referenced
this pull request
Jun 15, 2025
This PR is to reduce the chance of having crashes when a process is shutting down. The main idea is: if we know the process is shutting down(or if we know the ORT DLL won't be reloaded), we do not need to run C++ object destructors. The approach is recommended by Windows SDK's official document and many Windows developers. For example, 18 years ago Raymond Chen wrote a blog [The old-fashioned theory on how processes exit](https://devblogs.microsoft.com/oldnewthing/20070502-00/?p=27023). This is ORT's current behavior. Raymond Chen also wrote a blog [what a better approach is](https://devblogs.microsoft.com/oldnewthing/20120105-00/?p=8683) In our case, when onnxruntime is built as a python package, the DLL(onnxruntime_pybind11_state.pyd ) will never be manually unloaded. Same on Linux. Python does not unload the DLLs on exit. Therefore, we do not need to worry about the potential memory leaks caused by any global variables. Therefore, we do not need to call OrtEnv's destructors, and we do not need to unload any EP dlls. In most cases, people do not unload DLLs on Windows. And, on Linux it is even more complicated because GCC needs to maintain a unique table to avoid odr violations, and this feature makes most C++ shared library cannot be unloaded. So, this change detects if the os is Windows and if the process is shutdown when calling destructors. If yes, the destructor will do nothing. After this change on Windows in most cases OrtEnv will not be destroyed. The only exception is: if someone manually load the DLL and manually unload the DLL, and also do not have a global threadpool. Then I think the user is an advanced user and they should know that they need to destroy all inference session objects and the OrtEnv singleton object before unloading the DLL. Besides, if they have enabled global thread pool, the DLL won't be unloaded if they haven't shutdown the thread pool and delete the OrtEnv object. And, even if the user has manually loaded/unloaded the DLL, there would still be some memory leaks(that are not related to this change). It's hard to get 100% clean.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is to reduce the chance of having crashes when a process is shutting down. The main idea is: if we know the process is shutting down(or if we know the ORT DLL won't be reloaded), we do not need to run C++ object destructors. The approach is recommended by Windows SDK's official document and many Windows developers. For example, 18 years ago Raymond Chen wrote a blog The old-fashioned theory on how processes exit. This is ORT's current behavior. Raymond Chen also wrote a blog what a better approach is
In our case, when onnxruntime is built as a python package, the DLL(onnxruntime_pybind11_state.pyd ) will never be manually unloaded. Same on Linux. Python does not unload the DLLs on exit. Therefore, we do not need to worry about the potential memory leaks caused by any global variables. Therefore we do not need to call OrtEnv's destructors, and we do not need to unload any EP dlls.
In most cases, people do not unload DLLs on Windows. And, on Linux it is even more complicated because GCC needs to maintain a unique table to avoid odr violations, and this feature makes most C++ shared library cannot be unloaded.
So, this change detects if the os is Windows and if the process is shutdown when calling destructors. If yes, the destructor will do nothing.
After this change on Windows in most cases OrtEnv will not be destroyed. The only exception is: if someone manually load the DLL and manually unload the DLL, and also do not have a global threadpool. Then I think the user is an advanced user and they should know that they need to destroy all inference session objects and the OrtEnv singleton object before unloading the DLL. Besides, if they have enabled global thread pool, the DLL won't be unloaded if they haven't shutdown the thread pool and delete the OrtEnv object. And, even if the user has manually loaded/unloaded the DLL, there would still be memory leaks(that are not related to this change). It's hard to get 100% clean.