KEMBAR78
[torch deploy] add register_module_source by zdevito · Pull Request #58290 · pytorch/pytorch · GitHub
Skip to content

[torch deploy] add register_module_source #58290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 3 commits into from
Closed

[torch deploy] add register_module_source #58290

wants to merge 3 commits into from

Conversation

@zdevito
Copy link
Contributor

@zdevito zdevito commented May 14, 2021 •
edited
Loading

Stack from ghstack:

this is a helper function to get some python source code loaded
on each interpreter without having to use the standard import system
or packages. Useful for debugging or for writing wrapper classes for
handling loaded modules.

Differential Revision: D28435306

@zdevito
[torch deploy] add register_module_source
ce5c873 
this is a helper function to get some python source code loaded
on each interpreter without having to use the standard import system
or packages. Useful for debugging or for writing wrapper classes for
handling loaded modules.

[ghstack-poisoned]
@zdevito zdevito mentioned this pull request May 14, 2021
Closed
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented May 14, 2021 •
edited
Loading

💊 CI failures summary and remediations

As of commit d13c000 (more details on the Dr. CI page):

  • 1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See GitHub Actions build Lint / clang-tidy (1/1)

Step: "Run clang-tidy" (full log | diagnosis details | 🔁 rerun)

2021-06-15T04:34:12.2362142Z /__w/pytorch/pytor...nown type name 'PyConfig' [clang-diagnostic-error] 
2021-06-15T04:34:12.2323171Z   File "tools/clang_tidy.py", line 208, in run_clang_tidy
2021-06-15T04:34:12.2323712Z     output = run_shell_command(command)
2021-06-15T04:34:12.2324288Z   File "tools/clang_tidy.py", line 80, in run_shell_command
2021-06-15T04:34:12.2325496Z     raise RuntimeError(f"Error executing {' '.join(arguments)}: {error_output}")
2021-06-15T04:34:12.2344044Z RuntimeError: Error executing clang-tidy -p build -config {"InheritParentConfig": true, "Checks": " bugprone-*, -bugprone-forward-declaration-namespace, -bugprone-macro-parentheses, -bugprone-lambda-function-name, -bugprone-reserved-identifier, cppcoreguidelines-*, -cppcoreguidelines-avoid-magic-numbers, -cppcoreguidelines-interfaces-global-init, -cppcoreguidelines-macro-usage, -cppcoreguidelines-owning-memory, -cppcoreguidelines-pro-bounds-array-to-pointer-decay, -cppcoreguidelines-pro-bounds-constant-array-index, -cppcoreguidelines-pro-bounds-pointer-arithmetic, -cppcoreguidelines-pro-type-cstyle-cast, -cppcoreguidelines-pro-type-reinterpret-cast, -cppcoreguidelines-pro-type-static-cast-downcast, -cppcoreguidelines-pro-type-union-access, -cppcoreguidelines-pro-type-vararg, -cppcoreguidelines-special-member-functions, -facebook-hte-RelativeInclude, hicpp-exception-baseclass, hicpp-avoid-goto, modernize-*, -modernize-concat-nested-namespaces, -modernize-return-braced-init-list, -modernize-use-auto, -modernize-use-default-member-init, -modernize-use-using, -modernize-use-trailing-return-type, performance-*, -performance-noexcept-move-constructor, -performance-unnecessary-value-param, ", "HeaderFilterRegex": "torch/csrc/.*", "AnalyzeTemporaryDtors": false, "CheckOptions": null} -line-filter [{"name": "torch/csrc/deploy/interpreter/interpreter_impl.cpp", "lines": [[11, 12], [116, 118], [135, 157], [356, 372]]}, {"name": "torch/csrc/deploy/test_deploy.cpp", "lines": [[232, 241]]}] torch/csrc/deploy/interpreter/interpreter_impl.cpp torch/csrc/deploy/test_deploy.cpp: /__w/pytorch/pytorch/torch/csrc/deploy/interpreter/interpreter_impl.cpp:298:5: error: unknown type name 'PyPreConfig' [clang-diagnostic-error]
2021-06-15T04:34:12.2357882Z     PyPreConfig preconfig;
2021-06-15T04:34:12.2358297Z     ^
2021-06-15T04:34:12.2359491Z /__w/pytorch/pytorch/torch/csrc/deploy/interpreter/interpreter_impl.cpp:300:5: error: unknown type name 'PyStatus' [clang-diagnostic-error]
2021-06-15T04:34:12.2360553Z     PyStatus status = Py_PreInitialize(&preconfig);
2021-06-15T04:34:12.2361035Z     ^
2021-06-15T04:34:12.2362142Z /__w/pytorch/pytorch/torch/csrc/deploy/interpreter/interpreter_impl.cpp:303:5: error: unknown type name 'PyConfig' [clang-diagnostic-error]
2021-06-15T04:34:12.2363061Z     PyConfig config;
2021-06-15T04:34:12.2363406Z     ^
2021-06-15T04:34:12.2365250Z /__w/pytorch/pytorch/torch/csrc/deploy/test_deploy.cpp:233:1: warning: variable 'test_info_' provides global access to a non-const object; consider making the pointed-to data 'const' [cppcoreguidelines-avoid-non-const-global-variables]
2021-06-15T04:34:12.2366930Z TEST(TorchpyTest, RegisterModule) {
2021-06-15T04:34:12.2367403Z ^
2021-06-15T04:34:12.2368360Z /__w/pytorch/pytorch/cmake/../third_party/googletest/googletest/include/gtest/gtest.h:2287:42: note: expanded from macro 'TEST'
2021-06-15T04:34:12.2369298Z # define TEST(test_case_name, test_name) GTEST_TEST(test_case_name, test_name)
2021-06-15T04:34:12.2369858Z                                          ^
2021-06-15T04:34:12.2370847Z /__w/pytorch/pytorch/cmake/../third_party/googletest/googletest/include/gtest/gtest.h:2281:3: note: expanded from macro 'GTEST_TEST'
2021-06-15T04:34:12.2371691Z   GTEST_TEST_(test_case_name, test_name, \
This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@zdevito
Copy link
Contributor Author

zdevito commented May 14, 2021

@zdevito has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@zdevito zdevito requested a review from wconstab May 14, 2021 05:19
dgl-intel pushed a commit to dgl-intel/pytorch that referenced this pull request May 14, 2021
@zdevito
[torch deploy] add register_module_source
925ca13 
this is a helper function to get some python source code loaded
on each interpreter without having to use the standard import system
or packages. Useful for debugging or for writing wrapper classes for
handling loaded modules.

ghstack-source-id: 67596c7
Pull Request resolved: pytorch#58290
@zdevito
Update on "[torch deploy] add register_module_source"
0495aa0 
this is a helper function to get some python source code loaded
on each interpreter without having to use the standard import system
or packages. Useful for debugging or for writing wrapper classes for
handling loaded modules.

Differential Revision: [D28435306](https://our.internmc.facebook.com/intern/diff/D28435306)

[ghstack-poisoned]
@zdevito
Copy link
Contributor Author

zdevito commented May 14, 2021

@zdevito has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

wconstab
wconstab approved these changes May 14, 2021
View reviewed changes
Copy link
Contributor

@wconstab wconstab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. does it work for creating modules other than at the top level of main?

(could you create or append modules to existing module heirarchies?) not sure you'd want to anyway, and in cases of more complex codebases you clearly suggest package instead.

@codecov
Copy link

codecov bot commented May 14, 2021 •
edited
Loading

Codecov Report

Merging #58290 (0495aa0) into gh/zdevito/198/base (087ac75) will increase coverage by 10.37%.
The diff coverage is n/a.

❗ Current head 0495aa0 differs from pull request most recent head d13c000. Consider uploading reports for the commit d13c000 to get more accurate results

@@                   Coverage Diff                    @@
##           gh/zdevito/198/base   #58290       +/-   ##
========================================================
+ Coverage                66.07%   76.44%   +10.37%     
========================================================
  Files                     2018     1992       -26     
  Lines                   201251   199843     -1408     
========================================================
+ Hits                    132980   152774    +19794     
+ Misses                   68271    47069    -21202

@zdevito
Update on "[torch deploy] add register_module_source"
d13c000 
this is a helper function to get some python source code loaded
on each interpreter without having to use the standard import system
or packages. Useful for debugging or for writing wrapper classes for
handling loaded modules.

Differential Revision: [D28435306](https://our.internmc.facebook.com/intern/diff/D28435306)

[ghstack-poisoned]
@zdevito
Copy link
Contributor Author

zdevito commented Jun 15, 2021

@zdevito has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@zdevito merged this pull request in 8ce6d0c.

@facebook-github-bot facebook-github-bot deleted the gh/zdevito/198/head branch June 20, 2021 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wconstab wconstab wconstab approved these changes

Assignees

No one assigned

Labels

cla signed Merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zdevito @facebook-github-bot @wconstab