Use new canonical path for our own includes of c10d #60660

lw · 2021-06-24T16:31:26Z

Stack from ghstack:

Use new canonical path for our own includes of c10d #60660 Use new canonical path for our own includes of c10d

Now that c10d is part of libtorch it's more natural for its headers to be included from torch/csrc/distributed.

Differential Revision: D29364181

NOTE FOR REVIEWERS: This PR has internal Facebook specific changes or comments, please review them on Phabricator!

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @agolynski @SciPioneer @H-Huang @mrzzd @cbalioglu @gcramer23

Now that c10d is part of libtorch it's more natural for its headers to be included from torch/csrc/distributed. Differential Revision: [D29364181](https://our.internmc.facebook.com/intern/diff/D29364181/) **NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D29364181/)! [ghstack-poisoned]

facebook-github-bot · 2021-06-24T16:31:31Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/60660
📄 Preview docs built from this PR

💊 CI failures summary and remediations

As of commit cb4bb7b (more details on the Dr. CI page):

7/20 failures possibly* introduced in this PR
- 1/7 non-scanned failure(s)
13/20 tentatively recognized as flaky ❄️
- Click here to rerun these jobs

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_linux_bionic_py3_6_clang9_noarch_test (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Jun 24 22:22:25 RuntimeError: test_autograd failed! Received signal: SIGSEGV

Jun 24 22:22:24 Total time based on python measurements:  37.484ms
Jun 24 22:22:24 CPU time measurement python side overhead: 1.91%
Jun 24 22:22:24 ok (1.800s)
Jun 24 22:22:24   test_profiler_aggregation_table (__main__.TestAutograd) ... ok (0.003s)
Jun 24 22:22:24   test_profiler_function_event_avg (__main__.TestAutograd) ... ok (0.002s)
Jun 24 22:22:25   test_profiler_propagation (__main__.TestAutograd) ... Traceback (most recent call last):
Jun 24 22:22:25   File "test/run_test.py", line 1310, in <module>
Jun 24 22:22:25     main()
Jun 24 22:22:25   File "test/run_test.py", line 1289, in main
Jun 24 22:22:25     raise RuntimeError(err_message)
Jun 24 22:22:25 RuntimeError: test_autograd failed! Received signal: SIGSEGV
Jun 24 22:22:25 
Jun 24 22:22:25 real	4m56.954s
Jun 24 22:22:25 user	4m47.486s
Jun 24 22:22:25 sys	0m10.874s
Jun 24 22:22:25 + cleanup
Jun 24 22:22:25 + retcode=1
Jun 24 22:22:25 + set +x
Jun 24 22:22:25 =================== sccache compilation log ===================
Jun 24 22:22:25 =========== If your build fails, please take a look at the log above for possible reasons ===========
Jun 24 22:22:25 Compile requests                      28

5 failures not recognized by patterns:

Job	Step	Action
^{docker-pytorch-linux-xenial-cuda11.1-cudnn8-py3-gcc7}	^{Check if image should be built}	🔁 rerun
^{docker-pytorch-linux-xenial-py3.6-gcc5.4}	^{Check if image should be built}	🔁 rerun
^{docker-pytorch-linux-xenial-py3-clang5-android-ndk-r19c}	^{Check if image should be built}	🔁 rerun
^{docker-pytorch-linux-xenial-py3.6-gcc7}	^{Check if image should be built}	🔁 rerun
^{docker-pytorch-linux-xenial-cuda10.2-cudnn7-py3-gcc7}	^{Check if image should be built}	🔁 rerun

❄️ 13 failures tentatively classified as flaky

but reruns have not yet been triggered to confirm:

pytorch_linux_xenial_cuda11_1_cudnn8_py3_gcc7_test2 (1/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:32:00 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:32:00   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:32:00     build_directory=temp_dir
Jun 24 22:32:00   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:32:00     keep_intermediates=keep_intermediates)
Jun 24 22:32:00   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:32:00     is_standalone=is_standalone)
Jun 24 22:32:00   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:32:00     error_prefix=f"Error building extension '{name}'")
Jun 24 22:32:00   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:32:00     raise RuntimeError(message) from e
Jun 24 22:32:00 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:32:00 FAILED: cpp_c10d_extension.o 
Jun 24 22:32:00 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:32:00 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:32:00                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:32:00 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:10: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:32:00  #include <torch/csrc/distributed/c10d/Types.hpp>
Jun 24 22:32:00           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jun 24 22:32:00 compilation terminated.
Jun 24 22:32:00 ninja: build stopped: subcommand failed.
Jun 24 22:32:00

pytorch_linux_xenial_py3_6_gcc5_4_test (2/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:22:09 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:22:09   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:22:09     build_directory=temp_dir
Jun 24 22:22:09   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:22:09     keep_intermediates=keep_intermediates)
Jun 24 22:22:09   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:22:09     is_standalone=is_standalone)
Jun 24 22:22:09   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:22:09     error_prefix=f"Error building extension '{name}'")
Jun 24 22:22:09   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:22:09     raise RuntimeError(message) from e
Jun 24 22:22:09 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1009\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:22:09 FAILED: cpp_c10d_extension.o 
Jun 24 22:22:09 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1009\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:22:09 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:22:09                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:22:09 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:49: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:22:09 compilation terminated.
Jun 24 22:22:09 ninja: build stopped: subcommand failed.
Jun 24 22:22:09 
Jun 24 22:22:09 
Jun 24 22:22:09 ----------------------------------------------------------------------

pytorch_linux_xenial_py3_6_gcc7_test (3/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:11:01 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:11:01   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:11:01     build_directory=temp_dir
Jun 24 22:11:01   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:11:01     keep_intermediates=keep_intermediates)
Jun 24 22:11:01   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:11:01     is_standalone=is_standalone)
Jun 24 22:11:01   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:11:01     error_prefix=f"Error building extension '{name}'")
Jun 24 22:11:01   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:11:01     raise RuntimeError(message) from e
Jun 24 22:11:01 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:11:01 FAILED: cpp_c10d_extension.o 
Jun 24 22:11:01 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:11:01 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:11:01                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:11:01 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:10: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:11:01  #include <torch/csrc/distributed/c10d/Types.hpp>
Jun 24 22:11:01           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jun 24 22:11:01 compilation terminated.
Jun 24 22:11:01 ninja: build stopped: subcommand failed.
Jun 24 22:11:01

pytorch_linux_xenial_cuda10_2_cudnn7_py3_nogpu_NO_AVX2_test (4/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:32:56 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:32:56   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:32:56     build_directory=temp_dir
Jun 24 22:32:56   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:32:56     keep_intermediates=keep_intermediates)
Jun 24 22:32:56   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:32:56     is_standalone=is_standalone)
Jun 24 22:32:56   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:32:56     error_prefix=f"Error building extension '{name}'")
Jun 24 22:32:56   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:32:56     raise RuntimeError(message) from e
Jun 24 22:32:56 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:32:56 FAILED: cpp_c10d_extension.o 
Jun 24 22:32:56 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:32:56 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:32:56                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:32:56 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:10: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:32:56  #include <torch/csrc/distributed/c10d/Types.hpp>
Jun 24 22:32:56           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jun 24 22:32:56 compilation terminated.
Jun 24 22:32:56 ninja: build stopped: subcommand failed.
Jun 24 22:32:56

pytorch_linux_xenial_cuda10_2_cudnn7_py3_slow_test (5/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:47:08 ConnectionResetError: [Errno 104] Connection reset by peer

Jun 24 22:47:08   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 455, in accept
Jun 24 22:47:08     deliver_challenge(c, self._authkey)
Jun 24 22:47:08   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 722, in deliver_challenge
Jun 24 22:47:08     response = connection.recv_bytes(256)        # reject large message
Jun 24 22:47:08   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 216, in recv_bytes
Jun 24 22:47:08     buf = self._recv_bytes(maxlength)
Jun 24 22:47:08   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 407, in _recv_bytes
Jun 24 22:47:08     buf = self._recv(4)
Jun 24 22:47:08   File "/opt/conda/lib/python3.6/multiprocessing/connection.py", line 379, in _recv
Jun 24 22:47:08     chunk = read(handle, remaining)
Jun 24 22:47:08 ConnectionResetError: [Errno 104] Connection reset by peer
Jun 24 22:47:08 /opt/conda/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 14 leaked semaphores to clean up at shutdown
Jun 24 22:47:08   len(cache))
Jun 24 22:47:12 Process ErrorTrackingProcess-88:
Jun 24 22:47:12 Traceback (most recent call last):
Jun 24 22:47:12   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
Jun 24 22:47:12     self.run()
Jun 24 22:47:12   File "/var/lib/jenkins/workspace/test/test_dataloader.py", line 374, in run
Jun 24 22:47:12     super(ErrorTrackingProcess, self).run()
Jun 24 22:47:12   File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run
Jun 24 22:47:12     self._target(*self._args, **self._kwargs)

pytorch_linux_bionic_py3_8_gcc9_coverage_test2 (6/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 23:25:48 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 23:25:48   File "distributed/test_distributed_fork.py", line 78, in _load_test_backend
Jun 24 23:25:48     extension = torch.utils.cpp_extension.load(
Jun 24 23:25:48   File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1080, in load
Jun 24 23:25:48     return _jit_compile(
Jun 24 23:25:48   File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1293, in _jit_compile
Jun 24 23:25:48     _write_ninja_file_and_build_library(
Jun 24 23:25:48   File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1405, in _write_ninja_file_and_build_library
Jun 24 23:25:48     _run_ninja_build(
Jun 24 23:25:48   File "/opt/conda/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 23:25:48     raise RuntimeError(message) from e
Jun 24 23:25:48 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1013\" -isystem /opt/conda/lib/python3.8/site-packages/torch/include -isystem /opt/conda/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.8/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.8/site-packages/torch/include/THC -isystem /opt/conda/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 23:25:48 FAILED: cpp_c10d_extension.o 
Jun 24 23:25:48 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1013\" -isystem /opt/conda/lib/python3.8/site-packages/torch/include -isystem /opt/conda/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.8/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.8/site-packages/torch/include/THC -isystem /opt/conda/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 23:25:48 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15,
Jun 24 23:25:48                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 23:25:48 /opt/conda/lib/python3.8/site-packages/torch/include/c10d/ProcessGroup.hpp:13:10: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 23:25:48    13 | #include <torch/csrc/distributed/c10d/Types.hpp>
Jun 24 23:25:48       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jun 24 23:25:48 compilation terminated.
Jun 24 23:25:48 ninja: build stopped: subcommand failed.
Jun 24 23:25:48

pytorch_linux_bionic_cuda10_2_cudnn7_py3_9_gcc7_test2 (7/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:42:56 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:42:56   File "/var/lib/jenkins/workspace/test/distributed/test_distributed_fork.py", line 78, in _load_test_backend
Jun 24 22:42:56     extension = torch.utils.cpp_extension.load(
Jun 24 22:42:56   File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1080, in load
Jun 24 22:42:56     return _jit_compile(
Jun 24 22:42:56   File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1293, in _jit_compile
Jun 24 22:42:56     _write_ninja_file_and_build_library(
Jun 24 22:42:56   File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1405, in _write_ninja_file_and_build_library
Jun 24 22:42:56     _run_ninja_build(
Jun 24 22:42:56   File "/opt/conda/lib/python3.9/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:42:56     raise RuntimeError(message) from e
Jun 24 22:42:56 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.9/site-packages/torch/include -isystem /opt/conda/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.9/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.9/site-packages/torch/include/THC -isystem /opt/conda/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:42:56 FAILED: cpp_c10d_extension.o 
Jun 24 22:42:56 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.9/site-packages/torch/include -isystem /opt/conda/lib/python3.9/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.9/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.9/site-packages/torch/include/THC -isystem /opt/conda/include/python3.9 -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:42:56 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:42:56                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:42:56 /opt/conda/lib/python3.9/site-packages/torch/include/c10d/ProcessGroup.hpp:13:10: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:42:56  #include <torch/csrc/distributed/c10d/Types.hpp>
Jun 24 22:42:56           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jun 24 22:42:56 compilation terminated.
Jun 24 22:42:56 ninja: build stopped: subcommand failed.
Jun 24 22:42:56

pytorch_linux_xenial_py3_clang5_asan_test2 (8/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 23:38:44 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 23:38:44   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 23:38:44     build_directory=temp_dir
Jun 24 23:38:44   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 23:38:44     keep_intermediates=keep_intermediates)
Jun 24 23:38:44   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 23:38:44     is_standalone=is_standalone)
Jun 24 23:38:44   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 23:38:44     error_prefix=f"Error building extension '{name}'")
Jun 24 23:38:44   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 23:38:44     raise RuntimeError(message) from e
Jun 24 23:38:44 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 23:38:44 FAILED: cpp_c10d_extension.o 
Jun 24 23:38:44 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 23:38:44 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 23:38:44                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 23:38:44 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:49: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 23:38:44 compilation terminated.
Jun 24 23:38:44 ninja: build stopped: subcommand failed.
Jun 24 23:38:44 
Jun 24 23:38:44 
Jun 24 23:38:44 ----------------------------------------------------------------------

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test2 (9/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:24:36 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:24:36   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:24:36     build_directory=temp_dir
Jun 24 22:24:36   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:24:36     keep_intermediates=keep_intermediates)
Jun 24 22:24:36   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:24:36     is_standalone=is_standalone)
Jun 24 22:24:36   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:24:36     error_prefix=f"Error building extension '{name}'")
Jun 24 22:24:36   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:24:36     raise RuntimeError(message) from e
Jun 24 22:24:36 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:24:36 FAILED: cpp_c10d_extension.o 
Jun 24 22:24:36 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:24:36 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:24:36                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:24:36 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:10: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:24:36  #include <torch/csrc/distributed/c10d/Types.hpp>
Jun 24 22:24:36           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jun 24 22:24:36 compilation terminated.
Jun 24 22:24:36 ninja: build stopped: subcommand failed.
Jun 24 22:24:36

pytorch_macos_10_13_py3_test (10/13)

Step: "Test" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:19:29 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:19:29   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:19:29     build_directory=temp_dir
Jun 24 22:19:29   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:19:29     keep_intermediates=keep_intermediates)
Jun 24 22:19:29   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:19:29     is_standalone=is_standalone)
Jun 24 22:19:29   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:19:29     error_prefix=f"Error building extension '{name}'")
Jun 24 22:19:29   File "/Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:19:29     raise RuntimeError(message) from e
Jun 24 22:19:29 RuntimeError: Error building extension 'torch_test': [1/2] clang++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include -isystem /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include/TH -isystem /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include/THC -isystem /Users/distiller/workspace/miniconda3/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -c /Users/distiller/project/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:19:29 FAILED: cpp_c10d_extension.o 
Jun 24 22:19:29 clang++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_clang\" -DPYBIND11_STDLIB=\"_libcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1002\" -isystem /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include -isystem /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -isystem /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include/TH -isystem /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include/THC -isystem /Users/distiller/workspace/miniconda3/include/python3.7m -D_GLIBCXX_USE_CXX11_ABI=0 -fPIC -std=c++14 -c /Users/distiller/project/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:19:29 In file included from /Users/distiller/project/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:19:29 In file included from /Users/distiller/project/test/cpp_extensions/cpp_c10d_extension.hpp:15:
Jun 24 22:19:29 /Users/distiller/workspace/miniconda3/lib/python3.7/site-packages/torch/include/c10d/ProcessGroup.hpp:13:10: fatal error: 'torch/csrc/distributed/c10d/Types.hpp' file not found
Jun 24 22:19:29 #include <torch/csrc/distributed/c10d/Types.hpp>
Jun 24 22:19:29          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jun 24 22:19:29 1 error generated.
Jun 24 22:19:29 ninja: build stopped: subcommand failed.
Jun 24 22:19:29

pytorch_parallelnative_linux_xenial_py3_6_gcc5_4_test (11/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:16:26 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:16:26   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:16:26     build_directory=temp_dir
Jun 24 22:16:26   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:16:26     keep_intermediates=keep_intermediates)
Jun 24 22:16:26   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:16:26     is_standalone=is_standalone)
Jun 24 22:16:26   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:16:26     error_prefix=f"Error building extension '{name}'")
Jun 24 22:16:26   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:16:26     raise RuntimeError(message) from e
Jun 24 22:16:26 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1009\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:16:26 FAILED: cpp_c10d_extension.o 
Jun 24 22:16:26 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1009\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:16:26 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:16:26                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:16:26 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:49: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:16:26 compilation terminated.
Jun 24 22:16:26 ninja: build stopped: subcommand failed.
Jun 24 22:16:26 
Jun 24 22:16:26 
Jun 24 22:16:26 ----------------------------------------------------------------------

pytorch_paralleltbb_linux_xenial_py3_6_gcc5_4_test (12/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:22:38 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:22:38   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:22:38     build_directory=temp_dir
Jun 24 22:22:38   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:22:38     keep_intermediates=keep_intermediates)
Jun 24 22:22:38   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:22:38     is_standalone=is_standalone)
Jun 24 22:22:38   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:22:38     error_prefix=f"Error building extension '{name}'")
Jun 24 22:22:38   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:22:38     raise RuntimeError(message) from e
Jun 24 22:22:38 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1009\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:22:38 FAILED: cpp_c10d_extension.o 
Jun 24 22:22:38 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1009\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:22:38 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:22:38                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:22:38 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:49: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:22:38 compilation terminated.
Jun 24 22:22:38 ninja: build stopped: subcommand failed.
Jun 24 22:22:38 
Jun 24 22:22:38 
Jun 24 22:22:38 ----------------------------------------------------------------------

pytorch_linux_xenial_cuda10_2_cudnn7_py3_nogpu_NO_AVX_test (13/13)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Jun 24 22:31:33 RuntimeError: Error building ex...ons/cpp_c10d_extension.cpp -o cpp_c10d_extension.o

Jun 24 22:31:33   File "distributed/test_distributed_fork.py", line 81, in _load_test_backend
Jun 24 22:31:33     build_directory=temp_dir
Jun 24 22:31:33   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1092, in load
Jun 24 22:31:33     keep_intermediates=keep_intermediates)
Jun 24 22:31:33   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1303, in _jit_compile
Jun 24 22:31:33     is_standalone=is_standalone)
Jun 24 22:31:33   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
Jun 24 22:31:33     error_prefix=f"Error building extension '{name}'")
Jun 24 22:31:33   File "/opt/conda/lib/python3.6/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
Jun 24 22:31:33     raise RuntimeError(message) from e
Jun 24 22:31:33 RuntimeError: Error building extension 'torch_test': [1/2] c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:31:33 FAILED: cpp_c10d_extension.o 
Jun 24 22:31:33 c++ -MMD -MF cpp_c10d_extension.o.d -DTORCH_EXTENSION_NAME=torch_test -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -fPIC -std=c++14 -c /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp -o cpp_c10d_extension.o 
Jun 24 22:31:33 In file included from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.hpp:15:0,
Jun 24 22:31:33                  from /var/lib/jenkins/workspace/test/cpp_extensions/cpp_c10d_extension.cpp:1:
Jun 24 22:31:33 /opt/conda/lib/python3.6/site-packages/torch/include/c10d/ProcessGroup.hpp:13:10: fatal error: torch/csrc/distributed/c10d/Types.hpp: No such file or directory
Jun 24 22:31:33  #include <torch/csrc/distributed/c10d/Types.hpp>
Jun 24 22:31:33           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jun 24 22:31:33 compilation terminated.
Jun 24 22:31:33 ninja: build stopped: subcommand failed.
Jun 24 22:31:33

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm4.2-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Now that c10d is part of libtorch it's more natural for its headers to be included from torch/csrc/distributed. Differential Revision: [D29364181](https://our.internmc.facebook.com/intern/diff/D29364181/) **NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D29364181/)! ghstack-source-id: 132309062 Pull Request resolved: #60660

Now that c10d is part of libtorch it's more natural for its headers to be included from torch/csrc/distributed. Differential Revision: [D29364181](https://our.internmc.facebook.com/intern/diff/D29364181/) **NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D29364181/)! [ghstack-poisoned]

Pull Request resolved: #60660 Now that c10d is part of libtorch it's more natural for its headers to be included from torch/csrc/distributed. ghstack-source-id: 132350455 Differential Revision: [D29364181](https://our.internmc.facebook.com/intern/diff/D29364181/) **NOTE FOR REVIEWERS**: This PR has internal Facebook specific changes or comments, please review them on [Phabricator](https://our.internmc.facebook.com/intern/diff/D29364181/)!

github-actions · 2021-08-24T08:03:03Z

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
Stale pull requests will automatically be closed 30 days after being marked Stale

lw requested review from H-Huang, cbalioglu, mingzhe09088, mrshenli, pritamdamania87, rohan-varma, wayi1 and zhaojuanmao as code owners June 24, 2021 16:31

facebook-github-bot added the oncall: distributed Add this issue/PR to distributed oncall triage queue label Jun 24, 2021

facebook-github-bot added the cla signed label Jun 24, 2021

lw mentioned this pull request Jun 24, 2021

Move torch/lib/c10d to torch/csrc/distributed/c10d #60543

Closed

lw mentioned this pull request Jun 24, 2021

cmake: Make TORCH_SRC_DIR/lib PRIVATE #60493

Closed

lw added the ci/master label Jun 24, 2021

malfet approved these changes Jun 24, 2021

View reviewed changes

github-actions bot added the Stale label Aug 24, 2021

github-actions bot closed this Apr 13, 2022

facebook-github-bot deleted the gh/lw/223/head branch May 13, 2022 14:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Use new canonical path for our own includes of c10d #60660

Use new canonical path for our own includes of c10d #60660

Uh oh!

lw commented Jun 24, 2021 •

edited by pytorch-probot bot

Loading

Uh oh!

facebook-github-bot commented Jun 24, 2021 •

edited

Loading

Uh oh!

github-actions bot commented Aug 24, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Use new canonical path for our own includes of c10d #60660

Use new canonical path for our own includes of c10d #60660

Uh oh!

Conversation

lw commented Jun 24, 2021 • edited by pytorch-probot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Jun 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_linux_bionic_py3_6_clang9_noarch_test (1/1)

5 failures not recognized by patterns:

❄️ 13 failures tentatively classified as flaky

pytorch_linux_xenial_cuda11_1_cudnn8_py3_gcc7_test2 (1/13)

pytorch_linux_xenial_py3_6_gcc5_4_test (2/13)

pytorch_linux_xenial_py3_6_gcc7_test (3/13)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_nogpu_NO_AVX2_test (4/13)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_slow_test (5/13)

pytorch_linux_bionic_py3_8_gcc9_coverage_test2 (6/13)

pytorch_linux_bionic_cuda10_2_cudnn7_py3_9_gcc7_test2 (7/13)

pytorch_linux_xenial_py3_clang5_asan_test2 (8/13)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test2 (9/13)

pytorch_macos_10_13_py3_test (10/13)

pytorch_parallelnative_linux_xenial_py3_6_gcc5_4_test (11/13)

pytorch_paralleltbb_linux_xenial_py3_6_gcc5_4_test (12/13)

pytorch_linux_xenial_cuda10_2_cudnn7_py3_nogpu_NO_AVX_test (13/13)

ci.pytorch.org: 1 failed

Uh oh!

github-actions bot commented Aug 24, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lw commented Jun 24, 2021 •

edited by pytorch-probot bot

Loading

facebook-github-bot commented Jun 24, 2021 •

edited

Loading