KEMBAR78
[Build] Do not regenerate code endlessly without XPU by malfet · Pull Request #140438 · pytorch/pytorch · GitHub
Skip to content

Conversation

@malfet
Copy link
Contributor

@malfet malfet commented Nov 12, 2024

Before this change, if one builds PyTorch without XPU build process will
be perpetually regenerating code because of the reference to non-existing
file, that will make autograd codegened files always out of date, see part of the ninja -d explain torch_cpu output:

ninja explain: output ../torch/csrc/inductor/aoti_torch/generated/c_shim_xpu.cpp doesn't exist
ninja explain: output third_party/kineto/libkineto/CMakeFiles/libkineto_defs.bzl of phony edge with no inputs doesn't exist
ninja explain: third_party/kineto/libkineto/CMakeFiles/libkineto_defs.bzl is dirty
ninja explain: /Users/malfet/git/pytorch/pytorch/torch/csrc/autograd/generated/Functions.cpp is dirty

This is a regression introduced by #139025.

After this change, incremental rebuilds with no changes cause no build actions:

% ninja -j1 -v -d explain -n torch_cpu
ninja explain: output third_party/kineto/libkineto/CMakeFiles/libkineto_defs.bzl of phony edge with no inputs doesn't exist
ninja explain: third_party/kineto/libkineto/CMakeFiles/libkineto_defs.bzl is dirty
ninja: no work to do.

Test plan: Wait for at least on XPU build to finish...

Fixes #140432

Before this change, if one builds PyTorch without XPU build process will
be perpetually regenrating code because of the reference to non-existing
file, that will make autograd codegened files always out of date

THis is a regression introduced by #139025

Fixes #140432
@pytorch-bot
Copy link

pytorch-bot bot commented Nov 12, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/140438

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit cb6ce49 with merge base 5f4a21d (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Nov 12, 2024
@malfet malfet added the ciflow/xpu Run XPU CI tasks label Nov 12, 2024
@malfet malfet requested review from a team and desertfire November 12, 2024 19:13
@malfet
Copy link
Contributor Author

malfet commented Nov 12, 2024

@pytorchbot merge -f "Lint + XPU builds are green"

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pobin6 pushed a commit to pobin6/pytorch that referenced this pull request Dec 5, 2024
Before this change, if one builds PyTorch without XPU build process will
be perpetually regenerating code because of the reference to non-existing
file, that will make autograd codegened files always out of date, see part of the `ninja -d explain torch_cpu` output:
```
ninja explain: output ../torch/csrc/inductor/aoti_torch/generated/c_shim_xpu.cpp doesn't exist
ninja explain: output third_party/kineto/libkineto/CMakeFiles/libkineto_defs.bzl of phony edge with no inputs doesn't exist
ninja explain: third_party/kineto/libkineto/CMakeFiles/libkineto_defs.bzl is dirty
ninja explain: /Users/malfet/git/pytorch/pytorch/torch/csrc/autograd/generated/Functions.cpp is dirty
```

This is a regression introduced by pytorch#139025.

After this change, incremental rebuilds with no changes cause no build actions:
```
% ninja -j1 -v -d explain -n torch_cpu
ninja explain: output third_party/kineto/libkineto/CMakeFiles/libkineto_defs.bzl of phony edge with no inputs doesn't exist
ninja explain: third_party/kineto/libkineto/CMakeFiles/libkineto_defs.bzl is dirty
ninja: no work to do.
```

Test plan: Wait for at least on XPU build to finish...

Fixes pytorch#140432

Pull Request resolved: pytorch#140438
Approved by: https://github.com/kit1980, https://github.com/huydhn
@github-actions github-actions bot deleted the malfet/do-not-rebuild-without-xpu branch December 14, 2024 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/xpu Run XPU CI tasks Merged topic: not user facing topic category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incremental builds always contain of 100+ steps if build without XPU

4 participants