Export XPU libs to be public #136974

guangyey · 2024-09-30T03:01:30Z

Stack from ghstack (oldest at bottom):

Motivation

Export XPU-related libs to be public. Now they are included in TORCH_LIBRARIES

pytorch-bot · 2024-09-30T03:01:33Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/136974

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 20e71fd with merge base 5c3ba6f ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

xpu / linux-jammy-xpu-py3.9 / test (default, 3, 4, linux.idc.xpu) (gh) (similar failure)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoXPU::test_comprehensive_nn_functional_unfold_xpu_bool

This comment was automatically generated by Dr. CI and updates every 15 minutes.

guangyey · 2024-09-30T03:15:07Z

cmake/TorchConfig.cmake.in

  if(@BUILD_SHARED_LIBS@)
-    find_library(C10_CUDA_LIBRARY c10_cuda PATHS "${TORCH_INSTALL_PREFIX}/lib")
-    list(APPEND TORCH_CUDA_LIBRARIES ${C10_CUDA_LIBRARY} ${Caffe2_PUBLIC_CUDA_DEPENDENCY_LIBS})
+    append_torchlib_if_found(c10_cuda torch_cuda)


Previously here miss libtorch_cuda.so

ghstack-source-id: 53b0007 Pull Request resolved: #136974

guangyey · 2024-09-30T03:53:26Z

cmake/TorchConfig.cmake.in

    find_library(C10_CUDA_LIBRARY c10_cuda PATHS "${TORCH_INSTALL_PREFIX}/lib")
-    list(APPEND TORCH_CUDA_LIBRARIES ${C10_CUDA_LIBRARY} ${Caffe2_PUBLIC_CUDA_DEPENDENCY_LIBS})
+    find_library(TORCH_CUDA_LIBRARY c10_cuda PATHS "${TORCH_INSTALL_PREFIX}/lib")
+    list(APPEND TORCH_CUDA_LIBRARIES ${C10_CUDA_LIBRARY} ${TORCH_CUDA_LIBRARY} ${Caffe2_PUBLIC_CUDA_DEPENDENCY_LIBS})


To keep TORCH_CUDA_LIBRARIES BC, here doesn't directly use append_torchlib_if_found. Because append_torchlib_if_found doesn't update TORCH_CUDA_LIBRARIES.

This change seems unrelated to the PR title. Do you mind proposing it in separate PR (and posting some sort of examples of what behavior it fixes)

[ghstack-poisoned]

guangyey · 2024-10-02T07:03:06Z

@pytorchbot rebase

pytorchmergebot · 2024-10-02T07:04:42Z

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[ghstack-poisoned]

pytorchmergebot · 2024-10-02T07:04:56Z

Successfully rebased gh/guangyey/70/orig onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via ghstack checkout https://github.com/pytorch/pytorch/pull/136974)

ghstack-source-id: ce7560b Pull Request resolved: #136974

guangyey · 2024-10-09T02:28:35Z

@pytorchbot rebase

pytorchmergebot · 2024-10-09T02:30:05Z

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[ghstack-poisoned]

pytorchmergebot · 2024-10-09T02:30:17Z

Successfully rebased gh/guangyey/70/orig onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via ghstack checkout https://github.com/pytorch/pytorch/pull/136974)

ghstack-source-id: 9fcf121 Pull Request resolved: #136974

guangyey · 2024-10-10T03:42:06Z

@malfet @atalman May I know if you could help review this RP?
The CI failure is unrelated to this PR.

malfet

Can you please help me understand why find_library(TORCH_CUDA_LIBRARY is needed?

gujinghui · 2024-10-10T05:45:16Z

Can you please help me understand why find_library(TORCH_CUDA_LIBRARY is needed?

@malfet Good catch! It should be a typo? @guangyey pls. fix it.

ghstack-source-id: 1ad886e Pull Request resolved: #136974

guangyey · 2024-10-10T05:58:18Z

Can you please help me understand why find_library(TORCH_CUDA_LIBRARY is needed?

Sorry about that. Previously, libtorch_cuda.so was neither added to TORCH_CUDA_LIBRARIES nor exported in TORCH_LIBRARIES. find_library(TORCH_CUDA_LIBRARY intends to export libtorch_cuda.so to TORCH_LIBRARIES.
Why do we prefer find_library(TORCH_CUDA_LIBRARY instead of append_torchlib_if_found?
append_torchlib_if_found doesn't update TORCH_CUDA_LIBRARIES. And we expect libtorch_cuda.so to be added to TORCH_CUDA_LIBRARIES to facilitate user experience.
Is it reasonable to you?

malfet

XPU changes looks ok to me, please make changes to TORCH_CUDA_LIBRARIES in separate PR (and perhaps create an issue to explain what problem does it solve)

malfet · 2024-10-10T13:05:19Z

cmake/TorchConfig.cmake.in

    find_library(C10_CUDA_LIBRARY c10_cuda PATHS "${TORCH_INSTALL_PREFIX}/lib")
-    list(APPEND TORCH_CUDA_LIBRARIES ${C10_CUDA_LIBRARY} ${Caffe2_PUBLIC_CUDA_DEPENDENCY_LIBS})
+    find_library(TORCH_CUDA_LIBRARY c10_cuda PATHS "${TORCH_INSTALL_PREFIX}/lib")
+    list(APPEND TORCH_CUDA_LIBRARIES ${C10_CUDA_LIBRARY} ${TORCH_CUDA_LIBRARY} ${Caffe2_PUBLIC_CUDA_DEPENDENCY_LIBS})


This change seems unrelated to the PR title. Do you mind proposing it in separate PR (and posting some sort of examples of what behavior it fixes)

[ghstack-poisoned]

guangyey · 2024-10-10T14:46:40Z

XPU changes looks ok to me, please make changes to TORCH_CUDA_LIBRARIES in separate PR (and perhaps create an issue to explain what problem does it solve)

I would be happy to separate PR. I file an issue to explain the problem in #137694 and fix it in #137693.

[ghstack-poisoned]

guangyey · 2024-10-12T02:28:36Z

@malfet May I know if I have addressed your comments?

[ghstack-poisoned]

malfet

Looks good to me

malfet · 2024-10-16T03:33:48Z

@pytorchbot merge

pytorchmergebot · 2024-10-16T03:35:37Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

guangyey · 2024-10-16T03:38:16Z

Looks good to me

Thanks very much for your approval.

ghstack-source-id: a4a9966 Pull Request resolved: pytorch/pytorch#136974

guangyey added topic: not user facing topic category ciflow/xpu Run XPU CI tasks labels Sep 30, 2024

guangyey commented Sep 30, 2024

View reviewed changes

guangyey requested a review from malfet September 30, 2024 03:15

pytorchbot added the open source label Sep 30, 2024

guangyey requested review from EikanWang, albanD, atalman and gujinghui September 30, 2024 03:16

guangyey added a commit that referenced this pull request Sep 30, 2024

Export XPU libs to be public

abd0c6e

ghstack-source-id: 53b0007 Pull Request resolved: #136974

guangyey commented Sep 30, 2024

View reviewed changes

guangyey added 2 commits September 30, 2024 10:43

Update

1da6579

[ghstack-poisoned]

Update

68a4b3d

[ghstack-poisoned]

EikanWang approved these changes Oct 1, 2024

View reviewed changes

Update

76afc17

[ghstack-poisoned]

pytorchmergebot pushed a commit that referenced this pull request Oct 2, 2024

Export XPU libs to be public

9b38ac4

ghstack-source-id: ce7560b Pull Request resolved: #136974

albanD removed their request for review October 2, 2024 15:22

Update

f657703

[ghstack-poisoned]

pytorchmergebot pushed a commit that referenced this pull request Oct 9, 2024

Export XPU libs to be public

d406023

ghstack-source-id: 9fcf121 Pull Request resolved: #136974

malfet reviewed Oct 10, 2024

View reviewed changes

guangyey added a commit that referenced this pull request Oct 10, 2024

Export XPU libs to be public

8a0fea9

ghstack-source-id: 1ad886e Pull Request resolved: #136974

guangyey requested a review from malfet October 10, 2024 06:08

malfet reviewed Oct 10, 2024

View reviewed changes

Update

bbe5e87

[ghstack-poisoned]

guangyey mentioned this pull request Oct 10, 2024

Export libtorch_cuda.so to be public #137693

Closed

guangyey requested a review from malfet October 10, 2024 14:44

guangyey added the release notes: xpu release notes category label Oct 10, 2024

Update

16de699

[ghstack-poisoned]

guangyey added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 15, 2024

Update

20e71fd

[ghstack-poisoned]

malfet approved these changes Oct 16, 2024

View reviewed changes

pytorchmergebot added the merging label Oct 16, 2024

pytorchmergebot added the Merged label Oct 16, 2024

pytorchmergebot closed this in 2c94c54 Oct 16, 2024

pytorchmergebot removed the merging label Oct 16, 2024

github-actions bot deleted the gh/guangyey/70/head branch November 16, 2024 02:07

Esquains pushed a commit to Esquains/study1 that referenced this pull request Dec 15, 2024

Export XPU libs to be public

90b5d44

ghstack-source-id: a4a9966 Pull Request resolved: pytorch/pytorch#136974

Export XPU libs to be public #136974

Export XPU libs to be public #136974

Uh oh!

Conversation

guangyey commented Sep 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Uh oh!

pytorch-bot bot commented Sep 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/136974

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

guangyey Sep 30, 2024

Choose a reason for hiding this comment

Uh oh!

guangyey Sep 30, 2024

Choose a reason for hiding this comment

Uh oh!

malfet Oct 10, 2024

Choose a reason for hiding this comment

Uh oh!

guangyey commented Oct 2, 2024

Uh oh!

pytorchmergebot commented Oct 2, 2024

Uh oh!

pytorchmergebot commented Oct 2, 2024

Uh oh!

guangyey commented Oct 9, 2024

Uh oh!

pytorchmergebot commented Oct 9, 2024

Uh oh!

pytorchmergebot commented Oct 9, 2024

Uh oh!

guangyey commented Oct 10, 2024

Uh oh!

malfet left a comment

Choose a reason for hiding this comment

Uh oh!

gujinghui commented Oct 10, 2024

Uh oh!

guangyey commented Oct 10, 2024

Uh oh!

malfet left a comment

Choose a reason for hiding this comment

Uh oh!

malfet Oct 10, 2024

Choose a reason for hiding this comment

Uh oh!

guangyey commented Oct 10, 2024

Uh oh!

guangyey commented Oct 12, 2024

Uh oh!

malfet left a comment

Choose a reason for hiding this comment

Uh oh!

malfet commented Oct 16, 2024

Uh oh!

pytorchmergebot commented Oct 16, 2024

Merge started

Uh oh!

guangyey commented Oct 16, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

guangyey commented Sep 30, 2024 •

edited

Loading

pytorch-bot bot commented Sep 30, 2024 •

edited

Loading