Friendly catch exception when fail to initialize XPU devices #141658
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/141658
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 87d974a with merge base 6e61ff4 ( FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
guangyey
added
ciflow/xpu
c10/xpu/XPUFunctions.cpp
Outdated
| enumDevices(gDevicePool.devices); | ||
| } catch (const sycl::exception& e) { | ||
| TORCH_WARN( | ||
| "Failed to initialize XPU devices. Did you install the driver correctly?"); |
There was a problem hiding this comment.
The message is not informative. Could you help refine the message?
There was a problem hiding this comment.
Updated. Do you think it is OK?
guangyey
changed the title
| TORCH_WARN( | ||
| "Failed to initialize XPU devices. The driver may not be installed, installed incorrectly, or incompatible with the current setup. ", | ||
| "Please refer to the guideline (https://github.com/pytorch/pytorch?tab=readme-ov-file#intel-gpu-support) for proper installation and configuration."); | ||
| return; |
There was a problem hiding this comment.
With this change, torch will not crash itself even without any XPU device.
Sounds good to provide a smooth warning, instead of ugly crash.
I assume we have well-defined behavior for below cases in this scenario,
- The init value of device pool is always empty
- No undefined behavior happens even if user does not check the device count.
Pls. note, initDevicePoolCallOnce is called once, that means, the warning will be raised only once.
There was a problem hiding this comment.
Yes, the device pool is always initialized to empty. PyTorch will raise RuntimeError in the following code if the user would like to call the other runtime API. We deliberately warn this message only once to avoid interrupting the user over time.
pytorch/c10/xpu/XPUFunctions.cpp
Lines 173 to 178 in f35bb55
|
"Unrelated failure, there is an issue to track it #141705" |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
…#141658) Pull Request resolved: pytorch#141658 Approved by: https://github.com/EikanWang
5 participants
Stack from ghstack (oldest at bottom):
cc @gujinghui @EikanWang @fengyuan14