KEMBAR78
Fix warning when pickle.load torch.Storage by mikaylagawarecki · Pull Request #130246 · pytorch/pytorch · GitHub
Skip to content

Conversation

@mikaylagawarecki
Copy link
Contributor

@mikaylagawarecki mikaylagawarecki commented Jul 8, 2024

Fixes #130242

Since torch.save does not use pickle for storages, the torch.load in _load_from_bytes should not ever be called when torch.load-ing a checkpoint. Setting weights_only=False explicitly in _load_from_bytes to avoid the weights_only warning when using the pickle module

Stack from ghstack (oldest at bottom):

[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Jul 8, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/130246

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ No Failures

As of commit ec4dc8a with merge base ce4d951 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

mikaylagawarecki added a commit that referenced this pull request Jul 8, 2024
ghstack-source-id: 8cacad8
Pull Request resolved: #130246
[ghstack-poisoned]
@pytorch-bot pytorch-bot bot added the release notes: nn release notes category label Jul 10, 2024
mod.eval()
self.assertTrue(len(w) == 0)

def test_parameterlistdict_pickle(self):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverting changes to this test in #129239 whoops :)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to ask why I couldn't find the test_nn.py file in my environment? I have already modified the storage.py file, but I still receive this warning.

[ghstack-poisoned]
mikaylagawarecki added a commit that referenced this pull request Jul 10, 2024
ghstack-source-id: 46b0a35
Pull Request resolved: #130246
@mikaylagawarecki mikaylagawarecki marked this pull request as ready for review July 10, 2024 20:47
Copy link
Collaborator

@albanD albanD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good!

@mikaylagawarecki mikaylagawarecki added release notes: python_frontend python frontend release notes category topic: bug fixes topic category and removed release notes: nn release notes category labels Jul 10, 2024
@mikaylagawarecki
Copy link
Contributor Author

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jul 10, 2024
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@mikaylagawarecki
Copy link
Contributor Author

mikaylagawarecki commented Jul 16, 2024

From the discussion re FutureWarnings on #130658 in triage review yesterday, it was discussed that resolving those should be milestoned for 2.4.1

This PR fixed another related weights_only warning when pickle.dump torch.Storage mentioned here #130663 (comment), so I am also adding this one to track that this PR should be in 2.4.1 @atalman @malfet, let me know if this makes sense

xuhancn pushed a commit to xuhancn/pytorch that referenced this pull request Jul 25, 2024
Fixes pytorch#130242

Since `torch.save` does not use pickle for storages, the `torch.load` in `_load_from_bytes` should not ever be called when `torch.load`-ing a checkpoint. Setting weights_only=False explicitly in `_load_from_bytes` to avoid the weights_only warning when using the pickle module

Pull Request resolved: pytorch#130246
Approved by: https://github.com/albanD
@atalman
Copy link
Contributor

atalman commented Aug 15, 2024

@pytorchbot cherry-pick --onto release/2.4 -c critical --fixes #130242

pytorchbot pushed a commit that referenced this pull request Aug 15, 2024
Fixes #130242

Since `torch.save` does not use pickle for storages, the `torch.load` in `_load_from_bytes` should not ever be called when `torch.load`-ing a checkpoint. Setting weights_only=False explicitly in `_load_from_bytes` to avoid the weights_only warning when using the pickle module

Pull Request resolved: #130246
Approved by: https://github.com/albanD

(cherry picked from commit dfd1d19)
@pytorchbot
Copy link
Collaborator

Cherry picking #130246

The cherry pick PR is at #133597 and it is linked with issue #130242. The following tracker issues are updated:

Details for Dev Infra team Raised by workflow job

atalman pushed a commit that referenced this pull request Aug 21, 2024
Fix warning when pickle.load torch.Storage (#130246)

Fixes #130242

Since `torch.save` does not use pickle for storages, the `torch.load` in `_load_from_bytes` should not ever be called when `torch.load`-ing a checkpoint. Setting weights_only=False explicitly in `_load_from_bytes` to avoid the weights_only warning when using the pickle module

Pull Request resolved: #130246
Approved by: https://github.com/albanD

(cherry picked from commit dfd1d19)

Co-authored-by: Mikayla Gawarecki <mikaylagawarecki@gmail.com>
@github-actions github-actions bot deleted the gh/mikaylagawarecki/230/head branch September 17, 2024 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk Trigger trunk jobs on your pull request Merged release notes: python_frontend python frontend release notes category topic: bug fixes topic category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants