KEMBAR78
[MPS] Fix large copy by pytorchbot · Pull Request #126104 · pytorch/pytorch · GitHub
Skip to content

[MPS] Fix large copy #126104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Parent: [Cherry-pick][DCP][AC] Add test for apply AC with FSDP1 (#126935)
merged 1 commit into from
May 13, 2024
Merged

[MPS] Fix large copy #126104

merged 1 commit into from
May 13, 2024

Conversation

@pytorchbot
Copy link
Collaborator

By slicing copyFromBuffer:sourceOffset:toBuffer:destinationOffset:size: into 2Gb chunks

Add regression test, but limit it to machines with 12Gb of RAM or more, and MacOS 14+, as on MacOS 13 attempt to alloc 4Gb tensor fails with:

/AppleInternal/Library/BuildRoots/c651a45f-806e-11ed-a221-7ef33c48bc85/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:724: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'

Fixes #124335

@malfet @pytorchbot
[MPS] Fix large copy (#124635)
fba7509 
By slicing `copyFromBuffer:sourceOffset:toBuffer:destinationOffset:size:` into 2Gb chunks

Add regression test, but limit it to machines with 12Gb of RAM or more, and MacOS 14+, as on MacOS 13 attempt to alloc 4Gb tensor fails with:
```
/AppleInternal/Library/BuildRoots/c651a45f-806e-11ed-a221-7ef33c48bc85/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:724: failed assertion `[MPSNDArray initWithDevice:descriptor:] Error: total bytes of NDArray > 2**32'
```

Fixes #124335

Pull Request resolved: #124635
Approved by: https://github.com/kulinseth

(cherry picked from commit abf3f90)
@pytorchbot pytorchbot mentioned this pull request May 13, 2024
Closed
@pytorch-bot
Copy link

pytorch-bot bot commented May 13, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126104

Note: Links to docs will display an error until the docs builds have been completed.

❌ 47 New Failures

As of commit fba7509 with merge base 86a2d67 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added ciflow/mps Run MPS tests (subset of trunk) release notes: mps Release notes category labels May 13, 2024
@atalman atalman mentioned this pull request May 13, 2024
Closed
@huydhn
Copy link
Contributor

huydhn commented May 13, 2024

Merging now as MPS tests are passing

@huydhn huydhn merged commit 768e4b9 into release/2.3 May 13, 2024
@github-actions github-actions bot deleted the cherry-pick-124635-by-pytorch_bot_bot_ branch June 13, 2024 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@huydhn huydhn huydhn approved these changes

@malfet malfet malfet approved these changes

@kulinseth kulinseth Awaiting requested review from kulinseth kulinseth is a code owner

Assignees

No one assigned

Labels

ciflow/mps Run MPS tests (subset of trunk) open source release notes: mps Release notes category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pytorchbot @huydhn @malfet