KEMBAR78
Adding warning on isend about modifying after send by zasdfgbnm · Pull Request #61875 · pytorch/pytorch · GitHub
Skip to content

Conversation

@zasdfgbnm
Copy link
Collaborator

This is a standard limitation on communication collective libraries. For example:

https://www.open-mpi.org/doc/v4.0/man3/MPI_Isend.3.php

A nonblocking send call indicates that the system may start copying data out of the send buffer. The sender should not modify any part of the send buffer after a nonblocking send operation is called, until the send completes.

http://openucx.github.io/ucx/api/latest/html/group___u_c_p___c_o_m_m.html#ga8323878b60f426c630d4ff8996ede3cc

The user should not modify any part of the buffer after this operation is called, until the operation completes.

@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Jul 19, 2021

💊 CI failures summary and remediations

As of commit 7e6315d (more details on the Dr. CI page and at hud.pytorch.org/pr/61875):


💚 💚 Looks good so far! There are no failures yet. 💚 💚


Preview docs built from this PR

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Copy link
Contributor

@mrshenli mrshenli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for improving. Do we need this for other c10d operations, e.g., all_reduce, all_gather? Please feel free to submit followup PRs for those.

@facebook-github-bot
Copy link
Contributor

@mrshenli has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@mrshenli merged this pull request in 2d10302.

@zasdfgbnm zasdfgbnm deleted the zasdfgbnm-patch-1 branch July 29, 2021 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed Merged oncall: distributed Add this issue/PR to distributed oncall triage queue open source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants