If a leaf tensor's .grad is None before backward, its AccumulateGrad function may steal a reference to the incoming gradient from whatever backward op produced it (instead of accumulated onto an existing .grad). @mruberry @ngimel and I are semi-confident AccumulateGrad functions and the autograd engine insert the right leaf stream syncs (such that ops following backward() can safely immediately use stolen .grads) but I should double check the code and PR a dedicated test. Filing so I don't forget.

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @lezcano @ngimel