🐛 Describe the bug

import torch

assert torch.is_grad_enabled()

@torch.set_grad_enabled(False)  # unexpectedly, this mutates the grad mode!
def inner_func(x):
    return x.sin()

assert torch.is_grad_enabled()  # AssertionError

Rather, I think that if set_grad_enabled were called on a function after initialization, it needs to reset to its exit state, prior to becoming a decorator ctx manager.

Versions

main

cc: @yanboliang

cc @ezyang @gchanan @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @lezcano @Varal7