When the input h0 to an instance of nn.GRU has a wrong shape it doesn’t raise an error (at least in some cases). For instance, if it expects a hidden state of shape (2, n, p) but receives one of shape (4, n, p) it will not complain. I didn't check with LSTM if the same happens.