🚀 The feature, motivation and pitch

Currently, the SequentialLR scheduler doesn't work if one of the schedulers that you pass it is a ChainedScheduler. This is because the ChainedScheduler does not have a last_epoch attribute, so when the SequentialLR initialisation tries to ""Undo" the step performed by other schedulers":

        # "Undo" the step performed by other schedulers
        for scheduler in self._schedulers:
            scheduler.last_epoch -= 1

it fails when it gets to a ChainedScheduler.

It would be good if the SequentialLR could actually take in ChainedSchedulers.

Alternatives

Currently, I have created my own version of the SequentialLR that allows ChainedSchedulers. This requires a fairly minimal change to the SequentialLR class. In the SequentialLR's __init__, I change the lines:

        # "Undo" the step performed by other schedulers
        for scheduler in self._schedulers:
            scheduler.last_epoch -= 1

to be:

        # "Undo" the step performed by other schedulers
        self.recursive_undo()

and add the a recursive_undo method to the class, which checks if the scheduler itself contains a list of _schedulers or otherwise a last_epoch attribute, and resets the last_epoch when encountered:

    def recursive_undo(self, sched=None):
        # recursively undo any step performed by the initialisation of
        # schedulers
        scheds = self if sched is None else sched

        if hasattr(scheds, "_schedulers"):
            for s in scheds._schedulers:
                self.recursive_undo(s)
        elif hasattr(scheds, "last_epoch"):
            scheds.last_epoch -= 1

I can add a PR with this request if it is useful.

Additional context

No response