LGTM

Hi @sayakpaul . Would you please review this PR? Thanks!

Hi @sayakpaul . I have fixed your comments. For mixed-precision, I am not sure because we have torch_dtype and weight_dtype here. In my view, the mix-precision should work on lora module but not SD based model because the weight_dtype has already been converted to the mix-precision.

Just one comment and we should be good to go.

Have you verified if it works effectively? If so, could you share some results?

For cuda, it just enabled the DDP finetune and had no impact on the result as previously.

The result of Intel Xeon CPU is:

Sorry, I cannot release the latency performance data cause it is not allowed by Intel.

For cuda, it just enabled the DDP finetune and had no impact on the result as previously.

What do you mean by this? I don't see the dog from the training set appear here. Am I missing something?

For cuda, it just enabled the DDP finetune and had no impact on the result as previously.

What do you mean by this? I don't see the dog from the training set appear here. Am I missing something?

Sorry for I didn't upload the cuda result image successfully, I will run again to get the result and give you feedback ASAP.

Hi @sayakpaul , here is the cuda result runs on 2*A100 cards

And how about the XPU result?

And how about the XPU result?

Just got the XPU result with 2 DDP across 2 cards:

The XPU needs to tune the args to get a high-quality result.

Nice, this is good.

The failed test seems unrelated to my changes. Please let me know what needs to be changed or request any other reviewers before merging. Thanks!

Thanks much!