[ued] Investigate diffuser pipeline transformer recompilations due to different width/height #150702
Description
🐛 Describe the bug
From a user: huggingface/diffusers#10795 (comment)
- Changing the pipeline resolution triggers a recompilation, this happens with both dynamic=None and dynamic=True and resolution affecting the compilation is the most annoying issue right now.
We should look into the root cause of this. The user's testing script is here, but I didn't see any recompilation with the following minimal repro (dynamic=None also recompiles only once):
Repro:
import torch
from diffusers import (
AuraFlowPipeline,
GGUFQuantizationConfig,
AuraFlowTransformer2DModel,
)
transformer = AuraFlowTransformer2DModel.from_single_file(
"https://huggingface.co/city96/AuraFlow-v0.3-gguf/blob/main/aura_flow_0.3-Q2_K.gguf",
quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16),
torch_dtype=torch.bfloat16,
)
pipeline = AuraFlowPipeline.from_pretrained(
"fal/AuraFlow-v0.3",
torch_dtype=torch.bfloat16,
transformer=transformer,
).to("cuda")
pipeline.transformer = torch.compile(pipeline.transformer, fullgraph=True, dynamic=True)
pipeline("A cute pony", width=256, height=256, num_inference_steps=1)
pipeline("A cute pony", width=384, height=384, num_inference_steps=1)
pipeline("A cute pony", width=512, height=512, num_inference_steps=1)Error logs
No response
Versions
bb98749, python 3.11
cc @chauhang @penguinwu @ezyang @bobrenjc93 @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @amjames