KEMBAR78
[LoRA] fix peft state dict parsing by sayakpaul · Pull Request #10532 · huggingface/diffusers · GitHub
Skip to content

Conversation

@sayakpaul
Copy link
Member

@sayakpaul sayakpaul commented Jan 11, 2025

What does this PR do?

Fixes #9542 (comment).

Fixes #10599.

https://civitai.com/models/200251?modelVersionId=1081295 has both peft and non peft state dict inside it 🤷

Testing code:
from diffusers import FluxPipeline
import torch

pipeline = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16
).to("cuda")
pipeline.load_lora_weights(
    "sayakpaul/different-lora-from-civitai", weight_name="sharp_detailed_foot.safetensors"
)

prompt = """
UHD, 4k, ultra detailed, cinematic, a photograph of <lora:sharp detailed image (foot focus) v1.1:0.9> sharp detailed image of a female person standing on a black mat with their bare feet in front of camera, perfect image, perfect body, perfect anatomy, sharp image, detailed image, cinematic style, high quality photography, sharp detailed image style, solo, barefoot, feet, toes, black background, toenails, close-up, foot focus, muscular, veins, soles, facing camera, feet in front of camera, polished nails, closeup, feet focus, epic, beautiful lighting, inpsiring
"""

_ = pipeline(prompt, num_inference_steps=20, generator=torch.manual_seed(2810038592))

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link

@RIOFornium RIOFornium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sayakpaul
Copy link
Member Author

@DN6 a gentle ping.

@sayakpaul
Copy link
Member Author

Will first run the Flux tests and then merge the PR. I don't think the failing tests are related.

@sayakpaul
Copy link
Member Author

Have run all the integration tests related to Flux LoRA and Flux Control LoRA and they pass. Merging.

@sayakpaul sayakpaul merged commit 9f5ad1d into main Feb 10, 2025
13 of 15 checks passed
@sayakpaul sayakpaul deleted the rejig-peft-state-dict-kohya branch February 10, 2025 13:17
@sayakpaul
Copy link
Member Author

@RIOFornium, thanks for reporting the bugs and testing the PRs. Hope things are resolved now.

@RIOFornium
Copy link

@sayakpaul Thank you too!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[LoRA] "Incompatible keys detected" error when open popular LoRA models from civitai

4 participants