KEMBAR78
Flux with controlnet and image2image support · Issue #9158 · huggingface/diffusers · GitHub
Skip to content

Flux with controlnet and image2image support #9158

@BasimBashir

Description

@BasimBashir

Is your feature request related to a problem? Please describe.
I'm always frustrated when I can't leverage the powerful capabilities of Flux ControlNet with the img2img pipeline in the diffusers library. Flux ControlNet has recently emerged as a potential new state-of-the-art model for img2img tasks, and the lack of integration with diffusers limits the ability to explore and utilize this model within the ecosystem.

Describe the solution you'd like.
I would love to see the diffusers package include support for a FluxControlNetImg2ImgPipeline, similar to the existing StableDiffusionXLControlNetImg2ImgPipeline. This addition would enable users to fully utilize the Flux ControlNet model for img2img tasks, allowing for more flexibility and innovation in generating images.

Describe alternatives you've considered.
As an alternative, I've considered manually integrating Flux ControlNet with the existing pipelines, but this approach is complex and not as seamless or efficient as having native support within the diffusers library. Another option is waiting for updates or third-party implementations, but having official support would be the most reliable and user-friendly solution.

Additional context.
Flux ControlNet was released recently and is quickly becoming a significant player in the AI image generation space. Integrating this with diffusers would be a huge benefit to the community and help keep the library at the forefront of AI advancements. Thank you for considering this feature!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions