Flux Image-To-Image pipeline

[dsocek]

FLUX Image-to-Image Pipeline

List of Contributions in this PR

• FLUX.1 image-to-image pipeline Gaudi added
• Added support for FLUX.1 inference in examples/stable-diffusion/image_to_image_generation.py sample
• Updated documentation with FLUX.1 image-to-image examples/command-lines
• Added FLUX.1 image-to-image CI tests

Quality Check

Quality of output image shows good guidance from the input image and accompanying prompt.

Input image:

Input prompt:

"cat wizard, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney, 8k"

Simple inference script:

import torch
from diffusers.utils import load_image
from optimum.habana.diffusers import GaudiFluxImg2ImgPipeline

pipe = GaudiFluxImg2ImgPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    torch_dtype=torch.bfloat16,
    use_habana=True,
    use_hpu_graphs=True,
    gaudi_config="Habana/stable-diffusion",
)

url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png"
init_image = load_image(url).resize((1024, 1024))

prompt = "cat wizard, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney, 8k"

image = pipe(
    prompt=prompt, image=init_image, num_inference_steps=30, strength=0.9, guidance_scale=3.5
).images[0]
image.save("flux_img2img.png")

Output image:

Performance

Performance matches the Gaudi FLUX.1 text-to-image pipeline performance.

Tests

The added CI tests for Gaudi FLUX.1 image-to-image pipeline PASSED:

python -m pytest tests/test_diffusers.py -v -s -k "GaudiFluxImg2ImgPipelineTester"
...
============ 2 passed, 162 deselected, 1 warning in 11.50s =============

[imangohari1]

Hi Daniel,
Great work! Here are some basic suggestions. I am still testing some of img-to-image pipeline cases.
The main thing right now is to make sure that the readme changes are giving somewhat the high(est) perf for the flux. can you make sure we are aiming for that?

[dsocek]

@imangohari1 rebased

[imangohari1]

I have tested the REAME and added diffuser tests and they are all good.
Still running some diffuser tests. Overal should be good. @dsocek @regisss

[imangohari1]

LGMT!
All the fast and slow diffuser tests passed.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========== 150 passed, 15 skipped, 324 warnings in 5169.22s (1:26:09) ==========

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[regisss]

LGTM!

Can you just merge the updated main branch into your branch so that it triggers the whole CI please? I pushed some changes in the last couple of days because the CI runs on different Github runners now.

[dsocek]

LGTM!

Can you just merge the updated main branch into your branch so that it triggers the whole CI please? I pushed some changes in the last couple of days because the CI runs on different Github runners now.

Done