Add back ControlNet multi-inputs upload tab

[huchenlei]

Description

Closes #98

IP-Adapter face_id and photomaker verified to be working with multi-inputs.
InstantID not working with multi-inputs now. @lllyasviel PTAL if we can somehow make multi-inputs work for InstantID. In sd-webui-controlnet, the approach is approximation via averaging the insightface embedding.

Screenshots/videos:

Checklist:

• I have read contributing wiki page
• I have performed a self-review of my own code
• My code follows the style guidelines
• My code passes tests

[strawberrymelonpanda]

Thanks!

[lllyasviel]

this will be taken a look later since it needs more considerations

[StellarBeing25]

this will be taken a look later since it needs more considerations

Please move the Batch Folder and Batch Upload together under a Batch tab and restore the Multi Input tab to avoid confusion. It is very important for IP Adapters like FaceID Portrait which requires 5 input images.

[a3nima]

this will be taken a look later since it needs more considerations

Please move the Batch Folder and Batch Upload together under a Batch tab and restore the Multi Input tab to avoid confusion. It is very important for IP Adapters like FaceID Portrait which requires 5 input images.

also my (and many others) ip adapter face id workflow requires a set of image as base. maybe show multi-input only, where it actually works -if that's the problem?

[strawberrymelonpanda]

I've been using the PR - it works great, exactly as expected. Please consider merging this soon, it's very useful for IP Adapter workflows.

[saunderez]

Like to also report this PR is working great in conjunction with Face ID IP Adaptor, greatly improved likeness using multiple source images.

[ScionoicS]

Was trying to use instant id today when i noticed I was unable to use multiple image inputs like I can on the regular controlnet extension.

Why remove?

[strawberrymelonpanda]

@sohamaa in #344 raises a valid point: The "Multiple Images" tab is shown in the screenshots of the README.

[CRCODE22]

I added it back to my own installation.

Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

[JaB43a]

I added it back to my own installation.

Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.

I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.

Thank you

[saunderez]

I added it back to my own installation.
Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.

I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.

Thank you

Everything you need to know is at the link below.

https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

[CRCODE22]

I added it back to my own installation.
Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.

I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.

Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

[JaB43a]

I added it back to my own installation.
Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.
I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.
Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

I have tried it, but it doesn't work. At least in my case.
Although I upload several images, the end result is that it only takes one of them (testing with the same config and seed, the results are the same, the same face appears)

Thank you anyway :)

[CRCODE22]

I added it back to my own installation.
Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.
I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.
Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

I have tried it, but it doesn't work. At least in my case. Although I upload several images, the end result is that it only takes one of them (testing with the same config and seed, the results are the same, the same face appears)

Thank you anyway :)

When using photomaker are you making sure your positive prompt does not exceed 75 tokens? I noticed once you go past that photomaker will not generate faces.

Here it does work:

Moving model(s) has taken 0.68 seconds
2024-02-27 04:00:06,151 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched.
Moving model(s) skipped. Cleanup has taken 0.00 seconds
2024-02-27 04:00:06,422 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched.
Moving model(s) skipped. Cleanup has taken 0.00 seconds
2024-02-27 04:00:06,620 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched.
Moving model(s) skipped. Cleanup has taken 0.00 seconds
2024-02-27 04:00:06,833 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched.
Moving model(s) skipped. Cleanup has taken 0.00 seconds
2024-02-27 04:00:07,188 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched.
Moving model(s) skipped. Cleanup has taken 0.00 seconds
2024-02-27 04:00:07,650 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched.
To load target model SDXL

[JaB43a]

I added it back to my own installation.
Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.
I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.
Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

I have tried it, but it doesn't work. At least in my case. Although I upload several images, the end result is that it only takes one of them (testing with the same config and seed, the results are the same, the same face appears)
Thank you anyway :)

When using photomaker are you making sure your positive prompt does not exceed 75 tokens? I noticed once you go past that photomaker will not generate faces.

Here it does work:

Moving model(s) has taken 0.68 seconds 2024-02-27 04:00:06,151 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,422 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,620 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,833 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,188 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,650 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. To load target model SDXL

I haven't used Photomaker much, the one I use is InstantID, and it is with InstantID that it doesn't work.

[banjaminicc]

Please consider the merge, it is extremely useful for FaceID ipadapter and InstantID

I added it back to my own installation.
Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.
I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.
Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

I have tried it, but it doesn't work. At least in my case. Although I upload several images, the end result is that it only takes one of them (testing with the same config and seed, the results are the same, the same face appears)
Thank you anyway :)

When using photomaker are you making sure your positive prompt does not exceed 75 tokens? I noticed once you go past that photomaker will not generate faces.
Here it does work:
Moving model(s) has taken 0.68 seconds 2024-02-27 04:00:06,151 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,422 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,620 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:06,833 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,188 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. Moving model(s) skipped. Cleanup has taken 0.00 seconds 2024-02-27 04:00:07,650 - ControlNet - INFO - ControlNet Method ClipVision (Photomaker) patched. To load target model SDXL

I haven't used Photomaker much, the one I use is InstantID, and it is with InstantID that it doesn't work.

It is working for me with InstantID though

[webinksupport]

Multi-image tab doesn't seem to work on inpainting (possibly all of img2img). The tab shows, but it doesn't have any effect on the image.

[a3nima]

this will be taken a look later since it needs more considerations

why is it not implented yet?

[lhucklen]

Please implement, this is such a good quality of life setting

[PeanutButterSkeletonCrew]

I would also like to see this re-implemented. I'm very surprised it was removed in the first place.

[a3nima]

I added it back to my own installation.
Thank you @huchenlei for making the changes it is as simple as overwriting 5 files.

I'm very interested in knowing how you did it. What files did you change and what did you change about each of them.
I also join the request that this option be officially present again, it is very useful for me to generate certain characters that I can only generate with different images.
Thank you

These 5 files from here https://github.com/lllyasviel/stable-diffusion-webui-forge/pull/264/files

Overwrite the files in your local installation and it will work at least when I tested it 5 days ago and have not updated Forge since.

While this still works, it would be nice to have the multi upload function AND the "use mask" function working together. I have to switch to single image to get the mask drawing/use mask feature atm. which makes them unusable together.

[lhucklen]

Any updates on this? @lllyasviel

[a3nima]

Btw I've found that with the code provided by @huchenlei I do get an error, when trying to use controlnet inpaint. multi-inputs for ip-adapter still works though.

[gohan2091]

I really need multiple inputs for Face ID. Really hope this can be added back.

[strawberrymelonpanda]

huchenlei merged commit 61a2a9d into lllyasviel:dev2

I've been asking for this pretty much since Forge launched, ha. Thanks @huchenlei!👍

[GatorsUF]

huchenlei merged commit 61a2a9d into lllyasviel:dev2

I've been asking for this pretty much since Forge launched, ha. Thanks @huchenlei!👍

I just tried it on dev2 branch and I can't get controlnet/ipadapter to work with multi-inputs. In fact I can't get ipadapter to work at all on the dev2 branch even with a single image input.

[tealytee]

i get this error

*** Error loading script: controlnet.py
Traceback (most recent call last):
File "G:\SD F\sd forge\webui\modules\scripts.py", line 525, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "G:\SD F\sd forge\webui\modules\script_loading.py", line 13, in load_module
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "G:\SD F\sd forge\webui\extensions-builtin\sd_forge_controlnet1\scripts\controlnet.py", line 15, in
from lib_controlnet.utils import (
File "G:\SD F\sd forge\webui\extensions-builtin\sd_forge_controlnet1\lib_controlnet\utils.py", line 9, in
from modules_forge.forge_util import HWC3
ModuleNotFoundError: No module named 'modules_forge.forge_util'

[NeiroNext]

The error is still present, where can I find out the status of its fix?

[Desu21212]

Hello. This was a very good thing to have. It's better than an actual lora. SD. 1.5 just wins over forge in face ID department because of the the multi-upload. You have to add it back please!

[a3nima]

Hello. This was a very good thing to have. It's better than an actual lora. SD. 1.5 just wins over forge in face ID department because of the the multi-upload. You have to add it back please!

I dont think Illyasviel will add it back, because it's somehow not a technically clean method. But you can use 2 Controlnets with 1 image each and adjust the weights to "mix" faces in forge.

[Esperontre]

I dont think Illyasviel will add it back, because it's somehow not a technically clean method. But you can use 2 Controlnets with 1 image each and adjust the weights to "mix" faces in forge.

Is there a way for more than 3 images? Multi-controlnet input was useful for my end as I used IP-Adapter to toss in 5 images of armor in order to produce quick armor combinations, rather than individually changing the image per controlnet which can cause time to pile up if multiple combos are being tested.

[a3nima]

yeah, there are other reasons (the one you called and also (probably?!) less cpu/ram ressources needed with just 1controlnet active) for adding it back somehow. I've had added it back with some change of code that was provided in the github posts, but I dont think that works anymore.