Using GFPGAN with dream causes black cast over face

[cvar66]

Create an image with a realistic face
Create it again but now with using -G1 -U2
GFPGAN will add a dark cast to the face that makes the colors and shadows look blackish and very muddy

When I run the original image through GFPGAN directly with default settings, the results are considerably better.

[blessedcoolant]

Can you show me some previews? Also can you try to use the value -G0.99 instead of 1 and tell me if the same problem persists.

[cvar66]

original

using -G1 -U2 ( 0.99 made no difference)

Using GFPGAN directly

After doing the comparison it seems it not just color that is affected. Also if the picture is really well lit and bright it is much less noticeable.

[blessedcoolant]

This discoloration seemed to have been happening even without today's changes. Let me look and see why that is the case.

[blessedcoolant]

Fixed it. This is not an issue with this repo but rather the utils function provided by GFPGAN in itself.

You can fix this by replacing utils.py in the GFPGAN/gfpgan folder with the file below.

https://github.com/blessedcoolant/files/blob/ed4aa1f20384cef1ed32aa7a921d021a02974310/utils.py

---

If you want to make the changes yourself, replace this part of the code.

for cropped_face in self.face_helper.cropped_faces:
            # prepare data
            cropped_face_t = img2tensor(cropped_face / 255., bgr2rgb=False, float32=True)
            normalize(cropped_face_t, (0.5, 0.5, 0.5), (0.5, 0.5, 0.5), inplace=True)
            cropped_face_t = cropped_face_t.unsqueeze(0).to(self.device)

            try:
                output = self.gfpgan(cropped_face_t, return_rgb=True)[0]
                # convert to image
                restored_face = tensor2img(output.squeeze(0), rgb2bgr=False, min_max=(-1, 1))
            except RuntimeError as error:
                print(f'\tFailed inference for GFPGAN: {error}.')
                restored_face = cropped_face

            restored_face = restored_face.astype('uint8')
            self.face_helper.add_restored_face(restored_face)

Change both bgr2rgb from True to False and return rgb in the output from False to True

cropped_face_t = img2tensor(cropped_face / 255., bgr2rgb=False, float32=True)
output = self.gfpgan(cropped_face_t, return_rgb=True)[0]
restored_face = tensor2img(output.squeeze(0), rgb2bgr=False, min_max=(-1, 1))

That seems to solve the issue of the discoloration.

[cvar66]

This didn't work for me. I used the file in your link. When using the command line it looks the same. When using the web version the effects are calculated but are just not applied anymore.

[blessedcoolant]

This didn't work for me. I used the file in your link. When using the command line it looks the same. When using the web version the effects are calculated but are just not applied anymore.

I've generated more images. The discoloration definitely is fixed on my end with the above changes. I'll wait on others to tell me their experience before exploration more solutions if need be.

[cvar66]

hmm OK, I probably mucked something up. I will try it again. Thanks!

[Ratinod]

I'll wait on others to tell me their experience before exploration more solutions if need be.

Yes. This solves the problem, but breaks the standalone GFPGAN work.

[cvar66]

OK so this is still not working for me. I think maybe because I initially installed GFPGAN in the stable diffusion folder. This worked until I tried to use it with dream so I just copied the whole GFPGAN folder into my root directly so the links would work. I'm still pretty new to this. Only reason I can think why this isn't working for me.

I also tried replacing utils.py in both locations

[blessedcoolant]

OK so this is still not working for me. I think maybe because I initially installed GFPGAN in the stable diffusion folder. This worked until I tried to use it with dream so I just copied the whole GFPGAN folder into my root directly so the links would work. I'm still pretty new to this. Only reason I can think why this isn't working for me.

I also tried replacing utils.py in both locations

Are you using the 1.3 model of GFPGAN?

[cvar66]

Still doesn't work. Also I am using 1.4

[blessedcoolant]

OK so I have more info. I was using k_euler_a with this issue. I tried switching to the default sampler and it works like you said. Then I went back to k_euler_a and the problem is back. This is on the command line. When I use the --web option there are no effects applied to the image after it calculates the face fix. Also I am using 1.4

There's no 1.4 afaik? The latest version of the model if 1.3 I think.

The above image I posted is k_euler_a and I've checked with other models. It's fixes it. Can you share the exact prompt with the seed you are generating? I can try that and see if I get discoloration on my end.

The web UI is being updated. There'll be a patch for that. We made a lot of changes to how GFPGAN and scaling works today.

[cvar66]

Sorry it's late here. Yes I am using 1.3. I doubled checked the default sampler images and the problem still exists, I just didn't notice because its a bright picture and the cast is not strong.

[cvar66]

"Megan Fox low light" -s20 -b1 -W512 -H512 -C7.5 -mk_euler_a -G1.0 -S2352774659

[blessedcoolant]

Here's my result. Same prompt. Using the fix I mentioned above.

[cvar66]

OK, so the problem must be on my end somehow. I'll eventually figure it out. I appreciate all your help! Thank you!

[cvar66]

I just wanted to add the solution to my issue. I am using Miniconda in Windows. The reason why the fix did not work for me was because dream was calling utils.py from C:\Users\username\ .conda\envs\ldm\Lib\site-packages\gfpgan not GFPGAN/gfpgan. After editing the file there everything works perfect now!

[lstein]

I checked and I have this problem too (using GFPGAN 1.3.4). The recommended solution worked as well, but I again had to replace utils.py in site-packages.

I would have thought that either running "python setup.py develop" or "pip install -e ." in the GFPGAN directory would copy the changed utils.py in the source directory into the site-packes directory. However this is not the case. Is there a recipe for patching and installing libraries the right way?

[thelamedia]

Any luck on a patch for this yet? In docker images it's a major pain to get to this file manually.

[lstein]

I still don't have an explanation of why changing gfpgan/utils.py in the GFPGAN directory and then doing pip install -e . isn't updating in the site-packages directory. Happy if someone will point in the right direction.

I have sent a PR to the GFPGAN maintainers with the fix. However, this package hasn't been updated in several months and they may not respond quickly.

[Ratinod]

This fix still fixes the work of GFPGAN through dream AND breaks (black cast over face) the stand-alone work of GFPGAN? Has this fix been tested on a standalone GFPGAN? At the moment I'm using a .bat file for easy file replacement if I need to use just GFPGAN or GFPGAN through dream.
I am using Anaconda and C:\Users\username\ .conda has only environments.txt file.

[nh256]

[nh256]