Poor results or wrong useage of GLEAN in face images.

[ewrfcas]

Hi, I have tried GLEAN in mmediting for 64->1024 face SR. But the generated results are very poor.
My command is python restoration_demo.py configs/restorers/glean/glean_ffhq_16x.py workdirs/glean_ffhq_16x_20210527-61a3afad.pth tests/data/1009.png preds/1009.png --device 2

My input is 64x64 face image

and the output is

[ewrfcas]

Here is the log

[ckkelvinchan]

Hello, did you use bicubic downsampling kernel?

In addition, it seems that restoration_inference did not normalize the images from [-1, 1] back to [0, 1]. As a workaround, you can normalize the output in your backbone by using output = (output + 1) / 2.0 and see how the results look. I will see how to modify restoration_inference later.

[yzxing87]

Same issue here. In my case, the max value of output exceeds 1 and the min value is greater than 0.

[ckkelvinchan]

Hello, did you use bicubic downsampling kernel?

In addition, it seems that restoration_inference did not normalize the images from [-1, 1] back to [0, 1]. As a workaround, you can normalize the output in your backbone by using output = (output + 1) / 2.0 and see how the results look. I will see how to modify restoration_inference later.

I am sorry that it seems that this is not the problem. The code should have normalized it back to [0, 1].

I just tried two images in CelebA-HQ and they work fine. Could you please try the images in CelebA-HQ? Please note that MATLAB imresize should be used for downsampling.

[ckkelvinchan]

How about trying the images here? These two images work fine on my side.

[yzxing87]

How about trying the images here? These two images work fine on my side.

Hi, I have tried your images with command

python demo/restoration_demo.py configs/restorers/glean/glean_ffhq_16x.py \
    pretrain/glean_ffhq_16x_20210527-61a3afad.pth \
    00001.png \
    results/00001.png \
    --device 1

But the output is still poor.

Is it because of my wrong usage? Thank you

[ckkelvinchan]

I used this command:

python demo/restoration_demo.py configs/restorers/glean/glean_ffhq_16x.py https://download.openmmlab.com/mmediting/restorers/glean/glean_ffhq_16x_20210527-61a3afad.pth ./00001.png outputs/00001.png

Did you modify the codes?

Here are my results: results.zip

[ewrfcas]

Still fails with bicubic resizing and data rescaling to [0,1].
Here is the code in restoration_inference.py

Failed results.

[ckkelvinchan]

Still fails with bicubic resizing and data rescaling to [0,1].
Here is the code in restoration_inference.py

Failed results.

How about removing data['lq'] = (data(['lq'] + 1) / 2). When adding this line, the input to the network would be [0, 1], which is incorrect.

I used the latest version of MMEditing codes without any modifications.

[ewrfcas]

The output should be converted back to [-1,1]?

[ckkelvinchan]

The output should be converted back to [-1,1]?

The code will handle the conversion itself. There is no need to modify the code. The original code will do.

[ewrfcas]

Still fails with bicubic resizing and data rescaling to [0,1].
Here is the code in restoration_inference.py

Failed results.

How about removing data['lq'] = (data(['lq'] + 1) / 2). When adding this line, the input to the network would be [0, 1], which is incorrect.

I used the latest version of MMEditing codes without any modifications.

Sorry, I am confused for the input range. Which is the correct value range, [0,1] or [-1,1]?

[ckkelvinchan]

Still fails with bicubic resizing and data rescaling to [0,1].
Here is the code in restoration_inference.py

Failed results.

How about removing data['lq'] = (data(['lq'] + 1) / 2). When adding this line, the input to the network would be [0, 1], which is incorrect.
I used the latest version of MMEditing codes without any modifications.

Sorry, I am confused for the input range. Which is the correct value range, [0,1] or [-1,1]?

The inputs and outputs of GLEAN will be [-1, 1]. But you do not need to modify the code, as I have implemented the conversion.

[ewrfcas]

Still fails with bicubic resizing and data rescaling to [0,1].
Here is the code in restoration_inference.py

Failed results.

How about removing data['lq'] = (data(['lq'] + 1) / 2). When adding this line, the input to the network would be [0, 1], which is incorrect.
I used the latest version of MMEditing codes without any modifications.

Sorry, I am confused for the input range. Which is the correct value range, [0,1] or [-1,1]?

The inputs and outputs of GLEAN will be [-1, 1]. But you do not need to modify the code, as I have implemented the conversion.

Thanks. I use the original codes to test the image given above, but the result is still failed.

[ckkelvinchan]

I am able to reproduce your error just now. Please remove --device 1. I will investigate why and please remove it for the moment.

[yzxing87]

Thanks, removing device works! It seems that the data and the model are placed on different devices. If I set the CUDA_VISIBLE_DEVICES in the very beginning, the demo can also work fine when --device is set to other GPUs.

[ewrfcas]

I am able to reproduce your error just now. Please remove --device 1. I will investigate why and please remove it for the moment.

Nice，it works. Maybe the model failed to load weights properly with --device?

[ckkelvinchan]

It may be, thank you for letting me know :) I will look into the problem~

I will keep this issue open for the moment in case others encounter this problem. Thanks again.