Skip to content

Latest commit

 

History

History
111 lines (88 loc) · 5.52 KB

README.md

File metadata and controls

111 lines (88 loc) · 5.52 KB

DeS3_Deshadow (AAAI'2024)

Introduction

This is an implementation of DeS3: Attention-driven Self and Soft Shadow Removal using ViT Similarity and Color Convergence

git clone https://github.com/jinyeying/DeS3_Deshadow.git
cd DeS3_Deshadow/

1. Datasets

  1. SRD Train|BaiduPan, Test. Shadow Masks

  2. AISTD|ISTD+ [link]

  3. LRSS: Soft Shadow Dataset [link]
    The LRSS dataset contains 134 shadow images (62 pairs of shadow and shadow-free images).
    We use 34 pairs for testing and 100 shadow images for training. For shadow-free training images, 28 from LRSS and 72 randomly selected from the USR dataset.

    [Dropbox] [BaiduPan (code:t9c7)]

  4. USR: Unpaired Shadow Removal Dataset [link]

  5. UCF, UIUC: Self Shadow [link]

2. SRD Dataset Results:

[Dropbox] [BaiduPan(code:blk7)]

SRD Dataset Evaluation

  1. set the paths of the shadow removal result and the dataset in evaluation/demo_SRD_RMSE.m and then run it.
demo_SRD_RMSE.m

Get the RMSE from Table 1 in the main paper on the SRD (size: 256x256):

Method Training Shadow Non-Shadow ALL
DeS3 Paired 5.88 2.83 3.72
  1. set the paths of the shadow removal result and the dataset in evaluation/evaluate_SRD_PSNR_SSIM.m and then run it.
evaluate_SRD_PSNR_SSIM.m

Get the PSNR & SSIM from Table 1 in the main paper on the SRD (size: 256x256):

PSNR PSNR PSNR SSIM SSIM SSIM
Method Training Shadow Non-Shadow ALL Shadow Non-Shadow ALL
DeS3 Paired 37.45 38.12 34.11 0.984 0.988 0.968

3. AISTD Dataset Results:

[Dropbox] [BaiduPan(code:blk7)]

AISTD Dataset Train, Test

  1. modify the path in

    DeS3_Deshadow/datasets/aistdshadow.py

    Line 30 in c294476

    filelist='/home1/yeying/DeS3_Deshadow/aistdtest.txt',

    DeS3_Deshadow/configs/AISTDshadow.yml

    Line 6 in c294476

    data_dir: "/home1/yeying/WeatherDiffusion_shadow/data/AISTD/"
  2. download the AISTD checkpoint [Dropbox] | [BaiduPan(code:aistd)]
CUDA_VISIBLE_DEVICES=1,2 python train_aistd.py --config 'AISTDshadow.yml' --resume '/home1/yeying/DeS3_Deshadow/ckpts/AISTDShadow_ddpm.pth.tar'

CUDA_VISIBLE_DEVICES=1 python eval_aistd.py --config 'AISTDshadow.yml' --resume '/home1/yeying/DeS3_Deshadow/ckpts/AISTDShadow_ddpm.pth.tar'

AISTD Dataset Evaluation

  1. set the paths of the shadow removal result and the dataset in evaluation/demo_AISTD_RMSE.m and then run it.
demo_AISTD_RMSE.m

Get the RMSE on the AISTD (size: 256x256):

Method Training Shadow Non-Shadow ALL
DeS3 Paired 6.56 3.40 3.94
  1. set the paths of the shadow removal result and the dataset in evaluation/evaluate_AISTD_PSNR_SSIM.m and then run it.
evaluate_AISTD_PSNR_SSIM.m

Get the PSNR & SSIM on the ISTD (size: 256x256):

PSNR PSNR PSNR SSIM SSIM SSIM
Method Training Shadow Non-Shadow ALL Shadow Non-Shadow ALL
DeS3 Paired 36.49 34.70 31.38 0.989 0.972 0.958

Acknowledgments

Code is implemented based WeatherDiffusion, we would like to thank them.

License

The code and models in this repository are licensed under the MIT License for academic and other non-commercial uses.
For commercial use of the code and models, separate commercial licensing is available. Please contact:

Citations

If this work is useful for your research, please cite our paper.

@inproceedings{jin2024des3,
  title={DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity},
  author={Jin, Yeying and Ye, Wei and Yang, Wenhan and Yuan, Yuan and Tan, Robby T},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={38},
  number={3},
  pages={2634--2642},
  year={2024}
}