README.md

DoubleDiffusion

Combining Heat Diffusion:fire: with Denoising Diffusion
for Generative Learning on 3D Meshes

Xuyang Wang🦘, Ziang Cheng🐧, Zhenyu Li🐫, Jiayu Yang🐧,
Haorui Ji🦘, Pan Ji🐧, Mehrtash Harandi🐨, Richard Hartley🦘, Hongdong Li🦘.
The Australian National University 🦘, Tencent XR Vision Labs 🐧, KAUST 🐫, Monash University 🐨

This repository is the official implementation of the paper **DoubleDiffusion**. The codes performs the denoising diffusion directly on the 3D meshes to generate texture or image on the input 3D Meshes. The idea is simple to follow. This paper and the associated codes adapt the idea of denoising image for the image generation on to the 3D meshes. Textured 3D meshes are generated through a denoising diffusion specifically designed for the 3D meshes.

Founctionbility

Infer - texture generation on 3D Meshes with our chekpoint

Input the 3D meshes, texture can be generated through the inferring pipeline.
Toy Checkpoint
This checkpoint is trained with different precisions of the 3D stanford bunny meshes, which are textured with Celeba-hq images.

Training - training to generate your own texture dataset

Repo's structure

DoubleDiffusion/
├── data/
    ├─── object/
    ├─── image/
    ├─── preprocessed/
├── experiments/ # for storing the output results
├── lib/models/diffusion_net
├── scripts/ # for storing the training and inference scripts
├── src
    ├─── baselines # mdf codes
    ├─── ssdm # double diffusion codes (aka. simple spectral diffusion model; aka. our model)
├── train.py # for training the model
├── infer.py # for inferring the model
├── infer_baseline.py # for inferring the baseline model
├── infer_edm.py # for inferring doubld diffusion with edm pipeline

Enviroment

conda create -n manifold python=3.8
conda activate manifold

# please make sure the torch version is compatible with the pytorch3d version, see https://github.com/facebookresearch/pytorch3d/blob/main/INSTALL.md for more details
pip install torch torchvision torchaudio
conda install pytorch3d -c pytorch3d
conda install numpy natsort tqdm networkx scikit-learn wandb
pip install potpourri3d pytorch3d pyvista robust-laplacian trimesh 
conda install -c conda-forge diffusers acceletor

Preprocessing

The dataset is preprocessed when first time running the 'train.py' code. The logic is that the laplacian matrix and its eigen products are computed for each mesh when initialized the ManifoldDataLoader for the first time. The preprocessed data is stored in the data/preprocessed folder.

Run the code

to run the training example on manfold bunny, please run the following command:

bash scripts/bunny/train_manifold_bunny.sh 

Speed

training speed

The speed is tested on a manifold bunny mesh with 52288 vertices.

GPU	Batch Size	Time per Epoch	Iteration per Time
2 * 3090	4	~37 mins	1.34 it/s

The speed is tested on a median bunny mesh with 2992 vertices.

GPU	Batch Size	Time per Epoch	Iteration per Time
2 * 3090	4	~2 mins	22.054 it/s

citation

If you find this work useful in your research, please consider citing:

@article{wang2025doublediffusion,
  title={DoubleDiffusion: Combining Heat Diffusion with Denoising Diffusion for Generative Learning on 3D Meshes},
  author={Wang, Xuyang and Cheng, Ziang and Li, Zhenyu and Yang, Jiayu and Ji, Haorui and Ji, Pan and Harandi, Mehrtash and Hartley, Richard and Li, Hongdong},
  journal={arXiv preprint arXiv:2501.03397},
  year={2025}
}