Skip to content
This repository has been archived by the owner on Dec 14, 2023. It is now read-only.

Convert diffusers to original MS format #52

Merged
merged 17 commits into from
Apr 17, 2023

Conversation

kabachuha
Copy link
Contributor

@kabachuha kabachuha commented Apr 17, 2023

  • Convert the In/Out blocks
  • Convert the Text encoder
  • Convert Unet ResBlocks
  • Testing

closes #29

@kabachuha
Copy link
Contributor Author

kabachuha commented Apr 17, 2023

ModelStructure reference

Diffusers:

message.txt

ModelScope:

model_info.txt

@kabachuha kabachuha changed the title [WIP] Convert diffusers to original MS format Convert diffusers to original MS format Apr 17, 2023
@kabachuha kabachuha marked this pull request as ready for review April 17, 2023 12:57
@kabachuha
Copy link
Contributor Author

The conversion works!

python convert_diffusers_to_original_ms_text_to_video.py --model_path animov --checkpoint_path text2video_pytorch_model.pth --clip_checkpoint_path open_clip_pytorch_model.bin 
Initializing the conversion map
Converting the UNET
Saving UNET
Saving CLIP
Operation successfull

The only thing left is to load the model and see if it works

@kabachuha
Copy link
Contributor Author

Fixed all the keys mismatch

@kabachuha
Copy link
Contributor Author

All complete

alUntitled.mp4

@ExponentialML ExponentialML merged commit 9eac110 into ExponentialML:main Apr 17, 2023
@ExponentialML
Copy link
Owner

This was on my todo list, but you knocked it out of the park. Thank you very much!

@kabachuha kabachuha deleted the convert-from-diffusers branch April 17, 2023 22:23
@kabachuha kabachuha restored the convert-from-diffusers branch April 22, 2023 11:52
@sanatorios
Copy link

This is amazing but I'm not really sure about the needed process. Would love to convert strangeman3107's animov-512x fine-tuned model into MS format as well. How it could be possiblle?

@howardgriffin
Copy link

howardgriffin commented Aug 2, 2023

Hi, I have noticed that the newest version(v3.0.0) of 'convert_diffusers_to_original_ms_text_to_video.py' doesn't save clip_checkpoint.
What's more, this script doesn't save the configuration.json as https://github.com/kabachuha/sd-webui-text2video mentioned.
So,I cann't test my own trained model with sd_webui. How to solve my problem? @kabachuha @ExponentialML

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Using a trained model on webui
4 participants