Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Sana]add 2K related model for Sana #10322

Merged
merged 1 commit into from
Dec 20, 2024
Merged

Conversation

lawrence-cj
Copy link
Contributor

What does this PR do?

We release 2K model of Sana. So I add the ckpt and 2K related code into the conversion script.

Cc: @a-r-r-o-w @sayakpaul

Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @lawrence-cj !

@yiyixuxu yiyixuxu merged commit a6288a5 into huggingface:main Dec 20, 2024
8 checks passed
danhipke pushed a commit to danhipke/diffusers that referenced this pull request Dec 20, 2024
@vladmandic
Copy link
Contributor

thanks @lawrence-cj! a bit off-topic, do you have plans for img2img sana pipeline?

@lawrence-cj
Copy link
Contributor Author

lawrence-cj commented Dec 21, 2024

We do. Will release controllnet code and model later. Any suggestion on other img2img pipeline?
@vladmandic

@vladmandic
Copy link
Contributor

i was thinking about having pure SanaImg2ImgPipeline using existing model,
should be possible by encoding input image and then using it while initializing latents in prepare_latents?
but of course controlnet is more than welcome!

@lawrence-cj
Copy link
Contributor Author

OK, I think it's ok to have a SanaImg2ImgPipeline with the current model. Just my bandwidth has been full recently. If you or anyone else is willing to help with the development, I will try my best to help.
@vladmandic

Foundsheep pushed a commit to Foundsheep/diffusers that referenced this pull request Dec 23, 2024
sayakpaul pushed a commit that referenced this pull request Dec 23, 2024
DN6 added a commit that referenced this pull request Jan 10, 2025
…ve load performance on network mounts (#10305)

* Add no_mmap arg.

* Fix arg parsing.

* Update another method to force no mmap.

* logging

* logging2

* propagate no_mmap

* logging3

* propagate no_mmap

* logging4

* fix open call

* clean up logging

* cleanup

* fix missing arg

* update logging and comments

* Rename to disable_mmap and update other references.

* [Docs] Update ltx_video.md to remove generator from `from_pretrained()` (#10316)

Update ltx_video.md to remove generator from `from_pretrained()`

* docs: fix a mistake in docstring (#10319)

Update pipeline_hunyuan_video.py

docs: fix a mistake

* [BUG FIX] [Stable Audio Pipeline] Resolve torch.Tensor.new_zeros() TypeError in function prepare_latents caused by audio_vae_length (#10306)

[BUG FIX] [Stable Audio Pipeline] TypeError: new_zeros(): argument 'size' failed to unpack the object at pos 3 with error "type must be tuple of ints,but got float"

torch.Tensor.new_zeros() takes a single argument size (int...) – a list, tuple, or torch.Size of integers defining the shape of the output tensor.

in function prepare_latents:
audio_vae_length = self.transformer.config.sample_size * self.vae.hop_length
audio_shape = (batch_size // num_waveforms_per_prompt, audio_channels, audio_vae_length)
...
audio = initial_audio_waveforms.new_zeros(audio_shape)

audio_vae_length evaluates to float because self.transformer.config.sample_size returns a float

Co-authored-by: hlky <hlky@hlky.ac>

* [docs] Fix quantization links (#10323)

Update overview.md

* [Sana]add 2K related model for Sana (#10322)

add 2K related model for Sana

* Update src/diffusers/loaders/single_file_model.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* Update src/diffusers/loaders/single_file.py

Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>

* make style

---------

Co-authored-by: hlky <hlky@hlky.ac>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Leojc <liao_junchao@outlook.com>
Co-authored-by: Aditya Raj <syntaxticsugr@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Junsong Chen <cjs1020440147@icloud.com>
Co-authored-by: Dhruv Nair <dhruv.nair@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants