Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable warm up for VAE #300

Merged
merged 1 commit into from
Oct 11, 2024
Merged

Enable warm up for VAE #300

merged 1 commit into from
Oct 11, 2024

Conversation

gty111
Copy link
Contributor

@gty111 gty111 commented Oct 10, 2024

This PR enables warm up run for VAE. The insight is that warm up can accelerate the execution of VAE.

The tested model is Flux. The output is two pic of 2048x1024. I use four L20 card and set ulysses degree to 4.

parallel VAE warm up elapsed time memory
N N 7.27s 42.34 GB
N Y 7.02s 42.34 GB
Y N 6.99s 36.85 GB
Y Y 6.31s 36.85 GB 
The first Patch vae: [elapsed_time: 0.98 sec, peak_memory: 36.853204992 GB]
The second Patch vae: [elapsed_time: 0.37 sec, peak_memory: 36.852156416 GB]

I guess the overhead of warm up mainly comes from tensor allocation.

Also fix compatibility issue between parallel vae and naive forward.

feifeibear
feifeibear approved these changes Oct 11, 2024
View reviewed changes
Copy link
Collaborator

@feifeibear feifeibear left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@feifeibear feifeibear merged commit f9edb23 into xdit-project:main Oct 11, 2024
2 of 3 checks passed
@gty111 gty111 deleted the warmup branch October 11, 2024 02:55
feifeibear pushed a commit to feifeibear/xDiT that referenced this pull request Oct 25, 2024
@gty111 @feifeibear
Fix compatibility issue between parallel vae and naive forward; Enabl…
c63d09c 
…e warmup for vae (xdit-project#300)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@feifeibear feifeibear feifeibear approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@gty111 @feifeibear