Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maisi VAE notebook #1757

Merged
merged 43 commits into from
Jul 12, 2024
Merged

Maisi VAE notebook #1757

merged 43 commits into from
Jul 12, 2024

Conversation

Can-Zhao
Copy link
Contributor

@Can-Zhao Can-Zhao commented Jul 11, 2024

Fixes # .

Description

Add MAISI notebook and adjust it to a setting that can run on 16G V100.

Checks

  • Avoid including large-size files in the PR.
  • Clean up long text outputs from code cells in the notebook.
  • For security purposes, please check the contents and remove any sensitive info such as user names and private key.
  • Ensure (1) hyperlinks and markdown anchors are working (2) use relative paths for tutorial repo files (3) put figure and graphs in the ./figure folder
  • Notebook runs automatically ./runner.sh -t <path to .ipynb file>

root and others added 20 commits July 10, 2024 04:26
Signed-off-by: root <root@ipp1-3397.ipp1u1.colossus.nvidia.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

pre-commit-ci bot and others added 4 commits July 11, 2024 04:30
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
@Can-Zhao Can-Zhao marked this pull request as ready for review July 11, 2024 17:16
pre-commit-ci bot and others added 4 commits July 11, 2024 17:16
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Can-Zhao and others added 9 commits July 11, 2024 18:14
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
@mingxin-zheng
Copy link
Contributor

mingxin-zheng commented Jul 12, 2024

Hi @Can-Zhao , the CI indicates the something may be wrong in the last two cells. Can you try running it locally?

Update: I finish running the notebook locally. The last cell fails probably due to out-of-memory in github CI V100 16G. Last mem use I capture during I run the notebook
2024/07/12 15:43:44.493, 16640 MiB

@mingxin-zheng
Copy link
Contributor

I think the PR overall looks good to me. It would be better to keep the outputs of the last 3 cells of the notebook.

@Can-Zhao
Copy link
Contributor Author

I think the PR overall looks good to me. It would be better to keep the outputs of the last 3 cells of the notebook.

May I ask if I need to finish the whole training process to generate the output cell?

@KumoLiu
Copy link
Contributor

KumoLiu commented Jul 12, 2024

I think the PR overall looks good to me. It would be better to keep the outputs of the last 3 cells of the notebook.

May I ask if I need to finish the whole training process to generate the output cell?

Yes, and after that you can reset the epoch to a small number such as 2 then the ci will not run the whole training.

Signed-off-by: Can-Zhao <volcanofly@gmail.com>
@Can-Zhao
Copy link
Contributor Author

Can-Zhao commented Jul 12, 2024

I think the PR overall looks good to me. It would be better to keep the outputs of the last 3 cells of the notebook.

May I ask if I need to finish the whole training process to generate the output cell?

Yes, and after that you can reset the epoch to a small number such as 2 then the ci will not run the whole training.

That takes a lot of time, I do not have another week...I did not use these datasets for released model anyway...

Signed-off-by: Can-Zhao <volcanofly@gmail.com>
@Can-Zhao
Copy link
Contributor Author

Hi @Can-Zhao , the CI indicates the something may be wrong in the last two cells. Can you try running it locally?

Update: I finish running the notebook locally. The last cell fails probably due to out-of-memory in github CI V100 16G. Last mem use I capture during I run the notebook 2024/07/12 15:43:44.493, 16640 MiB

I changed configs and it is resolved

Can-Zhao and others added 2 commits July 12, 2024 19:05
Signed-off-by: Can-Zhao <volcanofly@gmail.com>
@Can-Zhao
Copy link
Contributor Author

I think the PR overall looks good to me. It would be better to keep the outputs of the last 3 cells of the notebook.

Added output cells for 2 epochs

@Can-Zhao Can-Zhao merged commit b92f9db into Project-MONAI:main Jul 12, 2024
6 checks passed
@Can-Zhao Can-Zhao deleted the maisi_vae branch July 12, 2024 23:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants