Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force hook standalone tests to single device #20491

Merged
merged 3 commits into from
Dec 11, 2024

Conversation

lantiga
Copy link
Collaborator

@lantiga lantiga commented Dec 11, 2024

What does this PR do?

A few failures in standalone tests are due to the fact that hook tests cannot be run on multiple devices (at least the way they are written), since prepare_data only runs on rank 0. This was the cause of rather obscure failures.

Since hook tests are verifying the sanity of the sequence of hooks, it's ok to limit to single device.


📚 Documentation preview 📚: https://pytorch-lightning--20491.org.readthedocs.build/en/20491/

@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Dec 11, 2024
Copy link
Contributor

github-actions bot commented Dec 11, 2024

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow
Check ID Status
pl-cpu (macOS-14, lightning, 3.9, 2.1, oldest) success
pl-cpu (macOS-14, lightning, 3.10, 2.1) success
pl-cpu (macOS-14, lightning, 3.11, 2.2.2) success
pl-cpu (macOS-14, lightning, 3.11, 2.3) success
pl-cpu (macOS-14, lightning, 3.12, 2.4.1) success
pl-cpu (macOS-14, lightning, 3.12, 2.5.1) success
pl-cpu (ubuntu-20.04, lightning, 3.9, 2.1, oldest) success
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1) success
pl-cpu (ubuntu-20.04, lightning, 3.11, 2.2.2) success
pl-cpu (ubuntu-20.04, lightning, 3.11, 2.3) success
pl-cpu (ubuntu-22.04, lightning, 3.12, 2.4.1) success
pl-cpu (ubuntu-22.04, lightning, 3.12, 2.5.1) success
pl-cpu (windows-2022, lightning, 3.9, 2.1, oldest) success
pl-cpu (windows-2022, lightning, 3.10, 2.1) success
pl-cpu (windows-2022, lightning, 3.11, 2.2.2) success
pl-cpu (windows-2022, lightning, 3.11, 2.3) success
pl-cpu (windows-2022, lightning, 3.12, 2.4.1) success
pl-cpu (windows-2022, lightning, 3.12, 2.5.1) success
pl-cpu (macOS-14, pytorch, 3.9, 2.1) success
pl-cpu (ubuntu-20.04, pytorch, 3.9, 2.1) success
pl-cpu (windows-2022, pytorch, 3.9, 2.1) success
pl-cpu (macOS-14, pytorch, 3.12, 2.5.1) success
pl-cpu (ubuntu-22.04, pytorch, 3.12, 2.5.1) success
pl-cpu (windows-2022, pytorch, 3.12, 2.5.1) success

These checks are required after the changes to tests/tests_pytorch/models/test_hooks.py.

🟢 pytorch_lightning: Azure GPU
Check ID Status
pytorch-lightning (GPUs) (testing Lightning | latest) success
pytorch-lightning (GPUs) (testing PyTorch | latest) success

These checks are required after the changes to tests/tests_pytorch/models/test_hooks.py.


Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

Copy link

codecov bot commented Dec 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79%. Comparing base (9709c64) to head (5f6c325).
Report is 1 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (9709c64) and HEAD (5f6c325). Click for more details.

HEAD has 102 uploads less than BASE
Flag BASE (9709c64) HEAD (5f6c325)
cpu 48 24
lightning 37 18
pytest 26 0
python3.9 12 6
lightning_fabric 7 0
python3.11 12 6
python3.10 6 3
gpu 2 0
python3.12 18 9
Additional details and impacted files
@@            Coverage Diff            @@
##           master   #20491     +/-   ##
=========================================
- Coverage      88%      79%     -9%     
=========================================
  Files         267      264      -3     
  Lines       23382    23327     -55     
=========================================
- Hits        20479    18364   -2115     
- Misses       2903     4963   +2060     

@lantiga lantiga merged commit 60289d7 into master Dec 11, 2024
38 checks passed
@lantiga lantiga deleted the luca/force-some-standalone-tests-to-single-device branch December 11, 2024 11:17
@Borda
Copy link
Member

Borda commented Dec 11, 2024

Does the fallback to single device defeat the purpose of the test?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pl Generic label for PyTorch Lightning package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants