This repository has been archived by the owner on Mar 19, 2024. It is now read-only.
Add registry for the model heads #75
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This pull request was exported from Phabricator. Differential Revision: D22961720 |
Differential Revision: D22842068 fbshipit-source-id: f73a2d4739a097afb29b38332dccf365b77fd4b8
Differential Revision: D22856092 fbshipit-source-id: af1287ac6b87a97ac26773056bbf164f1ca7705b
Differential Revision: D22865193 fbshipit-source-id: ede4e27bbb1bb509204fedb204da8e64248d4fa0
…nd key access Differential Revision: D22872266 fbshipit-source-id: 6cfb9149158c63b8dc28a472f19469b07b9791ab
Differential Revision: D22874810 fbshipit-source-id: 6e4cc5deb0dd86d0cbcb9d1a564038494d6ae21b
Differential Revision: D22875254 fbshipit-source-id: a0752656418f7a6648f214e5b0488d984200d0e6
Differential Revision: D22877422 fbshipit-source-id: b07ad2091b2e73287611776ee7ce2007f35087ed
…s logic Differential Revision: D22878016 fbshipit-source-id: 1f1bed073ac31d394137064920d3ed275f5cdc0a
Differential Revision: D22878988 fbshipit-source-id: f09f4d3bb3d5fcaaab3b889e1acf3569574262b7
Differential Revision: D22909000 fbshipit-source-id: 16628c0e86993f2e606af513ee8092819d9c9d20
Differential Revision: D22909489 fbshipit-source-id: 53a604da7c458888f3bb7582dcdc9e455fd516f8
Differential Revision: D22909827 fbshipit-source-id: 18448fdd2e7356b8c20f30598fe3c7cae407f241
Differential Revision: D22961589 fbshipit-source-id: 1aa52dc07fc8e2cafe02ec9a47b91793b70f4be2
Differential Revision: D22961659 fbshipit-source-id: a7f4f86ac8dcb5a74a5edb708cd6f56aa0943d13
Differential Revision: D22961669 fbshipit-source-id: a8466b854dc2bda1ce483150f9f2725648450285
Differential Revision: D22961685 fbshipit-source-id: 8738c64893fdc908705cabcb7b0d21ce4a0febeb
…ase of multiple meters Differential Revision: D22961705 fbshipit-source-id: b27b63da82b33315bf87c5c2ff902a7a25adddd7
Summary: Pull Request resolved: #75 adding a registry for the heads as well. another components in vissl that's extendible Reviewed By: mannatsingh Differential Revision: D22961720 fbshipit-source-id: 3d54829a3e033a28aec2456b6e37e584b7209616
|
This pull request was exported from Phabricator. Differential Revision: D22961720 |
facebook-github-bot
pushed a commit
that referenced
this pull request
Aug 10, 2020
Summary: Pull Request resolved: #75 adding a registry for the heads as well. another components in vissl that's extendible Reviewed By: mannatsingh Differential Revision: D22961720 fbshipit-source-id: 481eaec04bd8410c24b9a2882dd1435265426407
facebook-github-bot
pushed a commit
that referenced
this pull request
Apr 15, 2021
Summary: Before this PR (facebookresearch/fairscale#543) was merged, we used to need the extra cuda() calls. Now, they are not needed. Unfortunately, this doesn't solve the long model init time issue we have. A FSDP model init still take >20 mins for me. This is really bad for debugging the regnet128 conv layer crash problem I am debugging. The following debugging output shows that most delays are in FSDP wrapping, some in BN wrapping and some in the layer wrapping. ``` INFO 2021-04-14 12:18:35,883 regnet_2.py: 159: block created INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:18:35,884 regnet_2.py: 161: cpu INFO 2021-04-14 12:19:07,388 regnet_2.py: 163: block bn wrapped INFO 2021-04-14 12:19:18,388 regnet_2.py: 166: block wrapped ``` In any case, this PR is pretty safe and should go in so that we don't need to do an extra `cuda()` call before wrapping. Pull Request resolved: fairinternal/ssl_scaling#75 Reviewed By: prigoyal Differential Revision: D27776285 Pulled By: min-xu-ai fbshipit-source-id: 3e43c6fe750fd6ee35933400b03a069d62040d8a
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: adding a registry for the heads as well. another components in vissl that's extendible
Reviewed By: mannatsingh
Differential Revision: D22961720