[Feature Branch] Logger Framework #1860
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bfineran
previously approved these changes
Dec 11, 2023
bfineran
previously approved these changes
Dec 13, 2023
|
This was referenced Dec 20, 2023
horheynm
previously approved these changes
Jan 3, 2024
author Rahul Tuli <rahul@neuralmagic.com> 1702308513 -0500 committer rahul-tuli <rahul@neuralmagic.com> 1704999325 -0500 [Feature] Add Model Level Logs to new Framework (#1783) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests `[Feature][Logger Framework]` Add convenience methods for logging strings (#1855) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests * Adds convenience methods to logger manager to better conform to python's `logging.Logger` [Feature] Info logs to console and debug logs to file (#1861) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests * Adds convenience methods to logger manager to better conform to python's `logging.Logger` * Move FileHandler creation above StreamHandler creation * Remove missed comment [Move] logger into it's own package (#1924) improves structure and separation of concerns Add local logs directory to .gitignore (#1925) [Add] frequency_manager to check log "readiness" (#1927) * Add frequency_manager to check log "readiness" * Refactor condition to be more readable [Logger Framework][Add] mode and frequency type to FrequencyManager (#1930) * Add mode and frequency type to FrequencyManager * move tests cases near the test
8706a78 to
ce2c9e7
Compare
* Make log_frequency a property of FrequencyManager Add FrequencyManager to LoggerManager Add log_wriiten and model_updated to LoggerManager * [Set] [OLD MODIFIERS] _last_log_epoch to None * Update old modifiers to use named arguments to log_ready * [Update][Rename] * `PossibleLoggingMode` --> `LoggingModeType` * `PossibleFrequencyType` --> `FrequencyType`
* Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger
* Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * Add session level logs
* Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * Add logs for loss * Add test * Use Sparsification Group logger (#1936) * Save last log step Add a stateless log_ready method * log only when loss is not None
bfineran
previously approved these changes
Jan 17, 2024
Satrat
previously approved these changes
Jan 17, 2024
* Add timer in logger manager * Address review comments * update docstring * fix argument
dsikka
approved these changes
Jan 19, 2024
Satrat
approved these changes
Jan 19, 2024
bfineran
approved these changes
Jan 19, 2024
rahul-tuli
added a commit
that referenced
this pull request
Jan 19, 2024
* parent 5007b8c author Rahul Tuli <rahul@neuralmagic.com> 1702308513 -0500 committer rahul-tuli <rahul@neuralmagic.com> 1704999325 -0500 [Feature] Add Model Level Logs to new Framework (#1783) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests `[Feature][Logger Framework]` Add convenience methods for logging strings (#1855) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests * Adds convenience methods to logger manager to better conform to python's `logging.Logger` [Feature] Info logs to console and debug logs to file (#1861) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests * Adds convenience methods to logger manager to better conform to python's `logging.Logger` * Move FileHandler creation above StreamHandler creation * Remove missed comment [Move] logger into it's own package (#1924) improves structure and separation of concerns Add local logs directory to .gitignore (#1925) [Add] frequency_manager to check log "readiness" (#1927) * Add frequency_manager to check log "readiness" * Refactor condition to be more readable [Logger Framework][Add] mode and frequency type to FrequencyManager (#1930) * Add mode and frequency type to FrequencyManager * move tests cases near the test * Style * [Use] frequency manager in logger (#1931) * Make log_frequency a property of FrequencyManager Add FrequencyManager to LoggerManager Add log_wriiten and model_updated to LoggerManager * [Set] [OLD MODIFIERS] _last_log_epoch to None * Update old modifiers to use named arguments to log_ready * [Update][Rename] * `PossibleLoggingMode` --> `LoggingModeType` * `PossibleFrequencyType` --> `FrequencyType` * Fix log format when step is None (#1933) * [Add] Wrap System and Metric Logging into it's own classes (#1932) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * [Logger Refactor] Session based Logs (#1920) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * Add session level logs * [Logger Framework] Log Losses (#1934) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * Add logs for loss * Add test * Use Sparsification Group logger (#1936) * Save last log step Add a stateless log_ready method * log only when loss is not None * Add tb style logging functions (#1919) * Add wandb log function (#1918) * [Test Update] Fix failing tests after adding back old logger (#1963) * [Logger Refactor] Add timer in logger manager (#1967) * Add timer in logger manager * Address review comments * update docstring * fix argument
rahul-tuli
added a commit
that referenced
this pull request
Jan 22, 2024
* parent 5007b8c author Rahul Tuli <rahul@neuralmagic.com> 1702308513 -0500 committer rahul-tuli <rahul@neuralmagic.com> 1704999325 -0500 [Feature] Add Model Level Logs to new Framework (#1783) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests `[Feature][Logger Framework]` Add convenience methods for logging strings (#1855) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests * Adds convenience methods to logger manager to better conform to python's `logging.Logger` [Feature] Info logs to console and debug logs to file (#1861) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests * Adds convenience methods to logger manager to better conform to python's `logging.Logger` * Move FileHandler creation above StreamHandler creation * Remove missed comment [Move] logger into it's own package (#1924) improves structure and separation of concerns Add local logs directory to .gitignore (#1925) [Add] frequency_manager to check log "readiness" (#1927) * Add frequency_manager to check log "readiness" * Refactor condition to be more readable [Logger Framework][Add] mode and frequency type to FrequencyManager (#1930) * Add mode and frequency type to FrequencyManager * move tests cases near the test * Style * [Use] frequency manager in logger (#1931) * Make log_frequency a property of FrequencyManager Add FrequencyManager to LoggerManager Add log_wriiten and model_updated to LoggerManager * [Set] [OLD MODIFIERS] _last_log_epoch to None * Update old modifiers to use named arguments to log_ready * [Update][Rename] * `PossibleLoggingMode` --> `LoggingModeType` * `PossibleFrequencyType` --> `FrequencyType` * Fix log format when step is None (#1933) * [Add] Wrap System and Metric Logging into it's own classes (#1932) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * [Logger Refactor] Session based Logs (#1920) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * Add session level logs * [Logger Framework] Log Losses (#1934) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * Add logs for loss * Add test * Use Sparsification Group logger (#1936) * Save last log step Add a stateless log_ready method * log only when loss is not None * Add tb style logging functions (#1919) * Add wandb log function (#1918) * [Test Update] Fix failing tests after adding back old logger (#1963) * [Logger Refactor] Add timer in logger manager (#1967) * Add timer in logger manager * Address review comments * update docstring * fix argument
bfineran
pushed a commit
that referenced
this pull request
Jan 22, 2024
* parent 5007b8c author Rahul Tuli <rahul@neuralmagic.com> 1702308513 -0500 committer rahul-tuli <rahul@neuralmagic.com> 1704999325 -0500 [Feature] Add Model Level Logs to new Framework (#1783) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests `[Feature][Logger Framework]` Add convenience methods for logging strings (#1855) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests * Adds convenience methods to logger manager to better conform to python's `logging.Logger` [Feature] Info logs to console and debug logs to file (#1861) * Empty commit * - Add loggable_items in ModifiableModel Contract - Add pytorch implementation od loggable_items - Attach logger to state - Make changes in lifecycle convenience functions to include loggers - Add log_model_info method in Modifier which is called on each `update_event` but logs only on epoch end * - Move log call at the end of update_event - Remove extra space * - Move model level logging responsibility to `ModelLoggingMixin_` * - Style * - Update docstring * - Move mixin to it's own file * - Add test * - update logic to check epoch end * - Add more tests * - log model level logs at session level - filter logs to only include percentages - filter logs to only include params with non zero sparsity - filter logs to include only quantized modules - LoggerMixin functions to helpers.py * - Expose model log cadence - Propagate cadence via session to state and loggers - log model info in session - update log condition in logger.py * - Migrate tests * Fix epoch number logging * - fix failing test * Address review comments! * Remove event_type from should log model info * Update docstring * Style * mock _log_model_info * Move to using log scalar over log string Do not log at last_log_epoch but only if current epoch has reached specified cadence Remove model log cadence Style Update tests * Adds convenience methods to logger manager to better conform to python's `logging.Logger` * Move FileHandler creation above StreamHandler creation * Remove missed comment [Move] logger into it's own package (#1924) improves structure and separation of concerns Add local logs directory to .gitignore (#1925) [Add] frequency_manager to check log "readiness" (#1927) * Add frequency_manager to check log "readiness" * Refactor condition to be more readable [Logger Framework][Add] mode and frequency type to FrequencyManager (#1930) * Add mode and frequency type to FrequencyManager * move tests cases near the test * Style * [Use] frequency manager in logger (#1931) * Make log_frequency a property of FrequencyManager Add FrequencyManager to LoggerManager Add log_wriiten and model_updated to LoggerManager * [Set] [OLD MODIFIERS] _last_log_epoch to None * Update old modifiers to use named arguments to log_ready * [Update][Rename] * `PossibleLoggingMode` --> `LoggingModeType` * `PossibleFrequencyType` --> `FrequencyType` * Fix log format when step is None (#1933) * [Add] Wrap System and Metric Logging into it's own classes (#1932) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * [Logger Refactor] Session based Logs (#1920) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * Add session level logs * [Logger Framework] Log Losses (#1934) * Add __repr__ Move string logging functions to SystemLoggingWrapper Move metric logging functions to MetricLoggingWrapper * minor fixes * Remove warnings * Add back old logger * Add logs for loss * Add test * Use Sparsification Group logger (#1936) * Save last log step Add a stateless log_ready method * log only when loss is not None * Add tb style logging functions (#1919) * Add wandb log function (#1918) * [Test Update] Fix failing tests after adding back old logger (#1963) * [Logger Refactor] Add timer in logger manager (#1967) * Add timer in logger manager * Address review comments * update docstring * fix argument
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR serves as the feature branch for the logger framework refactor, all PRs pertaining to loggers will be merged into this, and this branch will land into main, I have tried to use a stacked PR approach here:
sparseml.coreand Update existing modifiers to use logger manager fromsparseml.core: This has been merged into main [move][logger] Movelogger.pytosparseml.core#1759[Feature][Logger Framework]Add convenience methods for logging strings #1855.gitignoreFrequencyManager(w/o toggles)None#1933 Prettier log formatFrequencyManagertogglesFix a failing test: