Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creates LogLossMetric and generalizes WandbLogger and ExamplesPerSecondCallback #1085

Merged
merged 1 commit into from
May 10, 2023

Conversation

gabrielspmoreira
Copy link
Member

Goals ⚽

This PR moves some utility classes from the Quick-start for Ranking scripts to Merlin Models, as they are generic and are useful in general for building training scripts with the library: LogLossMetric, WandbLogger, ExamplesPerSecondCallback

Implementation Details 🚧

  • LogLossMetric: Log loss (binary cross entropy( is a common loss and metric for binary classification. This class is useful when your are using class weights for binary classification, as in that case the loss for training will be influenced by the class weights. Having this metric allows you to have the unweighted log loss metric.
  • WandbLogger - This class manages the logging to Weights&Biases, providing methods to initialize wandb, configure the run, log metrics and to return the tf.keras.WandbCallback() that can be used with Keras fit() and evaluate().
  • ExamplesPerSecondCallback - Logs the training or evaluation throughput, i.e. steps/sec (to wandb or to console)

Testing Details 🔍

  • Adds a test for LogLossMetric: test_logloss_metric
  • Changed retrieval integration tests to use the generalized WandbLogger and ExamplesPerSecondCallback

@gabrielspmoreira gabrielspmoreira requested review from rnyak and sararb May 8, 2023 19:52
@gabrielspmoreira gabrielspmoreira self-assigned this May 8, 2023
@gabrielspmoreira gabrielspmoreira added the enhancement New feature or request label May 8, 2023
@gabrielspmoreira gabrielspmoreira added this to the Merlin 23.05 milestone May 8, 2023
@github-actions
Copy link

github-actions bot commented May 8, 2023

Documentation preview

https://nvidia-merlin.github.io/models/review/pr-1085

@gabrielspmoreira gabrielspmoreira merged commit 61321a3 into main May 10, 2023
@rnyak rnyak deleted the tf/loglossmetric_callbacks branch May 10, 2023 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants