Skip to content

Commit

Permalink
Update gradient_centralization.py
Browse files Browse the repository at this point in the history
  • Loading branch information
tilakrayal authored Jan 29, 2025
1 parent 6681f9e commit 0a68ead
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion examples/vision/gradient_centralization.py
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ def prepare(ds, shuffle=False, augment=False):
subclass the `RMSProp` optimizer class modifying the
`keras.optimizers.Optimizer.get_gradients()` method where we now implement Gradient
Centralization. On a high level the idea is that let us say we obtain our gradients
through back propogation for a Dense or Convolution layer we then compute the mean of the
through back propagation for a Dense or Convolution layer we then compute the mean of the
column vectors of the weight matrix, and then remove the mean from each column vector.
The experiments in [this paper](https://arxiv.org/abs/2004.01461) on various
Expand Down

0 comments on commit 0a68ead

Please sign in to comment.