Bug in WeightedPersSampleLoss

[vrodriguezf]

This code:

dsid = 'NATOPS' 
X, y, splits = get_UCR_data(dsid, return_split=False)
learn = TSClassifier(X, y, splits=splits, bs=[64, 128], 
                     batch_tfms=[TSStandardize()], 
                     arch=InceptionTime,
                     cbs = [WeightedPerSampleLoss(np.arange(len(X)))],
                     metrics=accuracy)
learn.fit_one_cycle(25, lr_max=1e-3)
learn.plot_metrics()

leaves this output:

TypeError: int() argument must be a string, a bytes-like object or a number, not 'slice'

The line of the callback that is failing is this one

I think it has to do with the fact that learn.dls.train.idxs is a list, but learn.dls.valid.idxs is a slice object, and the callback expects a list.

Not sure how to fix it though :)

Thanks!!!

[oguiza]

Hi @vrodriguezf,

I've checked this and there's indeed a bug.
Could you please try this code? I think it works well. If so, I'll add it to the code base and load it to GitHub.

class WeightedPerSampleLoss(Callback):
    order = 65

    def __init__(self, instance_weights):
        store_attr()

    def before_fit(self):
        self.old_loss = self.learn.loss_func
        self.reduction = getattr(self.learn.loss_func, 'reduction', None)
        self.learn.loss_func = _PerInstanceLoss(crit=self.learn.loss_func)
        assert len(self.instance_weights) == len(self.learn.dls.train.dataset) + len(self.learn.dls.valid.dataset)
        self.instance_weights = torch.as_tensor(self.instance_weights, device=self.learn.dls.device)

    def before_batch(self):
        input_idxs = self.learn.dls.train.input_idxs if self.training else self.learn.dls.valid.input_idxs
        self.learn.loss_func.weights = self.instance_weights[input_idxs]

    def after_fit(self):
        self.learn.loss_func = self.old_loss
        if self.reduction is not None: self.learn.loss_func.reduction = self.reduction


class _PerInstanceLoss(Module):
    def __init__(self, crit):
        self.crit = crit
        self.crit.reduction = 'none'
        self.weights = None

    def forward(self, input, target):
        return (self.crit(input, target) * self.weights / self.weights.sum()).sum()

Please, let me know if it works.

[vrodriguezf]

It works like a charm :) thank you @oguiza !!!

[oguiza]

cc: @vrodriguezf
I've just loaded the updated code to GitHub.
Thanks for raising this!