Remove cast of y_true to y_pred data type in sparse categorical cross entropy loss

[old-school-kid]

Casting y_true to y_pred data type gives erroneous loss. Like in case of Mean Squared error if y_true is > 2**32 and y_pred data type is int32 then resulting loss will be wrong.
A detailed notebook on the error which also happens in mean absolute loss, categorical cross entropy loss and sparse categorical cross entropy loss and also contains the proposed changes.

The issue was raised here

[qlzh727]

I think the cast is mainly trying to address the dtype mismatch, eg int and float for label and prediction, which is quite common in normal case. The 2*32 seems to be a very extreme case, which we barely hit. Can we change the PR to only cast the dtype if the label and prediction are in different category?

[qlzh727]

I think we should only remove the cast for sparse_categorical_crossentropy(), since the label value could be large based on the dimension of the prediction. The rest of them like binary_crossentropy or categorical_crossentropy, the label value is either one_hot or just 0 and 1, which won't be affected when casting.

Also since backend.sparse_categorical_crossentropy will cast the y_true to int64 anyway, removing the y_true cast here is correct.

keras/keras/backend.py

Line 4952 in 00518dc

target = cast(target, 'int64')

[qlzh727]

I think you should keep this cast here, since the y_true are expect to be either {-1, +1} or {0, 1} (i.e. a one-hot-encoded tensor). See the docstring.

We probably want to fix the example in the docstring since the y_true is given in the range of 0-3.

[old-school-kid]

Yeah made the changes as requested.

[qlzh727]

Thanks for the fix.