Test & Score misclassification with SVM model

[VGBarauna]

What's wrong?

Some samples have higher % for "positive" but are being classified as "negative"

How can we reproduce the problem?

Chagas teste.xlsx

What's your environment?
3.37.00

• Operating system: Windows 11
• Orange version: 3.37.00
  Dowload from website

[ales-erjavec]

Probably https://scikit-learn.org/stable/modules/svm.html#scores-and-probabilities

TL;DR The base SVM method by construction does not support estimating probabilities. The probabilities that are reported by scikit-learn/libsvm are the result of a different model trained specifically to estimate probabilities on SVM decision function scores. But these can output probabilities that are not consistent with the base SVM prediction.

[VGBarauna]

Dear Ales-erjavec,
Thank you very much for your explanations. So, this data is about scores and not probabilities. So I should consider only the prediction column in the data table and ignore these metrics, right? The next problem is that the ROC curve will be wrong because it is built with the probabilities of each sample, and in this case, the widget is getting the scores, right? Is there no way to make the ROC curve from the SVM since the data table generates scores and not probabilities?

[thocevar]

@VGBarauna You can consider only the "SVM (positive)" column as the predicted probability of the positive class and compute anything else that you might need (e.g. actual class being positive if the prediction > 0.5) from that probability. You can also use these probabilities for ROC curve.

We might want to consider overriding the SklModel's class predictions with the argmax of the computed probability distribution (when available) for consistency.

orange3/Orange/base.py

Line 528 in afc6c18

return value, probs