[ONNX FE] Implement SoftmaxCrossEntropyLoss for ONNX frontend (opset 12 & 13)

[AJThePro99]

Softmax Cross-Entropy Loss Implementation for ONNX FE

Ticket #20547

This pull request implements the Softmax Cross-Entropy Loss operation for the ONNX frontend in OpenVINO, supporting opset 12 and opset 13. It includes the core functionality, a corresponding header file, a new test model, and a test case for validation.

Key Changes

1. Implementation of Softmax Cross-Entropy Loss

• softmax_crossentropy_loss.cpp
  Implements the onnx_softmax_crossentropy_loss function, handling inputs, attributes, and different reduction modes.

• softmax_crossentropy_loss.hpp
  Declares the softmax_crossentropy_loss function for opset 12 and opset 13.

2. Testing

• softmax_crossentropy_loss_sum.prototxt
  New ONNX test model for the Softmax Cross-Entropy Loss operation with the "sum" reduction mode.
• onnx_import.in.cpp
  Added a test case to validate the Softmax Cross-Entropy Loss operation using the new test model.

Additional Notes

• The implementation follows the ONNX specification for SoftmaxCrossEntropyLoss.
• The test model and case ensure correctness across various reduction modes.
• The PR has been tested for accuracy, and precision calculations were verified.

---

Next Steps

• Additional test models covering different use cases will be added in future updates.
• Check Python xfailed tests for added functionality
• Looking forward to feedback from the reviewers regarding any necessary improvements.

---

Proof of Precision Calculation

I will attach a comment with:

1. A screenshot of the Python script used for precision verification.
2. The output of the script.
3. A terminal instance showing the test case execution output.

Please review and let me know if any modifications are required!

[AJThePro99]

Verification of Correctness for SoftmaxCrossEntropyLoss

I verified that the implementation produces the correct results according to ONNX Runtime

---

Precision Check Python Script

What the screenshot shows:

• Python script(s) which uses the softmax_crossentropy_loss_sum.onnx generated during build and onnx runtime to calculate softmax crossentropy loss value of the given inputs (Same inputs given in onnx_import.in.cpp)

    ...
    test_case.add_input<float>({0.5f, 1.5f, 2.5f, 3.5f, 4.5f, 5.5f});
    test_case.add_input<int64_t>({1, 2});
    test_case.add_expected_output<float>(Shape{}, {1.81521f});

• softmax_crossentropy_loss_sum.onnx is based on softmax_crossentropy_loss_sum.prototxt

Since the test cases are now passing, this serves as a reference for maintainers to confirm that the operation behaves as expected and aligns with ONNX Runtime results.

[AJThePro99]

Hello @rkazants,
I've opened a new PR with fully working code.
Could you kindly review my PR when you get a chance?
Let me know if there’s anything I should improve.

Thanks for your time!

[rkazants]

build_jenkins

[AJThePro99]

Added the remaining test models, covering respectively:

Test Case Name	Reduction	Weights	Ignore Index	Score Shape	Label Shape	Output
softmax_crossentropy_loss_sum	sum	No	N/A	(2, 3)	(2)	Scalar
softmax_crossentropy_loss_mean_weight_ii	mean	Yes	-1	(2, 3, 2)	(2, 2)	Scalar
softmax_crossentropy_loss_none	none	No	N/A	(2, 3)	(2)	Vector (2)
softmax_crossentropy_loss_higher_dim	mean	No	N/A	(2, 3, 2, 2)	(2, 2, 2)	Scalar

[AJThePro99]

Verification for softmax_crossentropy_loss_mean_weight_ii

What the screenshot shows:

Python script(s) which uses the softmax_crossentropy_loss_mean_weight_ii generated during build and onnx runtime to calculate softmax crossentropy loss value of the given inputs (Same inputs given in onnx_import.in.cpp)

    ...
    test_case.add_input<float>({1.f, 2.f, 3.f, 4.f, 5.f, 6.f, 7.f, 8.f, 9.f, 10.f, 11.f, 12.f});
    test_case.add_input<int64_t>({0, -1, 2, 1});
    test_case.add_input<float>({0.5f, 1.0f, 2.0f});

    test_case.add_expected_output<float>(Shape{}, {1.2857887f});
    test_case.run_with_tolerance_as_fp(0.001f);

softmax_crossentropy_loss_mean_weight_ii.onnx is based on softmax_crossentropy_loss_mean_weight_ii.prototxt

[AJThePro99]

Verification for softmax_crossentropy_loss_none

What the screenshot shows:

Python script(s) which uses the softmax_crossentropy_loss_none generated during build and onnx runtime to calculate softmax crossentropy loss value of the given inputs (Same inputs given in onnx_import.in.cpp

    ...
    test_case.add_input<float>({1.f, 2.f, 3.f, 4.f, 5.f, 6.f});
    test_case.add_input<int64_t>({2, 1});

    test_case.add_expected_output<float>(Shape{2}, {0.40760595f, 1.4076059f});
    test_case.run_with_tolerance_as_fp(0.001f);

softmax_crossentropy_loss_none.onnx is based on softmax_crossentropy_loss_none.prototxt

[AJThePro99]

Verification for softmax_crossentropy_loss_higher_dim

What the screenshot shows:

Python script(s) which uses the softmax_crossentropy_loss_higher_dim generated during build and onnx runtime to calculate softmax crossentropy loss value of the given inputs (Same inputs given in onnx_import.in.cpp

    ...
    test_case.add_input<float>(std::vector<float>(24, 1.0f));
    test_case.add_input<int64_t>({1, 0, 2, 1, 0, 2, 1, 1});

    test_case.add_expected_output<float>(Shape{}, {1.0986123f});
    test_case.run_with_tolerance_as_fp(0.001f);

softmax_crossentropy_loss_higher_dim.onnx is based on softmax_crossentropy_loss_higher_dim.prototxt

[AJThePro99]

• Creation of sfce-loss .cpp and .hpp files
• Creation of test models for sfce-loss
• Checking Python xfailed tests (No applicable xfailed tests)

[AJThePro99]

Summary of Changes:

• Implemented SoftmaxCrossEntropyLoss for the ONNX frontend, supporting opsets 12 and 13.
• Added corresponding unit tests to validate functionality.

Hello @rkazants,
I have completed the planned implementation and testing. Could you please review the changes and provide feedback? Additionally, if there are any further modifications or enhancements you'd like to see, please let me know, and I'll be happy to incorporate them.

Thank you!

[AJThePro99]

Hi @rkazants , just following up on this PR. Let me know if any changes are needed.
Thanks!