[Good First Issue]: Extend ONNX Frontend with Function SoftmaxCrossEntropyLoss

[gkrivor]

Context

Neural networks are graphs consisting of nodes called operators. Each operator corresponds to a mathematical function, usually described in framework's documentation or an AI standard, such as ONNX.
OpenVINO ONNX Frontend is a component responsible for working with ONNX graphs and requires implementation of different ONNX operators in order to use ONNX models.
This task requires extending OpenVINO ONNX Frontend with Function SoftmaxCrossEntropyLoss.
Necessary help will be provided by ONNX Fronted team.

What needs to be done?

Operator details can be found in ONNX Operators
More details can be found in ONNX Changelog

1. Create .hpp and .cpp files for *Windows here
2. Prepare an implementation of this operator in form of a function. SoftmaxCrossEntropyLoss-12 should be placed in opset 1 namespace. SoftmaxCrossEntropyLoss-13 should be placed in opset 13 namespace.
3. Register the function in ops_bridge.cpp while keeping alphabetical order
4. Create test model(s) in ONNX models directory. OpenVINO test infrastructure then converts prototxt files to ONNX models - you will use those models later in tests
5. Add tests covering all use cases here
6. Check Python xfailed tests to find a test marked as a xfailed for added functionality. If any exist - remove corresponding lines and try to verify by using cmdline "python -m pytest -k name_of_test".
   More details in adding operators to ONNX Frontend guide

Example Pull Requests

• ONNX Unique op support #15076
• ONNX CastLike operator tests #14936
• [ONNX FE] Extend ONNX FE for operation GenerateProposals #12510
• [ONNX] Extend ONNX Frontend with BlackmanWindow, HammingWindow and HannWindow operators #19428

Resources

• Contribution guide - start here!
• What is OpenVINO?
• Contribution guide
• User documentation
• Guides on ONNX Frontend architecture and ONNX Frontend tests are also available
• You can also see a blog post on contributing to OpenVINO

Contact points

@gkrivor

Ticket

No response

[sydarb]

.take

[github-actions]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[p-wysocki]

Hello @sydarb! Thank you for taking a look, please let us know if you have any questions. Just yesterday our CONTRIBUTING.md has been updated with a technical guide - I highly recommend checking it out. :)

[p-wysocki]

Hello @sydarb, are you still working on this issue?

I am happy to announce that we have created a channel dedicated to Good First Issues support on our Intel DevHub Discord server! Join it to receive support, engage in discussions, ask questions and talk to OpenVINO developers.

[tanishka321]

.take

[github-actions]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[p-wysocki]

Hello @tanishka321, can we help you with anything?

[mitruska]

Hello @tanishka321, thank you for your contribution!
I've reviewed the PR #22650, unfortunately the proposed solution is not working yet.
Please take a look at the latest review comments.

@gkrivor As ONNX SoftmaxCrossEntropyLoss is training related operator, I would suggest to verify it's applicability for custom decomposition in OpenVINO ONNX FE.

[mlukasze]

hey @tanishka321
will you work on this ticket? Or maybe you don't have time and we should move it back for other person?

[kshitij01042002]

Hi @mlukasze if the current assignee isn't working on the issue, can I take it up?

[mlukasze]

moving to you @kshitij01042002, have fun :)

[kshitij01042002]

Hi @mlukasze I am working on this issue, sorry I was caught up with my academics and was not able to do it. Also there was a request can you please look into the mail sent by @PRATHAM-SPS. It was needed on an urgent basis.

cc: @p-wysocki

[AJThePro99]

.take

[github-actions]

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

[AJThePro99]

Hey! @gkrivor
Can I have an update on this issue?

I believe this issue hasnt been worked on for a long time, and I've found out that opset 1 namespace leads to nothing.

SoftmaxCrossEntropyLoss-12 should be placed in opset 1 namespace.

I'd really like an update for this issue so that I can progress on the solution for this issue.

Many thanks!

[p-wysocki]

bump @gkrivor

[AJThePro99]

Hello? Any update please?

I'm working on a the files, and I still have to figure out how to test the operator.
There seems to be an updated file structure, and I'd like guidance on how to properly structure the .cpp and .hpp files within the project directory.

I've got an idea of where to save the files (Referenced other operators) but I'd still like a heads up and other updates going forward with this ticket

Thanks in advance!

[AJThePro99]

Hello, @gkrivor, @p-wysocki

I'm really sorry for pinging you, but I am in need of help + clarification.

I'd like some help with my current PR for this ticket.

I'm facing some issues while running the tests.
I've followed the build instructions for Windows and also ONNX Tests

More specifically, this is what I did to build the project for running onnx tests:

cmake -G "Visual Studio 17 2022" -DENABLE_TESTS=ON -DCMAKE_BUILD_TYPE=Release ..
cmake --build . --config Release --verbose -j11

The build succeeds with a couple of warnings.

And have run the onnx tests command

<OV_REPO_DIR>/bin/intel64/<OV_BUILD_TYPE>/ov_onnx_frontend_tests

I am using Visual Studio 2022 for working with this project

But, Here is the result:

PS C:\Users\aadit\Contrib\openvino\bin\intel64\Release> .\ov_onnx_frontend_tests.exe
Running main() from src\frontends\tests\frontend\shared\gtest_main_manifest\main.cpp:20
Note: Google Test filter = *:-:ONNXLoadTest/FrontEndLoadFromTest.testLoadFromTwoFiles/onnx:ONNXLoadTest/FrontEndLoadFromTest.testLoadFromTwoStreams/onnx
[==========] Running 2220 tests from 29 test suites.
[----------] Global test environment set-up.
[----------] 738 tests from INTERPRETER
[ RUN      ] INTERPRETER.onnx_model_quantize_linear_const_scale_const_zero_p
unknown file: error: C++ exception with description "Exception from src\frontends\onnx\onnx_common\src\parser.cpp:24:
Could not open the file: "C:\Users\aadit\Contrib\openvino\bin\intel64\Release\test_model_zoo\onnx\quantize_linear_const.onnx"
" thrown in the test body.
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear_const_scale_const_zero_p (2 ms)
[ RUN      ] INTERPRETER.onnx_model_quantize_linear
unknown file: error: C++ exception with description "Exception from src\frontends\onnx\onnx_common\src\parser.cpp:24:
Could not open the file: "C:\Users\aadit\Contrib\openvino\bin\intel64\Release\test_model_zoo\onnx\quantize_linear.onnx"
" thrown in the test body.
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear (0 ms)
[ RUN      ] INTERPRETER.onnx_model_quantize_linear_u16
unknown file: error: C++ exception with description "Exception from src\frontends\onnx\onnx_common\src\parser.cpp:24:
Could not open the file: "C:\Users\aadit\Contrib\openvino\bin\intel64\Release\test_model_zoo\onnx\quantize_linear_u16.onnx"
" thrown in the test body.

...
...

[  FAILED  ] OnnxFeMMapReadModel/OnnxFeMmapFixture.onnx_external_data_uint4/0, where GetParam() = false (1 ms)
[ RUN      ] OnnxFeMMapReadModel/OnnxFeMmapFixture.onnx_external_data_uint4/1
unknown file: error: C++ exception with description "Exception from src\inference\src\cpp\core.cpp:95:
Check 'util::directory_exists(path) || util::file_exists(path)' failed at src\frontends\common\src\frontend.cpp:117:
FrontEnd API failed with GeneralFailure:
onnx: Could not open the file: "C:\Users\aadit\Contrib\openvino\bin\intel64\Release\./test_model_zoo/onnx/external_data/external_data_uint4.onnx"

" thrown in the test body.
[  FAILED  ] OnnxFeMMapReadModel/OnnxFeMmapFixture.onnx_external_data_uint4/1, where GetParam() = true (1 ms)
[----------] 42 tests from OnnxFeMMapReadModel/OnnxFeMmapFixture (70 ms total)

[----------] Global test environment tear-down
[==========] 2220 tests from 29 test suites ran. (1099 ms total)
[  PASSED  ] 41 tests.
[  SKIPPED ] 6 tests, listed below:
[  SKIPPED ] INTERPRETER.onnx_model_nonmaxsuppression_default_score_threshold
[  SKIPPED ] INTERPRETER.onnx_model_reduce_max_18
[  SKIPPED ] INTERPRETER.onnx_model_reduce_prod_18
[  SKIPPED ] INTERPRETER.onnx_model_reduce_min_18
[  SKIPPED ] INTERPRETER.onnx_model_reduce_min_20_boolean
[  SKIPPED ] ONNX/FrontendLibCloseTest.testUnloadLibBeforeDeletingDependentObject/onnx
[  FAILED  ] 2173 tests, listed below:
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear_const_scale_const_zero_p
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear_u16
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear_zero_point
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear_axis_zero
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear_axis_negative
[  FAILED  ] INTERPRETER.onnx_model_quantize_linear_scalar_ignore_axis

...
...

[  FAILED  ] OnnxFeMMapReadModel/OnnxFeMmapFixture.onnx_external_data_int4/0, where GetParam() = false
[  FAILED  ] OnnxFeMMapReadModel/OnnxFeMmapFixture.onnx_external_data_int4/1, where GetParam() = true
[  FAILED  ] OnnxFeMMapReadModel/OnnxFeMmapFixture.onnx_external_data_uint4/0, where GetParam() = false
[  FAILED  ] OnnxFeMMapReadModel/OnnxFeMmapFixture.onnx_external_data_uint4/1, where GetParam() = true

2173 FAILED TESTS
  YOU HAVE 409 DISABLED TESTS

---

I'd like to know how to sort out this issue, since I do not really understand what's happening here.

And, here is the result when I run the test with a filter on an already existing ONNX_TEST() function (softmax_0D):

PS C:\Users\aadit\Contrib\openvino\bin\intel64\Release> ./ov_onnx_frontend_tests --gtest_filter=*softmax_0D*
Running main() from src\frontends\tests\frontend\shared\gtest_main_manifest\main.cpp:20
Note: Google Test filter = *softmax_0D*:-:ONNXLoadTest/FrontEndLoadFromTest.testLoadFromTwoFiles/onnx:ONNXLoadTest/FrontEndLoadFromTest.testLoadFromTwoStreams/onnx
[==========] Running 3 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 1 test from INTERPRETER
[ RUN      ] INTERPRETER.onnx_model_softmax_0D
unknown file: error: C++ exception with description "Exception from src\frontends\onnx\onnx_common\src\parser.cpp:24:
Could not open the file: "C:\Users\aadit\Contrib\openvino\bin\intel64\Release\test_model_zoo\onnx\softmax_0D.onnx"
" thrown in the test body.
[  FAILED  ] INTERPRETER.onnx_model_softmax_0D (1 ms)
[----------] 1 test from INTERPRETER (1 ms total)

[----------] 1 test from IE_GPU
[ RUN      ] IE_GPU.onnx_model_softmax_0D
unknown file: error: C++ exception with description "Exception from src\frontends\onnx\onnx_common\src\parser.cpp:24:
Could not open the file: "C:\Users\aadit\Contrib\openvino\bin\intel64\Release\test_model_zoo\onnx\softmax_0D.onnx"
" thrown in the test body.
[  FAILED  ] IE_GPU.onnx_model_softmax_0D (1 ms)
[----------] 1 test from IE_GPU (1 ms total)

[----------] 1 test from IE_CPU
[ RUN      ] IE_CPU.onnx_model_softmax_0D
unknown file: error: C++ exception with description "Exception from src\frontends\onnx\onnx_common\src\parser.cpp:24:
Could not open the file: "C:\Users\aadit\Contrib\openvino\bin\intel64\Release\test_model_zoo\onnx\softmax_0D.onnx"
" thrown in the test body.
[  FAILED  ] IE_CPU.onnx_model_softmax_0D (1 ms)
[----------] 1 test from IE_CPU (1 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 3 test suites ran. (5 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 3 tests, listed below:
[  FAILED  ] INTERPRETER.onnx_model_softmax_0D
[  FAILED  ] IE_GPU.onnx_model_softmax_0D
[  FAILED  ] IE_CPU.onnx_model_softmax_0D

 3 FAILED TESTS
  YOU HAVE 3 DISABLED TESTS

But from what I gather, there in Could not open the file: "C:\Users\aadit\Contrib\openvino\bin\intel64\Release\test_model_zoo\onnx\softmax_0D.onnx" " thrown in the test body. There is no softmax_0D.onnx file being created.

I'd be really grateful if you could help me pinpoint the issue, because it's been delaying my progress for a long, long time on this ticket.

Many thanks!

[AJThePro99]

Update:
I've done a clean build (i.e. I cloned the repository again, and ran the build accodring to onnx build instructions

But the result is still the same.

...
...
[  FAILED  ] ONNX_Importer_Tests.ImportModelWithMultiOutput (1 ms)
[ RUN      ] ONNX_Importer_Tests.ImportModelWithNotSupportedOp
src\frontends\onnx\tests\onnx_importer_test.cpp(98): error: Value of: std::string("OpenVINO does not support the following ONNX operations: NotSupported")
  Actual: "OpenVINO does not support the following ONNX operations: NotSupported"
Expected: a substring of error.what()
Which is: "Exception from src\frontends\onnx\onnx_common\src\parser.cpp:24:
Could not open the file: "C:\Users\aadit\source\repos\openvino\bin\intel64\Release\./test_model_zoo/onnx/not_supported.onnx"
"
[  FAILED  ] ONNX_Importer_Tests.ImportModelWithNotSupportedOp (0 ms)
[ RUN      ] ONNX_Importer_Tests.ImportModelWhenFileDoesNotExist
[       OK ] ONNX_Importer_Tests.ImportModelWhenFileDoesNotExist (0 ms)
[ RUN      ] ONNX_Importer_Tests.ImportModelFromStream
src\frontends\onnx\tests\onnx_importer_test.cpp(119): error: Value of: model_file_stream.is_open()
  Actual: false
Expected: true
[  FAILED  ] ONNX_Importer_Tests.ImportModelFromStream (1 ms)
...
...

I do not understand why the path diverges to forward slashes while on windows, along with the ./

C:\Users\aadit\source\repos\openvino\bin\intel64\Release\./test_model_zoo/onnx/not_supported.onnx