SPARKNLP-1036: Onnx Example notebooks #14234
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings
1 task
maziyarpanahi
approved these changes
May 24, 2024
Merged
maziyarpanahi
added a commit
that referenced
this pull request
Jun 12, 2024
) * SPARKNLP-1036: Onnx Example notebooks (#14234) * SPARKNLP-1036: Fix dev python kernel names * SPARKNLP-1036: Bump transformers version * SPARKNLP-1036: Fix Colab buttons * SPARKNLP-1036: Pin onnx version for compatibility * SPARKNLP-1036: Upgrade Spark version * SPARKNLP-1036: Minor Fixes * SPARKNLP-1036: Clean Metadata * SPARKNLP-1036: Add/Adjust Documentation - Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings * Fixies (#14307) * adding fix for broken annotators --------- Co-authored-by: Devin Ha <33089471+DevinTDHa@users.noreply.github.com> Co-authored-by: Lev <agsfer@gmail.com> Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
maziyarpanahi
pushed a commit
that referenced
this pull request
Jun 21, 2024
* SPARKNLP-1036: Onnx Example notebooks (#14234) * SPARKNLP-1036: Fix dev python kernel names * SPARKNLP-1036: Bump transformers version * SPARKNLP-1036: Fix Colab buttons * SPARKNLP-1036: Pin onnx version for compatibility * SPARKNLP-1036: Upgrade Spark version * SPARKNLP-1036: Minor Fixes * SPARKNLP-1036: Clean Metadata * SPARKNLP-1036: Add/Adjust Documentation - Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings * Fixies (#14307) * Fix models link --------- Co-authored-by: Devin Ha <33089471+DevinTDHa@users.noreply.github.com> Co-authored-by: Lev <agsfer@gmail.com>
maziyarpanahi
added a commit
that referenced
this pull request
Jun 21, 2024
* SPARKNLP-1036: Onnx Example notebooks (#14234) * SPARKNLP-1036: Fix dev python kernel names * SPARKNLP-1036: Bump transformers version * SPARKNLP-1036: Fix Colab buttons * SPARKNLP-1036: Pin onnx version for compatibility * SPARKNLP-1036: Upgrade Spark version * SPARKNLP-1036: Minor Fixes * SPARKNLP-1036: Clean Metadata * SPARKNLP-1036: Add/Adjust Documentation - Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings * Fixies (#14307) * uploading UAEEmbeddings notebook Adding UAEEmbeddings example notebook --------- Co-authored-by: Devin Ha <33089471+DevinTDHa@users.noreply.github.com> Co-authored-by: Lev <agsfer@gmail.com> Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
maziyarpanahi
added a commit
that referenced
this pull request
Jun 28, 2024
* SPARKNLP-1036: Onnx Example notebooks (#14234) * SPARKNLP-1036: Fix dev python kernel names * SPARKNLP-1036: Bump transformers version * SPARKNLP-1036: Fix Colab buttons * SPARKNLP-1036: Pin onnx version for compatibility * SPARKNLP-1036: Upgrade Spark version * SPARKNLP-1036: Minor Fixes * SPARKNLP-1036: Clean Metadata * SPARKNLP-1036: Add/Adjust Documentation - Note for supported Spark Version of Annotators - added missing Documentation for BGEEmbeddings * Fixies (#14307) * refactor OpenAIEmbeddings in Scala * refactor OpenAIEmbeddings in Python * add pytest.mark.slow and improve doc --------- Co-authored-by: Devin Ha <33089471+DevinTDHa@users.noreply.github.com> Co-authored-by: Lev <agsfer@gmail.com> Co-authored-by: Maziyar Panahi <maziyar.panahi@iscpif.fr>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes all issues with the ONNX example notebooks and adjusts the documentation of the annotators if necessary.
The following issues have been fixed:
keras.enginecan't be imported4.34.1Unsupported model IR version: 10, max supported IR version: 9onnx==1.14.0java.lang.NoSuchMethodError: 'breeze.storage.Zero breeze.storage.Zero$.FloatZero()Motivation and Context
Users should be able to use these export notebooks without any issues.
How Has This Been Tested?
Run in Colab, all notebooks run without issues.
Screenshots (if appropriate):
Types of changes
Checklist: