Skip to content

Spark NLP 5.0.0: Introducing ONNX Support, State-of-the-Art Instructor Embeddings, E5 Text Embeddings, Document Similarity Ranker, and Much More!
Compare
Choose a tag to compare
@maziyarpanahi maziyarpanahi released this 03 Jul 20:54
· 376 commits to master since this release
5.0.0
cf9b75e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

📢 It's All About That Search!

We are delighted to announce the release of Spark NLP 🚀 5.0.0, featuring the highly anticipated support for ONNX! From the start of 2023, we have been working tirelessly to ensure that the integration of ONNX is not just possible but also seamless for all our users. With this support, you can look forward to faster inference, automatic optimization, and quantization when exporting your LLM models. Additionally, we are also set to release an array of new LLM models fine-tuned specifically for chat and instruction, now that we have successfully integrated ONNX Runtime into Spark NLP 🚀.

We have introduced two state-of-the-art models for text embedding, INSTRUCTOR and E5 embeddings. Currently, these models are leading the way on the MTEB leaderboard, even outperforming the widely recognized OpenAI text-embedding-ada-002. These cutting-edge models are now being utilized in production environments to populate Vector Databases. In addition, they are being paired with LLM models like Falcon, serving to augment their existing knowledge base and reduce the chances of hallucinations.

We want to thank our community for their valuable feedback, feature requests, and contributions. Our Models Hub now contains over 18,000+ free and truly open-source models & pipelines. 🎉

🔥 New Features

Spark NLP ❤️ ONNX

SPARK NLP

NEW: Introducing support for ONNX Runtime in Spark NLP🚀. Serving as a high-performance inference engine, ONNX Runtime can handle machine learning models in the ONNX format and has been proven to significantly boost inference performance across a multitude of models.

Our integration of ONNX Runtime has already led to substantial improvements when serving our LLM models, including BERT. Furthermore, this integration empowers Spark NLP users to optimize their model performance. As users export their models to ONNX, they can utilize the built-in features provided by libraries such as onnx-runtime, transformers, optimum, and PyTorch. Notably, these libraries offer out-of-the-box capabilities for optimization and quantization, enhancing model efficiency and performance.

image

In the realm of Vector Databases, the quest for faster and more efficient Embeddings models has become an imperative pursuit. Models like BERT, DistilBERT, and DeBERTa have revolutionized natural language processing tasks by capturing intricate semantic relationships between words. However, their computational demands and slow inference times pose significant challenges in the game of Vector Databases.

In Vector Databases, the speed at which queries are processed and embeddings are retrieved directly impacts the overall performance and responsiveness of the system. As these databases store vast amounts of vectorized data, such as documents, sentences, or entities, swiftly retrieving relevant embeddings becomes paramount. It enables real-time applications like search engines, recommendation systems, sentiment analysis, and chat/instruct-like products similar to ChatGPT to deliver timely and accurate results, ensuring a seamless user experience.

Keeping this in mind, we've initiated ONNX support for the following annotators:

  • We've introduced ONNX support for the BertEmbeddings annotator. Approximately 180 models of the same name have already been converted to the ONNX format to automatically benefit from the associated performance enhancements.
  • We've added ONNX support for the RoBertaEmbeddings annotator. Roughly 55 models of the same name have been imported in the ONNX format, thus allowing for automatic speed improvements.
  • ONNX support has been initiated for the DistilBertEmbeddings annotator. Around 25 models with the same name have been converted to the ONNX format, facilitating automatic speed enhancements.
  • We've incorporated ONNX support into the DeBertaEmbeddings annotator. About 12 models bearing the same name have been imported in the ONNX format, enabling them to automatically reap the benefits of speed improvements.

We have successfully identified all existing models for these annotators on our Models Hub, converted them into the ONNX format, and re-uploaded them under the same names. This process was carried out to ensure a seamless transition for our community starting with Spark NLP 5.0.0. We will continue to import additional models in the ONNX format in the days ahead. To keep track of the ONNX compatibility with Spark NLP, follow this issue: #13866.

INSTRUCTOR: Instruction-Finetuned Text Embeddings

NEW: Introducing InstructorEmbeddings annotator in Spark NLP 🚀. InstructorEmbeddings can load new state-of-the-art INSTRUCTOR Models inherited from Google T5 for Text embedding.

This annotator is compatible with all the models trained/fine-tuned by using T5EncoderModel for PyTorch or TFT5EncoderModel for TensorFlow models in HuggingFace 🤗

image

Instructor👨‍🏫, an instruction-finetuned text embedding model that can generate text embeddings tailored to any task (e.g., classification, retrieval, clustering, text evaluation, etc.) and domains (e.g., science, finance, etc.) by simply providing the task instruction, without any finetuning. Instructor👨‍ achieves sota on 70 diverse embedding tasks!
For more details, check out the official paper and the project page!

NOTEBOOK: https://github.com/JohnSnowLabs/spark-nlp/blob/master/examples/python/annotation/text/english/sentence-embeddings/InstructorEmbeddings.ipynb

E5: Text Embeddings by Weakly-Supervised Contrastive Pre-training

NEW: Introducing E5Embeddings annotator in Spark NLP 🚀. E5Embeddings can load new state-of-the-art E5 Models based on BERT for Text Embeddings.

image

Text Embeddings by Weakly-Supervised Contrastive Pre-training.
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022

NOTEBOOK: https://github.com/JohnSnowLabs/spark-nlp/blob/master/examples/python/annotation/text/english/sentence-embeddings/E5Embeddings.ipynb

Our new state-of-the-art annotators for Text Embeddings are currently dominating the top of the MTEB leaderboard positioning themselves above OpenAI text-embedding-ada-002
image

Massive Text Embedding Benchmark (MTEB) Leaderboard. To submit, refer to the MTEB GitHub repository 🤗

Document Similarity Ranker by LSH techniques

NEW: Introducing DocumentSimilarityRanker annotator in Spark NLP 🚀. DocumentSimilarityRanker is a new annotator that uses LSH techniques present in Spark ML lib to execute approximate nearest neighbors search on top of sentence embeddings, It aims to capture the semantic meaning of a document in a dense, continuous vector space and return it to the ranker search.

NOTEBOOK: https://github.com/JohnSnowLabs/spark-nlp/blob/master/examples/python/annotation/text/english/text-similarity/doc-sim-ranker/test_doc_sim_ranker.ipynb

  • Welcoming 6 new Databricks runtimes to our Spark NLP family:
    • Databricks 13.1 LTS
    • Databricks 13.1 LTS ML
    • Databricks 13.1 LTS ML GPU
    • Databricks 13.2 LTS
    • Databricks 13.2 LTS ML
    • Databricks 13.2 LTS ML GPU
  • Welcome AWS EMR 6.11 version to our Spark NLP family
  • Fix BART issue with input longer than the maxInputLength

💾 Models

Spark NLP 5.0.0 comes with more than 400+ new Large Language Models (LLMs) in ONNX format. We are also providing optimized and quantized versions of popular models that can be used immediately in any Spark NLP pipelines:

Featured Models

Model Name Lang
BertEmbeddings bert_base_cased en
BertEmbeddings bert_base_cased_opt en
BertEmbeddings bert_base_cased_quantized en
BertEmbeddings small_bert_L2_768 en
BertEmbeddings small_bert_L2_768_opt en
BertEmbeddings small_bert_L2_768_quantized en
DeBertaEmbeddings roberta_base en
DeBertaEmbeddings roberta_base_opt en
DeBertaEmbeddings roberta_base_quantized en
DistilBertEmbeddings distilbert_base_cased en
DistilBertEmbeddings distilbert_base_cased_opt en
DistilBertEmbeddings distilbert_base_cased_quantized en
DistilBertEmbeddings distilbert_base_uncased en
DistilBertEmbeddings distilbert_base_uncased_opt en
DistilBertEmbeddings distilbert_base_uncased_quantized en
DistilBertEmbeddings distilbert_base_multilingual_cased en
DistilBertEmbeddings distilbert_base_multilingual_cased_opt en
DistilBertEmbeddings distilbert_base_multilingual_cased_quantized en
RoBertaBertEmbeddings roberta_base en
RoBertaBertEmbeddings roberta_base_opt en
RoBertaBertEmbeddings roberta_base_quantized en
InstructorEmbeddings instructor_base en
InstructorEmbeddings instructor_large en
E5Embeddings e5_small en
E5Embeddings e5_small_v2 en
E5Embeddings e5_base en
E5Embeddings e5_base_v2 en
E5Embeddings e5_large en
E5Embeddings e5_large_v2 en

The complete list of all 18000+ models & pipelines in 230+ languages is available on Models Hub

📓 New Notebooks

Notebooks Colab
Instructor Embeddings Open In Colab
E5 Embeddings Open In Colab
Document Similarity Ranker Open In Colab

📖 Documentation

❤️ Community support

  • Slack For live discussion with the Spark NLP community and the team
  • GitHub Bug reports, feature requests, and contributions
  • Discussions Engage with other community members, share ideas,
    and show off how you use Spark NLP!
  • Medium Spark NLP articles
  • JohnSnowLabs official Medium
  • YouTube Spark NLP video tutorials

Installation

Python

#PyPI

pip install spark-nlp==5.0.0

Spark Packages

spark-nlp on Apache Spark 3.0.x, 3.1.x, 3.2.x, 3.3.x, and 3.4.x (Scala 2.12):

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.0.0

pyspark --packages com.johnsnowlabs.nlp:spark-nlp_2.12:5.0.0

GPU

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.0.0

pyspark --packages com.johnsnowlabs.nlp:spark-nlp-gpu_2.12:5.0.0

Apple Silicon (M1 & M2)

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-silicon_2.12:5.0.0

pyspark --packages com.johnsnowlabs.nlp:spark-nlp-silicon_2.12:5.0.0

AArch64

spark-shell --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:5.0.0

pyspark --packages com.johnsnowlabs.nlp:spark-nlp-aarch64_2.12:5.0.0

Maven

spark-nlp on Apache Spark 3.0.x, 3.1.x, 3.2.x, 3.3.x, and 3.4.x:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp_2.12</artifactId>
    <version>5.0.0</version>
</dependency>

spark-nlp-gpu:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-gpu_2.12</artifactId>
    <version>5.0.0</version>
</dependency>

spark-nlp-silicon:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-silicon_2.12</artifactId>
    <version>5.0.0</version>
</dependency>

spark-nlp-aarch64:

<dependency>
    <groupId>com.johnsnowlabs.nlp</groupId>
    <artifactId>spark-nlp-aarch64_2.12</artifactId>
    <version>5.0.0</version>
</dependency>

FAT JARs

What's Changed

Full Changelog: 4.4.4...5.0.0

Contributors

maziyarpanahi, prabod, and 3 other contributors
Assets 2
Loading
5 Join discussion