Models hub

docs/_posts/DevinTDHa/2023-08-28-asr_whisper_tiny_opt_xx.md

@@ -0,0 +1,119 @@
+---
+layout: model
+title: Official whisper-tiny Optimized
+author: John Snow Labs
+name: asr_whisper_tiny_opt
+date: 2023-08-28
+tags: [whisper, audio, open_source, asr, onnx, xx]
+task: Automatic Speech Recognition
+language: xx
+edition: Spark NLP 5.1.1
+spark_version: 3.0
+supported: true
+engine: onnx
+annotator: WhisperForCTC
+article_header:
+  type: cover
+use_language_switcher: "Python-Scala-Java"
+---
+
+## Description
+
+Official pretrained Whisper model, adapted from HuggingFace transformer and curated to provide scalability and production-readiness using Spark NLP.
+
+This is a multilingual model and supports the following languages:
+
+Afrikaans, Arabic, Armenian, Azerbaijani, Belarusian, Bosnian, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Galician, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Italian, Japanese, Kannada, Kazakh, Korean, Latvian, Lithuanian, Macedonian, Malay, Marathi, Maori, Nepali, Norwegian, Persian, Polish, Portuguese, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tagalog, Tamil, Thai, Turkish, Ukrainian, Urdu, Vietnamese, and Welsh.
+
+## Predicted Entities
+
+
+
+{:.btn-box}
+<button class="button button-orange" disabled>Live Demo</button>
+<button class="button button-orange" disabled>Open in Colab</button>
+[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/asr_whisper_tiny_opt_xx_5.1.1_3.0_1693213918398.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
+[Copy S3 URI](s3://auxdata.johnsnowlabs.com/public/models/asr_whisper_tiny_opt_xx_5.1.1_3.0_1693213918398.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}
+
+## How to use
+
+
+
+<div class="tabs-box" markdown="1">
+{% include programmingLanguageSelectScalaPythonNLU.html %}
+```python
+import sparknlp
+from sparknlp.base import *
+from sparknlp.annotator import *
+from pyspark.ml import Pipeline
+
+audioAssembler = AudioAssembler() \
+    .setInputCol("audio_content") \
+    .setOutputCol("audio_assembler")
+
+speechToText = WhisperForCTC.pretrained("asr_whisper_tiny_opt", "xx") \
+    .setInputCols(["audio_assembler"]) \
+    .setOutputCol("text")
+
+pipeline = Pipeline().setStages([audioAssembler, speechToText])
+processedAudioFloats = spark.createDataFrame([[rawFloats]]).toDF("audio_content")
+result = pipeline.fit(processedAudioFloats).transform(processedAudioFloats)
+result.select("text.result").show(truncate = False)
+```
+```scala
+import spark.implicits._
+import com.johnsnowlabs.nlp.base._
+import com.johnsnowlabs.nlp.annotators._
+import com.johnsnowlabs.nlp.annotators.audio.WhisperForCTC
+import org.apache.spark.ml.Pipeline
+
+val audioAssembler: AudioAssembler = new AudioAssembler()
+  .setInputCol("audio_content")
+  .setOutputCol("audio_assembler")
+
+val speechToText: WhisperForCTC = WhisperForCTC
+  .pretrained("asr_whisper_tiny_opt", "xx") 
+  .setInputCols("audio_assembler")
+  .setOutputCol("text")
+
+val pipeline: Pipeline = new Pipeline().setStages(Array(audioAssembler, speechToText))
+
+val bufferedSource =
+  scala.io.Source.fromFile("src/test/resources/audio/txt/librispeech_asr_0.txt")
+
+val rawFloats = bufferedSource
+  .getLines()
+  .map(_.split(",").head.trim.toFloat)
+  .toArray
+bufferedSource.close
+
+val processedAudioFloats = Seq(rawFloats).toDF("audio_content")
+
+val result = pipeline.fit(processedAudioFloats).transform(processedAudioFloats)
+result.select("text.result").show(truncate = false)
+```
+</div>
+
+## Results
+
+```bash
++------------------------------------------------------------------------------------------------------------------------------------------------+
+|document                                                                                                                                        |
++------------------------------------------------------------------------------------------------------------------------------------------------+
+|[{document, 0, 87,  Mr. Quilter is the apostle of the middle classes and we are glad to welcome his gospel., {length -> 93680, audio -> 0}, []}]|
++------------------------------------------------------------------------------------------------------------------------------------------------+
+```
+
+{:.model-param}
+## Model Information
+
+{:.table-model}
+|---|---|
+|Model Name:|asr_whisper_tiny_opt|
+|Compatibility:|Spark NLP 5.1.1+|
+|License:|Open Source|
+|Edition:|Official|
+|Input Labels:|[audio_assembler]|
+|Output Labels:|[document]|
+|Language:|xx|
+|Size:|239.3 MB|

docs/_posts/DevinTDHa/2023-09-20-image_captioning_vit_gpt2_en.md

@@ -0,0 +1,125 @@
+---
+layout: model
+title: Image Caption with VisionEncoderDecoder ViT GPT2
+author: John Snow Labs
+name: image_captioning_vit_gpt2
+date: 2023-09-20
+tags: [en, vit, gpt2, image, captioning, open_source, tensorflow]
+task: Image Captioning
+language: en
+edition: Spark NLP 5.1.2
+spark_version: 3.0
+supported: true
+engine: tensorflow
+annotator: VisionEncoderDecoderForImageCaptioning
+article_header:
+  type: cover
+use_language_switcher: "Python-Scala-Java"
+---
+
+## Description
+
+This is an image captioning model using ViT to encode images and GPT2 to generate captions. Original model from https://huggingface.co/nlpconnect/vit-gpt2-image-captioning
+
+## Predicted Entities
+
+
+
+{:.btn-box}
+<button class="button button-orange" disabled>Live Demo</button>
+<button class="button button-orange" disabled>Open in Colab</button>
+[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/image_captioning_vit_gpt2_en_5.1.2_3.0_1695215721202.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
+[Copy S3 URI](s3://auxdata.johnsnowlabs.com/public/models/image_captioning_vit_gpt2_en_5.1.2_3.0_1695215721202.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}
+
+## How to use
+
+
+
+<div class="tabs-box" markdown="1">
+{% include programmingLanguageSelectScalaPythonNLU.html %}
+```python
+import sparknlp
+from sparknlp.base import *
+from sparknlp.annotator import *
+from pyspark.ml import Pipeline
+imageDF = spark.read \
+    .format("image") \
+    .option("dropInvalid", value = True) \
+    .load("src/test/resources/image/")
+imageAssembler = ImageAssembler() \
+    .setInputCol("image") \
+    .setOutputCol("image_assembler")
+imageCaptioning = VisionEncoderDecoderForImageCaptioning \
+    .pretrained() \
+    .setBeamSize(2) \
+    .setDoSample(False) \
+    .setInputCols(["image_assembler"]) \
+    .setOutputCol("caption")
+pipeline = Pipeline().setStages([imageAssembler, imageCaptioning])
+pipelineDF = pipeline.fit(imageDF).transform(imageDF)
+pipelineDF \
+    .selectExpr("reverse(split(image.origin, '/'))[0] as image_name", "caption.result") \
+    .show(truncate = False)
+```
+```scala
+import com.johnsnowlabs.nlp.annotator._
+import com.johnsnowlabs.nlp.ImageAssembler
+import org.apache.spark.ml.Pipeline
+
+val imageDF: DataFrame = spark.read
+  .format("image")
+  .option("dropInvalid", value = true)
+  .load("src/test/resources/image/")
+
+val imageCaptioning = new ImageAssembler()
+  .setInputCol("image")
+  .setOutputCol("image_assembler")
+
+val imageClassifier = VisionEncoderDecoderForImageCaptioning
+  .pretrained()
+  .setBeamSize(2)
+  .setDoSample(false)
+  .setInputCols("image_assembler")
+  .setOutputCol("caption")
+
+val pipeline = new Pipeline().setStages(Array(imageAssembler, imageCaptioning))
+val pipelineDF = pipeline.fit(imageDF).transform(imageDF)
+
+pipelineDF
+  .selectExpr("reverse(split(image.origin, '/'))[0] as image_name", "caption.result")
+  .show(truncate = false)
+```
+</div>
+
+## Results
+
+```bash
++-----------------+---------------------------------------------------------+
+|image_name       |result                                                   |
++-----------------+---------------------------------------------------------+
+|palace.JPEG      |[a large room filled with furniture and a large window]  |
+|egyptian_cat.jpeg|[a cat laying on a couch next to another cat]            |
+|hippopotamus.JPEG|[a brown bear in a body of water]                        |
+|hen.JPEG         |[a flock of chickens standing next to each other]        |
+|ostrich.JPEG     |[a large bird standing on top of a lush green field]     |
+|junco.JPEG       |[a small bird standing on a wet ground]                  |
+|bluetick.jpg     |[a small dog standing on a wooden floor]                 |
+|chihuahua.jpg    |[a small brown dog wearing a blue sweater]               |
+|tractor.JPEG     |[a man is standing in a field with a tractor]            |
+|ox.JPEG          |[a large brown cow standing on top of a lush green field]|
++-----------------+---------------------------------------------------------+
+```
+
+{:.model-param}
+## Model Information
+
+{:.table-model}
+|---|---|
+|Model Name:|image_captioning_vit_gpt2|
+|Compatibility:|Spark NLP 5.1.2+|
+|License:|Open Source|
+|Edition:|Official|
+|Input Labels:|[image_assembler]|
+|Output Labels:|[caption]|
+|Language:|en|
+|Size:|890.3 MB|

docs/_posts/LIN-Yu-Ting/2023-09-18-AtgxRobertaBaseSquad2_en.md

@@ -0,0 +1,61 @@
+---
+layout: model
+title: Atgenomix Testing QA Model
+author: LIN-Yu-Ting
+name: AtgxRobertaBaseSquad2
+date: 2023-09-18
+tags: [en, open_source, tensorflow]
+task: Question Answering
+language: en
+edition: Spark NLP 4.4.3
+spark_version: 3.4
+supported: false
+engine: tensorflow
+annotator: RoBertaForQuestionAnswering
+article_header:
+  type: cover
+use_language_switcher: "Python-Scala-Java"
+---
+
+## Description
+
+Testing Question Answering model for Atgenomix usage
+
+## Predicted Entities
+
+
+
+{:.btn-box}
+<button class="button button-orange" disabled>Live Demo</button>
+<button class="button button-orange" disabled>Open in Colab</button>
+[Download](https://s3.amazonaws.com/community.johnsnowlabs.com/LIN-Yu-Ting/AtgxRobertaBaseSquad2_en_4.4.3_3.4_1695000774804.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
+[Copy S3 URI](s3://community.johnsnowlabs.com/LIN-Yu-Ting/AtgxRobertaBaseSquad2_en_4.4.3_3.4_1695000774804.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}
+
+## How to use
+
+
+
+<div class="tabs-box" markdown="1">
+{% include programmingLanguageSelectScalaPythonNLU.html %}
+```python
+import sparknlp
+spark = sparknlp.start()
+```
+
+</div>
+
+{:.model-param}
+## Model Information
+
+{:.table-model}
+|---|---|
+|Model Name:|AtgxRobertaBaseSquad2|
+|Compatibility:|Spark NLP 4.4.3+|
+|License:|Open Source|
+|Edition:|Community|
+|Input Labels:|[document_question, document_context]|
+|Output Labels:|[answer]|
+|Language:|en|
+|Size:|460.0 MB|
+|Case sensitive:|true|
+|Max sentence length:|512|