Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2023-02-07-asr_hubert_large_ls960_en #13481

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions docs/_posts/maziyarpanahi/2023-02-07-asr_hubert_large_ls960_en.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
layout: model
title: ASR HubertForCTC - asr_hubert_large_ls960
author: John Snow Labs
name: asr_hubert_large_ls960
date: 2023-02-07
tags: [open_source, hubert, audio, en, english, asr, speech, librispeech_asr, tensorflow]
task: Sentiment Analysis
language: en
edition: Spark NLP 4.3.0
spark_version: 3.0
supported: true
engine: tensorflow
annotator: HubertForCTC
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

Hubert Model with a language modeling head on top for Connectionist Temporal Classification (CTC).
Hubert was proposed in HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units by Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, Abdelrahman Mohamed.

The large model fine-tuned on 960h of Librispeech on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/asr_hubert_large_ls960_en_4.3.0_3.0_1675767067233.zip){:.button.button-orange}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/public/models/asr_hubert_large_ls960_en_4.3.0_3.0_1675767067233.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python

audio_assembler = AudioAssembler()\
.setInputCol("audio_content")\
.setOutputCol("audio_assembler")

speech_to_text = HubertForCTC.pretrained("asr_hubert_large_ls960", "en")\
.setInputCols("audio_assembler")\
.setOutputCol("text")

pipeline = Pipeline(stages=[
audio_assembler,
speech_to_text,
])

pipelineModel = pipeline.fit(audioDf)

pipelineDF = pipelineModel.transform(audioDf)
```
```scala

val audioAssembler = new AudioAssembler()
.setInputCol("audio_content")
.setOutputCol("audio_assembler")

val speechToText = HubertForCTC
.pretrained("asr_hubert_large_ls960", "en")
.setInputCols("audio_assembler")
.setOutputCol("text")

val pipeline = new Pipeline().setStages(Array(audioAssembler, speechToText))

val pipelineModel = pipeline.fit(audioDf)

val pipelineDF = pipelineModel.transform(audioDf)
```
</div>

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|asr_hubert_large_ls960|
|Compatibility:|Spark NLP 4.3.0+|
|License:|Open Source|
|Edition:|Official|
|Input Labels:|[audio_assembler]|
|Output Labels:|[text]|
|Language:|en|
|Size:|1.5 GB|

## References

[https://huggingface.co/facebook/hubert-large-ls960-ft](https://huggingface.co/facebook/hubert-large-ls960-ft)
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
layout: model
title: SwinForImageClassification - image_classifier_swin_base_patch4_window12_384_in22k
author: John Snow Labs
name: image_classifier_swin_base_patch4_window12_384_in22k
date: 2023-02-07
tags: [open_source, swin, image, en, english, image_classification, imagenet, tensorflow]
task: Sentiment Analysis
language: en
edition: Spark NLP 4.3.0
spark_version: 3.0
supported: true
engine: tensorflow
annotator: SwinForImageClassification
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

Pretrained Swin model for Image Classification, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.

Swin Transformer was introduced in the paper [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://arxiv.org/abs/2103.14030) by Liu et al.

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/image_classifier_swin_base_patch4_window12_384_in22k_en_4.3.0_3.0_1675783085913.zip){:.button.button-orange}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/public/models/image_classifier_swin_base_patch4_window12_384_in22k_en_4.3.0_3.0_1675783085913.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python

image_assembler = ImageAssembler()\
.setInputCol("image")
.setOutputCol("image_assembler")

imageClassifier = SwinForImageClassification.pretrained("image_classifier_swin_base_patch4_window12_384_in22k", "en")\
.setInputCols("image_assembler")\
.setOutputCol("class")

pipeline = Pipeline(stages=[
image_assembler,
imageClassifier,
])

pipelineModel = pipeline.fit(imageDF)

pipelineDF = pipelineModel.transform(imageDF)
```
```scala

val imageAssembler = new ImageAssembler()
.setInputCol("image")
.setOutputCol("image_assembler")

val imageClassifier = SwinForImageClassification
.pretrained("image_classifier_swin_base_patch4_window12_384_in22k", "en")
.setInputCols("image_assembler")
.setOutputCol("class")

val pipeline = new Pipeline().setStages(Array(imageAssembler, imageClassifier))

val pipelineModel = pipeline.fit(imageDF)

val pipelineDF = pipelineModel.transform(imageDF)
```
</div>

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|image_classifier_swin_base_patch4_window12_384_in22k|
|Compatibility:|Spark NLP 4.3.0+|
|License:|Open Source|
|Edition:|Official|
|Input Labels:|[image_assembler]|
|Output Labels:|[class]|
|Language:|en|
|Size:|826.8 MB|

## References

[https://huggingface.co/microsoft/swin_base_patch4_window12_384_in22k](https://huggingface.co/microsoft/swin_base_patch4_window12_384_in22k)
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
layout: model
title: SwinForImageClassification - image_classifier_swin_base_patch4_window7_224
author: John Snow Labs
name: image_classifier_swin_base_patch4_window7_224
date: 2023-02-07
tags: [open_source, swin, image, en, english, image_classification, imagenet, tensorflow]
task: Sentiment Analysis
language: en
edition: Spark NLP 4.3.0
spark_version: 3.0
supported: true
engine: tensorflow
annotator: SwinForImageClassification
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

Pretrained Swin model for Image Classification, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.

Swin Transformer was introduced in the paper [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://arxiv.org/abs/2103.14030) by Liu et al.

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/image_classifier_swin_base_patch4_window7_224_en_4.3.0_3.0_1675783112124.zip){:.button.button-orange}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/public/models/image_classifier_swin_base_patch4_window7_224_en_4.3.0_3.0_1675783112124.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python

image_assembler = ImageAssembler()\
.setInputCol("image")\
.setOutputCol("image_assembler")

imageClassifier = SwinForImageClassification.pretrained("image_classifier_swin_base_patch4_window7_224", "en")\
.setInputCols("image_assembler")\
.setOutputCol("class")

pipeline = Pipeline(stages=[
image_assembler,
imageClassifier,
])

pipelineModel = pipeline.fit(imageDF)

pipelineDF = pipelineModel.transform(imageDF)
```
```scala

val imageAssembler = new ImageAssembler()
.setInputCol("image")
.setOutputCol("image_assembler")

val imageClassifier = SwinForImageClassification
.pretrained("image_classifier_swin_base_patch4_window7_224", "en")
.setInputCols("image_assembler")
.setOutputCol("class")

val pipeline = new Pipeline().setStages(Array(imageAssembler, imageClassifier))

val pipelineModel = pipeline.fit(imageDF)

val pipelineDF = pipelineModel.transform(imageDF)
```
</div>

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|image_classifier_swin_base_patch4_window7_224|
|Compatibility:|Spark NLP 4.3.0+|
|License:|Open Source|
|Edition:|Official|
|Input Labels:|[image_assembler]|
|Output Labels:|[class]|
|Language:|en|
|Size:|665.2 MB|

## References

[https://huggingface.co/microsoft/swin_base_patch4_window7_224](https://huggingface.co/microsoft/swin_base_patch4_window7_224)
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
layout: model
title: SwinForImageClassification - image_classifier_swin_base_patch4_window7_224_in22k
author: John Snow Labs
name: image_classifier_swin_base_patch4_window7_224_in22k
date: 2023-02-07
tags: [open_source, swin, image, en, english, image_classification, imagenet, tensorflow]
task: Sentiment Analysis
language: en
edition: Spark NLP 4.3.0
spark_version: 3.0
supported: true
engine: tensorflow
annotator: SwinForImageClassification
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

Pretrained Swin model for Image Classification, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.

Swin Transformer was introduced in the paper [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://arxiv.org/abs/2103.14030) by Liu et al.

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/image_classifier_swin_base_patch4_window7_224_in22k_en_4.3.0_3.0_1675783139673.zip){:.button.button-orange}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/public/models/image_classifier_swin_base_patch4_window7_224_in22k_en_4.3.0_3.0_1675783139673.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python

image_assembler = ImageAssembler()\
.setInputCol("image")\
.setOutputCol("image_assembler")

imageClassifier = SwinForImageClassification.pretrained("image_classifier_swin_base_patch4_window7_224_in22k", "en")\
.setInputCols("image_assembler")\
.setOutputCol("class")

pipeline = Pipeline(stages=[
image_assembler,
imageClassifier,
])

pipelineModel = pipeline.fit(imageDF)

pipelineDF = pipelineModel.transform(imageDF)
```
```scala

val imageAssembler = new ImageAssembler()
.setInputCol("image")
.setOutputCol("image_assembler")

val imageClassifier = SwinForImageClassification
.pretrained("image_classifier_swin_base_patch4_window7_224_in22k", "en")
.setInputCols("image_assembler")
.setOutputCol("class")

val pipeline = new Pipeline().setStages(Array(imageAssembler, imageClassifier))

val pipelineModel = pipeline.fit(imageDF)

val pipelineDF = pipelineModel.transform(imageDF)
```
</div>

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|image_classifier_swin_base_patch4_window7_224_in22k|
|Compatibility:|Spark NLP 4.3.0+|
|License:|Open Source|
|Edition:|Official|
|Input Labels:|[image_assembler]|
|Output Labels:|[class]|
|Language:|en|
|Size:|825.5 MB|

## References

[https://huggingface.co/microsoft/swin_base_patch4_window7_224_in22k](https://huggingface.co/microsoft/swin_base_patch4_window7_224_in22k)
Loading