SparkNLP 1018 - Introducing NLLB

[prabod]

Description

Meta AI has built a single AI model, NLLB-200, that is the first to translate across 200 different languages with state-of-the-art quality that has been validated through extensive evaluations for each of them.

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING page.
• I have added tests to cover my changes.
• All new and existing tests passed.

[coveralls]

Pull Request Test Coverage Report for Build 10656276506

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage decreased (-0.007%) to 62.415%

---

Files with Coverage Reduction	New Missed Lines	%
src/main/scala/com/johnsnowlabs/nlp/annotators/sentence_detector_dl/SentenceDetectorDLModel.scala	1	81.68%

Totals
Change from base Build 10656260334:	-0.007%
Covered Lines:	8969
Relevant Lines:	14370

---

💛 - Coveralls

[yaronshanisima]

HI @prabod and thank you for the contribution!
Do you if this is still a work in progress?
I ask because when I run the example code here I get:
Can not find the model to download please check the name! with just the default pretrained value.
Also, when I try to run a query to SparkNLP model hub, I don't see any NLLB models. Is this something that will be added in the future? Do you have the models that you run tests on?
Thanks!

[maziyarpanahi]

HI @prabod and thank you for the contribution! Do you if this is still a work in progress? I ask because when I run the example code here I get: Can not find the model to download please check the name! with just the default pretrained value. Also, when I try to run a query to SparkNLP model hub, I don't see any NLLB models. Is this something that will be added in the future? Do you have the models that you run tests on? Thanks!

Hi,

This feature will be completed and released in the upcoming version in a week or two. At the moment it is missing some final pieces.