Skip to content

Commit

Permalink
added info on prompts and playground (#13498)
Browse files Browse the repository at this point in the history
Co-authored-by: diatrambitas <JSL.Git2018>
  • Loading branch information
diatrambitas authored and Jose J. Martinez committed Feb 17, 2023
1 parent 99fe627 commit 3c42c59
Show file tree
Hide file tree
Showing 4 changed files with 97 additions and 1 deletion.
4 changes: 4 additions & 0 deletions docs/_data/navigation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ annotation-lab:
url: /docs/en/alab/embeddings
- title: Rules
url: /docs/en/alab/rules
- title: Prompts
url: /docs/en/alab/prompts
- title: Playground
url: /docs/en/alab/playground
- subtitle: Project
children:
- title: Dashboard
Expand Down
44 changes: 44 additions & 0 deletions docs/en/alab/playground.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
layout: docs
comment: no
header: true
seotitle: NLP Lab | John Snow Labs
title: Playground
permalink: /docs/en/alab/playground
key: docs-training
modify_date: "2023-02-10"
use_language_switcher: "Python-Scala"
show_nav: true
sidebar:
nav: annotation-lab
---

<style>
es {
font-weight:400;
font-style: italic;
}
</style>

The Playground feature of the NLP Lab allows users to deploy and test models, rules, and/or prompts without going through the project setup wizard. This simplifies the initial resources exploration, and facilitates experiments on custom data. Any model, rule, or prompt can now be selected and deployed for testing by clicking on the "Open in Playground" button.


## Experiment with Rules

Rules can be deployed to the Playground from the rules page. When a particular rule is deployed in the playground, the user can also change the parameters of the rules via the rule definition form from the right side of the page. After saving the changes users need to click on the "Deploy" button to refresh the results of the pre-annotation on the provided text.

![ruledeploy](https://user-images.githubusercontent.com/33893292/209978724-355fd3a2-aab4-4c38-869d-4f3432c657d3.gif)

## Experiment with Prompts

NLP Lab's Playground also supports the deployment and testing of prompts. Users can quickly test the results of applying a prompt on custom text, can easily edit the prompt, save it, and deploy it right away to see the change in the pre-annotation results.

![demo3](https://user-images.githubusercontent.com/33893292/213699722-543d13f6-c410-4398-83a1-26a832a032ca.gif)


## Experiment with Models
Any Classification, NER or Assertion Status model available on the NLP Lab can also be deployed to Playground for testing on custom text.

![Playground deployment](https://user-images.githubusercontent.com/33893292/209965776-1c0a6b07-5526-496e-97f8-4b7a2cf2a6d1.gif)

Deployment of models and rules is supported by floating and air-gapped licenses. Healthcare, Legal, and Finance models require a license with their respective scopes to be deployed in Playground. Unlike pre-annotation servers, only one playground can be deployed at any given time.
48 changes: 48 additions & 0 deletions docs/en/alab/prompts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
layout: docs
comment: no
header: true
seotitle: NLP Lab | John Snow Labs
title: Prompts
permalink: /docs/en/alab/prompts
key: docs-training
modify_date: "2023-02-10"
use_language_switcher: "Python-Scala"
show_nav: true
sidebar:
nav: annotation-lab
---

<style>
es {
font-weight:400;
font-style: italic;
}
</style>

NLP Lab offers support for prompt engineering. On the `Prompts` page, from the resources `HUB`, users can easily discover and explore the existing prompts or create new prompts for identifying entities or relations. Currently, NLP Lab supports prompts for Healthcare, Finance, and Legal domains applied using pre-trained question-answering language models published on the NLP Models Hub and available to download in one click. The main advantage behind the use of prompts in entity or relation recognition is the ease of definition. Non-technical domain experts can easily create prompts, test and edit them on the `Playground` on custom text snippets and, when ready, deploy them for pre-annotation as part of larger NLP projects. Together with rules, prompts are very handy in situations where no pre-trained models exist, for the target entities and domains. With rules and prompts the annotators never start their projects from scratch but can capitalize on the power of zero-shot models and rules to help them pre-annotate the simple entities and relations and speed up the annotation process. As such, the NLP Lab ensures fewer manual annotations are required from any given task.

## Creating NER Prompts

NER prompts, can be used to identify entities in natural language text documents. Those can be created based on healthcare, finance, and legal zero-shot models selectable from the "Domain" dropdown. For one prompt, the user adds one or more questions for which the answer represents the target entity to annotate.

![entity_prompt](https://user-images.githubusercontent.com/26042994/211890279-2ea02cd5-36fa-4b56-86fd-38b0c20ba880.gif)

## Creating Relation Prompts

Prompts can also be used to identify relations between entities for healthcare, finance, and legal domains. The domain-specific zero-shot model to use for detecting relation can be selected from the "Domain" dropdown. The relation prompts are defined by a pair of entities related by a predicate. The entities can be selected from the available dropdowns listing all entities available in the current NLP Lab (included in available NER models or rules) for the specified domain.

![relation_prompt](https://user-images.githubusercontent.com/26042994/211890317-362f193c-b80b-4caa-b242-69df6fa8a257.gif)

## Mix and Match models, rules, and prompts

The project configuration page was simplified by grouping into one page all available resources that can be reused for pre-annotation: models, rules, and prompts. Users can easily mix and match the relevant resources and add them to their configuration.

![updated_configuration_page](https://user-images.githubusercontent.com/26042994/211890361-14c5b17c-762d-4d0a-a6a6-0ac235565aa0.gif)

**Note:** One project configuration can only reuse the prompts defined by one single zero-shot model. Prompts created based on multiple zero-shot models (e.g. finance or legal or healthcare) cannot be mixed into the same project because of high resource consumption. Furthermore, all prompts require a license with a scope that matches the domain of the prompt.

## Zero-Shot Models available in the NLP Models Hub
NLP Models Hub now lists the newly released zero-shot models that are used to define prompts. These models need to be downloaded to NLP Lab instance before prompts can be created. A valid license must be available for the models to be downloaded to NLP Lab.

![Zero-shot-models](https://user-images.githubusercontent.com/26042994/211890478-3aa90dfc-f474-42c8-a73f-ce6c3efecbbe.png)
2 changes: 1 addition & 1 deletion docs/en/alab/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: docs
comment: no
header: true
seotitle: Annotation Lab | John Snow Labs
seotitle: NLP Lab | John Snow Labs
title: Rules
permalink: /docs/en/alab/rules
key: docs-training
Expand Down

0 comments on commit 3c42c59

Please sign in to comment.