New multitask 9in1 (Singlelabel version) #213
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dilyararimovna
previously approved these changes
Jan 17, 2023
… 15% and to make the compatibility with current DEV version
dilyararimovna
requested changes
Jan 20, 2023
state_formatters/dp_formatters.py
Outdated
| annotations = dialog["human_utterances"][-1].get("annotations", {}) | ||
| if "combined_classification" in annotations: # use combined annotations | ||
| annotations = annotations["combined_classification"] | ||
| midas_dist = annotations.get("midas_classification", {}) |
There was a problem hiding this comment.
надо использовать get_intents из common, а не это
dilyararimovna
approved these changes
Jan 24, 2023
Merged
oserikov
pushed a commit
that referenced
this pull request
Nov 21, 2024
* Update data.json * Update scenario.py * Update test.py * Update tests.json * Update requirements.txt * Update skill.py * Update requirements.txt * Update test_no_annotations.json * Update combined_classifier.json * Codestyle using BLACK * Update utils.py * Update test.py * Update test.py * Update server.py * Update test.py * Update test.py * Update test.py * Update test.py * Update test.py * Update test.py * Update test.py * Update test.py * Update test.py * Update server.py * Update test.py * Update test.py * Update test.py * codestyle * Update utils.py * Renamed topic_classification, deleted unnesessary string * Speeded up the combined classifier * Update Dockerfile * New version of DeepPavlov * Clean new combined - with fixed bug in checkout * Update README.md * Further speeded up multitask BERT model * Update Dockerfile * I have done my best to speed up the multitask inference. * Update utils.py * DeepPavlov version after several fixes. Also, new distil model ( not final, in training!) * hh * Tests fixed * Update test.py * codestyle * Returned cuda cache * Update Dockerfile * Test change for memory profiling * It should work much faster now * It should work much faster now * It should work much faster now * Test editings to tackle test_dialog fail * Update server.py * Update combined_classifier.json * codestyle * Update Dockerfile * Update combined_classifier.json * Current test-passing version * Changed factoid criteria & postprocess for cobot topics and intents * Changed factoid criteria & postprocess for cobot topics and intents * Minor test fix - updated "random skills" list * codestyle * codestyle * Update factoid.py * Update connector.py * Utilize unified prob threshold in factoid skill. Before these changes, 2 different thresholds for factoid were set. One threshold 0.96 to turn the skill on, another 0.9 in the same skill. Given that the current model classifies question "where is montreal" as factoid with 0.55 probability, I have set the threshold 0.5 for both of these cases. And made the one threshold to use in both places to streamline the code * Dilya's suggestion https://github.com/deeppavlov/dream/pull/207/files#r1016588010 * Dilya's suggestion https://github.com/deeppavlov/dream/pull/207/files#r1016569392 * Dilya's suggestions * Dilya's suggestion * Dilya's suggestion * Dilya's comment * Update Dockerfile * Update Dockerfile * Update combined_classifier.json * Update README.md * current changes * Codestyle * Added dependency to fix bug fastapi/typer#377 * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update combined_classifier.json * Update test.py * Update server.py * Update combined_classifier.json * Update server.py * Update test.py * Passes skill tests now * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update Dockerfile * Update server.py * Fix bug * Update Dockerfile * Update dev.yml * Update combined_classifier.json * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Update requirements.txt * Added factoid threshold * Update connector.py * Update server.py * Codestyle * Codestyle * Suggested changes * current version * Fixed sentence len * Added setuptools dependency while numpy 1.18.0 not to fail on build * Added setuptools dependency while numpy 1.18.0 not to fail on build * Still facing bug numpy/numpy#22623 - restrict numpy version * h * Try to fix bug in test_dialog in utils/analyze_downloads.py while importing DeepPavlov - no module named ruamel. We might need to install it in dockerfile * Update utils.py * Update utils.py * Threshold fixes as siggested by Dilya * Cosmetic change * Update combined_classifier.json * fixed tests * codestyle * Update utils.py * Suggested phrase fix * Update dev.yml * Remove empty requirements changes * requirements fix * Update connector.py * Revert prev commit * Names as in original * added cache * Update utils.py * Update utils.py * Update animals.py * Update art.py * Update books.py * Update food.py * Update gaming.py * Update gaming.py * Update music.py * Update art.py * Update travel.py * Update science.py * Update books.py * Update food.py * Update gaming.py * Update gossip.py * Update sport.py * Update sport.py * Update music.py * Update movies.py * codestyle * codestyle * Classname fix in tests * suggested changes * Update test.py * Update test.py * Update Dockerfile * suggested changes * Update Dockerfile * Update combined_classifier.json * Update Dockerfile * Now we can infer for all postannotations * Update Dockerfile * Codestyle. Tests of combined (in old config) are passed 2x faster thanks to the improved caching * Decreased inference time by another 25% without quality loss * fixed tests * codestyle * New log format * Update combined_classifier.json * Changed singletask sentence length from 64 to 32 to speed model up by 15% and to make the compatibility with current DEV version * Update server.py * Remove MIDAS from pipeline * Remove MIDAS from pipeline * remove MIDAS from WAIT_HOSTS * MIDAS from pipeline * Убираю Мидас из WAIT_HOSTS и пайплайна * Убираю Мидас из WAIT_HOSTS и пайплайна * Not referring to midas annotations anymore and not waiting for midas * All combined classification timeouts set to 2 sec * Update dp_formatters.py * Update dp_formatters.py * Midas classification is a dict * Get intents by function from common * Update README.md * codestyle Co-authored-by: dimakarp1996 <dimakarp1996@github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Last singlelabel version of multitask distilbert. Added on Vasily request. PR is interchangeable with #207 , will be merged there if passes the tests