-
Notifications
You must be signed in to change notification settings - Fork 717
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SPARKNLP-835: ProtectedParam and ProtectedFeature #13797
Merged
maziyarpanahi
merged 7 commits into
JohnSnowLabs:release/443-release-candidate
from
DevinTDHa:feature/SPARKNLP-835-setProtected
May 25, 2023
Merged
SPARKNLP-835: ProtectedParam and ProtectedFeature #13797
maziyarpanahi
merged 7 commits into
JohnSnowLabs:release/443-release-candidate
from
DevinTDHa:feature/SPARKNLP-835-setProtected
May 25, 2023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DevinTDHa
added
enhancement
new-feature
Introducing a new feature
DON'T MERGE
Do not merge this PR
labels
May 11, 2023
DevinTDHa
changed the title
SPARKNLP-835: ProtectedParam and ProtectedFeature
[WIP] SPARKNLP-835: ProtectedParam and ProtectedFeature
May 11, 2023
DevinTDHa
changed the title
[WIP] SPARKNLP-835: ProtectedParam and ProtectedFeature
SPARKNLP-835: ProtectedParam and ProtectedFeature
May 16, 2023
maziyarpanahi
changed the base branch from
master
to
release/443-release-candidate
May 24, 2023 11:52
maziyarpanahi
approved these changes
May 25, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR enables
Param
s andFeature
s to be protected. Once set, they can't be changed again. This is useful for specific pretrained models, where some parameters should not be changed (as it could interfere with the functionality of the pretrained model).Protected
Feature
sProtected
Feature
s were already introduced in the PR #13777 but are finalized in this PR. An warning will be printed when trying to set a protected feature and it will not be set.Protected
Param
sProtected
Param
s are introduced with the traitHasProtectedParams
:https://github.com/DevinTDHa/spark-nlp/blob/0fdab1d0c6eb32125558b77f5d7e33adcbcd5346/src/main/scala/com/johnsnowlabs/nlp/HasProtectedParams.scala
It introduces an implicit conversion from a
Param
to aProtectedParam
. This can be done, by callingsetProtected()
on a regular Spark parameter. For Example:The conversion enables additional functionality while setting the value for the parameter. For the
set
function, it will check, whether or not the (now)ProtectedParam
is protected. If so, it will only allow for the parameter to be set once:spark-nlp/src/main/scala/com/johnsnowlabs/nlp/HasProtectedParams.scala
Lines 47 to 56 in 0fdab1d
Note: Default values do not count as set and can therefore be overridden
Motivation and Context
Some parameters should not be changed after initialization. This PR addresses this missing feature.
How Has This Been Tested?
Tests have been added. The behavior can be seen here:
spark-nlp/src/test/scala/com/johnsnowlabs/nlp/HasProtectedParamsTestSpec.scala
Lines 30 to 41 in 0fdab1d
TODO:
ProtectedParams
will need to override some additional functions.Types of changes
Checklist: