Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor model scitype check2 #732

Merged
merged 11 commits into from
Feb 7, 2022

Conversation

ablaom
Copy link
Member

@ablaom ablaom commented Jan 28, 2022

Continuation of #731.
Needs:

@pazzo83 Be good if you review this (the most recent commits). The main problems fixed in this continuation are:

  • A bug in MLJModelInterface for constructing the fallback for fit_data_scitype for supervised models supporting weights (so not part of this PR, but partly responsible for failure of tests in remove specific model checks for general scitype check #731)
  • Flawed logic for testing presence of Unknown in fit_data_scitype. This is now a separate function _contains_unknown with tests.
  • check_model_scitype was not returning the value of nowarns
  • a lot of logging checks needed updating

The difficulties encountered in #731 are due in some part to a lack of testing of the check_model_scitype "fallback" we are now using as the main method. The methods we are removing were covering up this lack of testing, but that's a poor excuse. My apologies for this.

To do:

  • Remove redundant code (mostly a bunch of warning strings that no longer apply)
  • Rebase this PR onto for-a-0-point-20-release. Technically, the only "breaking" change is changes to strings in the warnings. However, I think these changes are pretty critical for new users, for which the new warnings will be more cryptic. So, we may want to at least wait for some progress on [Discussion] Review model documentation strings MLJ.jl#898 before releasing the changes here. @pazzo83 You could continue development of "supervised" transformers using the for-a-0-point-20-release branch? This is being regularly rebased onto dev. What do you think?

@codecov-commenter
Copy link

codecov-commenter commented Jan 28, 2022

Codecov Report

❗ No coverage uploaded for pull request base (for-a-0-point-20-release@348f82e). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@                     Coverage Diff                     @@
##             for-a-0-point-20-release     #732   +/-   ##
===========================================================
  Coverage                            ?   85.06%           
===========================================================
  Files                               ?       36           
  Lines                               ?     3369           
  Branches                            ?        0           
===========================================================
  Hits                                ?     2866           
  Misses                              ?      503           
  Partials                            ?        0           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 348f82e...f4be7dc. Read the comment docs.

@pazzo83
Copy link
Collaborator

pazzo83 commented Jan 28, 2022

This looks good! I had figured the main issue was in the tests where we were checking for the presence of various log messages. I just checkout out your branch and ran through all the tests without issue! Thanks for helping me along here!

@ablaom ablaom changed the base branch from dev to for-a-0-point-20-release February 3, 2022 23:37
@ablaom
Copy link
Member Author

ablaom commented Feb 4, 2022

@pazzo83 I've rebased this PR onto for-a-0-point-20-release. Just working on the last item on the checklist.

@ablaom ablaom mentioned this pull request Feb 7, 2022
3 tasks
@ablaom ablaom merged commit 3f5229b into for-a-0-point-20-release Feb 7, 2022
@ablaom ablaom deleted the refactor_model_scitype_check2 branch February 7, 2022 19:57
@ablaom ablaom mentioned this pull request Apr 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants