-
Notifications
You must be signed in to change notification settings - Fork 303
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update SEVIRI ICARE reader to properly use dask. #2103
Conversation
This is a reboot of #1758 |
Looks good, anything you can add to the tests to make sure the same error doesn't reappear in the future? |
Codecov Report
@@ Coverage Diff @@
## main #2103 +/- ##
=======================================
Coverage 93.92% 93.92%
=======================================
Files 283 283
Lines 42715 42722 +7
=======================================
+ Hits 40120 40127 +7
Misses 2595 2595
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
The only thing I can think of adding to tests would be the use of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Would something like this be appropriate?
|
That looks about right! Just test this before and after your changes to make sure it works |
Ok, here we go, dask compute test added. In adding this test I also discovered that the original tests were using pure numpy arrays rather than dask arrays - that's fixed now too. With this new test, the pre-PR reader code fails, but the post-PR code passes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just one question.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! I'll just subscribe to @sfinkens 's question saw now it was answered already
pre-commit.ci autofix |
for more information, see https://pre-commit.ci
The iCARE/SEVIRI reader was not daskified, data was being stored as numpy arrays. This raised an error when trying to create a natural_color image as the Rayleigh correction code was expecting dask arrays.
This PR updates the reader so that the data is stored as a dask array.