Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 'ahi_hsd' reader crashing when 'observation_timeline' was invalid #2107

Merged
merged 7 commits into from
May 11, 2022

Conversation

simonrp84
Copy link
Member

If an AHI data file contains fill value for the 'observation_timeline' property then the ahi_hsd reader crashes when it tried to round the observation times across segments.
This PR checks whether the timeline is valid and only performs rounding if it is. If invalid, no rounding is done and a warning is shown to the user.

Copy link
Member

@sfinkens sfinkens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! Just one comment regarding readability. Would you mind adding a test for this?

@codecov
Copy link

codecov bot commented May 9, 2022

Codecov Report

Merging #2107 (03a62ef) into main (16b76bf) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #2107   +/-   ##
=======================================
  Coverage   93.95%   93.96%           
=======================================
  Files         283      283           
  Lines       42881    42901   +20     
=======================================
+ Hits        40290    40310   +20     
  Misses       2591     2591           
Flag Coverage Δ
behaviourtests 4.76% <0.00%> (-0.01%) ⬇️
unittests 94.51% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
satpy/readers/ahi_hsd.py 97.50% <100.00%> (+0.07%) ⬆️
satpy/tests/reader_tests/test_ahi_hsd.py 99.61% <100.00%> (+0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 16b76bf...03a62ef. Read the comment docs.

@coveralls
Copy link

coveralls commented May 9, 2022

Coverage Status

Coverage increased (+0.003%) to 94.455% when pulling 03a62ef on simonrp84:him_fix_obstime into 16b76bf on pytroll:main.

Copy link
Member

@sfinkens sfinkens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for adding the tests! Just one tiny comment :)

Comment on lines +40 to +44
'satellite': 'Himawari-8',
'observation_area': 'FLDK',
'observation_start_time': 58413.12523839,
'observation_end_time': 58413.12562439,
'observation_timeline': '0300',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does removing the np.array reflect what the real code is getting? At least in the NetCDF/HDF5 readers these libraries have cases where they return a scalar array of a single string and we have to do extra work to make sure they are converted to plain strings correctly. If you kept the original np.array usage and converted 300 to a string would that imitate the files well enough?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does: While debugging I had some print statements inside the ahi_hsd.py file and they showed that there's no numpy arrays for these values, they're all simple strings or floats.

@simonrp84
Copy link
Member Author

Tests are failing but I think the failures are not to do with the changes here.

@djhoese
Copy link
Member

djhoese commented May 9, 2022

The MacOS job seems to be a hiccup, the other one is the unstable environment which is failing because of changes in dask that are incompatible with xarray. We can track the dask/xarray issue at pydata/xarray#6578

I've restarted the MacOS job.

@simonrp84
Copy link
Member Author

Thanks, seems to have worked now

@djhoese djhoese added the bug label May 10, 2022
Copy link
Member

@sfinkens sfinkens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@djhoese djhoese changed the title Fix bug in Himawari HSD reader for rounding timelines Fix 'ahi_hsd' reader crashing when 'observation_timeline' was invalid May 11, 2022
@djhoese djhoese merged commit c181288 into pytroll:main May 11, 2022
@simonrp84 simonrp84 deleted the him_fix_obstime branch April 23, 2023 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants