Implement adaptive FCI chunks padding and create a new GEOVariableSegmentYAMLReader class

[ameraner]

This PR implements a new YAMLReader to handle FCI chunks. This was needed since a new functionality needed to be implemented: the chunk sizes of the FCI files is configurable by the data producer, hence the gaps to be padded can be of variable sizes. The new YAMLReader computes the chunks to be padded in an adaptive way.
To keep the GEOSegmentYAMLReader code clean and generic, it was decided to create this new class.

New tests have been added. The old tests were passing after minor adaptations for the new code structure (e.g. change from function to class method).

Code check with real data:

from satpy.scene import Scene
from glob import glob
from matplotlib import pyplot as plt

path_to_testdata = glob("/path_to_test_data_with_missing_chunks" + "/*BODY*.nc")
scn = Scene(filenames=path_to_testdata, reader=['fci_l1c_nc'])

scn.load(['vis_04'], calibration='radiance', upper_right_corner='NE')

v = scn['vis_04'].values
plt.figure()
plt.imshow(v)
plt.colorbar()
plt.show(block=False)

scn_r = scn.resample('msg_seviri_fes_3km')

v = scn_r['vis_04'].values
plt.figure()
plt.imshow(v)
plt.colorbar()
plt.show(block=False)

Both images look the same, showing that the full-disk padding works and also the geolocation of the padded chunks is correct.

Similarly for SEVIRI HRIT:

path_to_testdata = '/path_to_test_data_with_missing_chunks/' 
scn = Scene(filenames=glob(path_to_testdata + "H*"), reader=['seviri_l1b_hrit'])

scn.load([channel_name], upper_right_corner='NE')

plt.figure()
plt.imshow(scn[channel_name].values)
plt.show(block=False)

scn_r = scn.resample('msg_seviri_fes_1km')

plt.figure()
plt.imshow(scn_r[channel_name].values)
plt.colorbar()
plt.show(block=False)

• Tests added
• Fully documented

PS: On a side, it adds a # type: ignore for the cached_property imports, to avoid issues with the mypy git hooks.

[codecov]

Codecov Report

Merging #1919 (9af62ca) into main (20434bf) will increase coverage by 0.04%.
The diff coverage is 99.69%.

@@            Coverage Diff             @@
##             main    #1919      +/-   ##
==========================================
+ Coverage   93.89%   93.94%   +0.04%     
==========================================
  Files         283      283              
  Lines       42589    42869     +280     
==========================================
+ Hits        39991    40275     +284     
+ Misses       2598     2594       -4     

Flag	Coverage Δ
behaviourtests	4.74% <9.59%> (+0.04%)	⬆️
unittests	94.50% <99.69%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
satpy/readers/yaml_reader.py	97.27% <99.37%> (+0.44%)	⬆️
satpy/readers/fci_l1c_nc.py	98.05% <100.00%> (+0.02%)	⬆️
satpy/tests/test_yaml_reader.py	99.60% <100.00%> (+0.03%)	⬆️
satpy/readers/fci_l2_nc.py	99.45% <0.00%> (-0.01%)	⬇️
satpy/readers/cmsaf_claas2.py	100.00% <0.00%> (ø)
satpy/tests/reader_tests/test_cmsaf_claas.py	100.00% <0.00%> (ø)
satpy/tests/reader_tests/test_seviri_l1b_icare.py	100.00% <0.00%> (ø)
satpy/tests/test_scene.py	99.47% <0.00%> (+0.01%)	⬆️
satpy/scene.py	93.32% <0.00%> (+0.34%)	⬆️
satpy/readers/viirs_l1b.py	96.15% <0.00%> (+0.37%)	⬆️
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 20434bf...9af62ca. Read the comment docs.

[coveralls]

Coverage increased (+0.05%) to 94.444% when pulling 9af62ca on ameraner:implement_adaptive_fci_padding into 20434bf on pytroll:main.

[gerritholl]

the chunk sizes of the FCI files is configurable by the data producer, hence the gaps to be padded can be of variable sizes

I appreciate it's motivated by FCI, but should the reader perhaps have a more generic name, VariableChunksYAMLReader or so? Then write in the documentation that it's motivated by FCI.

[ameraner]

Thank you @gerritholl for your comment, indeed the new YAMLReader can also be formulated generically, that's a good idea to keep the YAMLReaders as instrument-independent as possible.
I will reformulate the function names/descriptions, and put the extraction of the chunk position information inside the FCI reader in a separate, standardized method, so that another reader can implement the same method and reuse the YAMLReader.

[pnuu]

Nice job! Few comments inline.

[mraspaud]

Thanks for creating this PR, it makes sense to create a new class indeed.

I have a few comments inline, but some more general comments here:

• The amount of code for the geo readers is getting big, should we have a separate modules for these?
• Many places have comments explaining exactly what is happening. I understand and appreciate the intent, however I wonder if in some cases these could be made obsolete by having small function with very descriptive names and/or variables with good names?

[ameraner]

@mraspaud @pnuu thank you for your review! I addressed all your points, let me know if you have any more comments. I also added a test to cover an untested overridden method.

Regarding the long yaml_reader.. I agree that we could put the dedicated geo yaml readers in another module, but I would leave it for a separate PR to keep the diff clearer.

[mraspaud]

LGTM

[pnuu]

LGTM

[ameraner]

Thank you for the review and the merge :)