Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial Scene/DataTree conversion via Scene.to_xarray_datatree #2452

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

djhoese
Copy link
Member

@djhoese djhoese commented Apr 20, 2023

Related to #2352, I thought I'd add a Scene.to_xarray_datatree method to the Scene. I want to use this as a stepping stone into more DataTree usage in Satpy and see where it takes us. I also wanted a reason to play around with DataTree more and see how it goes.

At the time of writing this is a pretty simple implementation, but also provides a lot of flexibility. When discussing this concept in #2352 I brought up the ideas of "schemes" or predefined tree-structuring methods that users could choose from. For example, "area"-based or "DataID"-based organizing. As I started working on this (TDD for the win!) I realized I could just structure it as "what metadata keys should I use for each 'group'". This means users can select the level of the hierarchy of the tree. For example, group by platform_name and sensor by specifying group_keys=("platform_name", "sensor").

So at this point, this works as described above, but I'm looking for feedback for others. I seem to remember @mraspaud and @gerritholl being part of this conversation and @ghiggi doing some reworking of Scene.to_xarray_dataset related to CF compatibility. I'm curious what you guys think of this.

There is still a lot TODO like:

  • handling grouping by area. For example, what do I use as the node name? area hash? Area name (if it exists, pyresample 2.0 will have optional names)?.
  • handling of other non-string properties including resolution (units?), calibration enum, etc.
  • What attributes to define on each group
  • Documentation
  • Closes #xxxx
  • Tests added
  • Fully documented

@djhoese djhoese added enhancement code enhancements, features, improvements component:scene labels Apr 20, 2023
@djhoese djhoese self-assigned this Apr 20, 2023
@codecov
Copy link

codecov bot commented Apr 20, 2023

Codecov Report

Merging #2452 (51c389d) into main (087acd3) will increase coverage by 0.05%.
The diff coverage is 93.24%.

@@            Coverage Diff             @@
##             main    #2452      +/-   ##
==========================================
+ Coverage   94.75%   94.81%   +0.05%     
==========================================
  Files         335      340       +5     
  Lines       48852    49417     +565     
==========================================
+ Hits        46291    46853     +562     
- Misses       2561     2564       +3     
Flag Coverage Δ
behaviourtests 4.41% <1.35%> (+<0.01%) ⬆️
unittests 95.43% <93.24%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
satpy/_scene_converters.py 91.42% <91.42%> (ø)
satpy/tests/scene_tests/test_conversions.py 97.29% <94.44%> (-2.71%) ⬇️
satpy/scene.py 93.54% <100.00%> (+0.02%) ⬆️

... and 75 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@coveralls
Copy link

Coverage Status

Coverage: 95.38% (-0.003%) from 95.383% when pulling 51c389d on djhoese:feature-scene-to-datatree into 5310f1a on pytroll:main.

@BENR0
Copy link
Collaborator

BENR0 commented May 25, 2023

I think that this could also be really useful for #2171. Currently the sorting is done by platform/sensor and the by area because this made the most sense when I started. But I think it could be useful to display the Scene by a different hierarchy. So this would make that task easier I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:scene enhancement code enhancements, features, improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants