Beam documentation

[rabernat]

No description provided.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[derekocallaghan]

Hi @rabernat, I'm going to try porting some of my recipes to use the Beam approach. I wanted to check which repo/branch I should use, e.g.:

• https://github.com/pangeo-forge/pangeo-forge-recipes/tree/beam-refactor
• https://github.com/rabernat/pangeo-forge-recipes/tree/beam-documentation
• https://github.com/rabernat/pangeo-forge-recipes/tree/beam-refactor

Thanks

[rabernat]

Thanks @derekocallaghan! That would be awesome!

The pangeo-forge-recipes/beam-refactor branch is where the latest code lives. beam-documentation is this PR, which is made against beam-refactor.

Have a look at the notebook from this PR if you want to get a feel for how beam recipes work.

[derekocallaghan]

Thanks @rabernat, I'll try out that branch. I'd had a look through that notebook already (and also https://pangeo-forge--445.org.readthedocs.build/en/445/introduction_tutorial/intro_tutorial_part2.html#)

I'll let you know how I get on

[derekocallaghan]

Hi @rabernat, I've created a couple of issues to track work related to porting the EOOffshore recipes to use the beam_refactor branch, which involves writing custom PTransforms. Here's the current status:

• CCMP v0.2.1.NRT recipe: a custom PTransform has been added to compute wind speed and direction for selected grid coordinates, and the expected pruned output Zarr is successfully generated.
• CMEMS ASCAT REP and NRT recipes: I'll create two custom PTransforms here, the first similar to that used by the CCMP recipe, while the second will sort the collection using the updated time coordinates.

As discussed in the call earlier this week, I'm currently using pangeo_forge_recipes.transforms._add_keys() to wrap the preprocessing transforms. I think it could be useful to expose this in the public API to reduce some of the code written by recipe authors, i.e. where they provide custom functions operating on e.g. Dataset parameters.

If it helps, I can look at completing the above transforms and recipes and writing some corresponding tutorial notebook-based documentation describing how custom transforms may be provided, similar to what you've done in the OISST notebook. I probably won't get to complete this until early-mid January if that's okay.

[cisaacstern]

If it helps, I can look at completing the above transforms and recipes and writing some corresponding tutorial notebook-based documentation describing how custom transforms may be provided

This would be incredibly valuable, of course on whatever timeline works well for you.

[rabernat]

I'm going to merge this and we can continue to build on it.