Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port tutorial notebooks to use Beam #487

Conversation

derekocallaghan
Copy link
Contributor

@derekocallaghan derekocallaghan commented Feb 2, 2023

This PR will be used to port tutorial notebooks to beam-refactor:

The first is an initial port of the NetCDF Zarr Multi-Variable Sequential Recipe: NOAA World Ocean Atlas notebook to use the beam-refactor approach:

  • A custom transform is provided, containing the preprocessor function logic in the non-Beam notebook version. I've included some links to Beam documentation here but this could be expanded if required.
  • The recipe approach is similar to that described in intro_tutorial_part2
  • I've run it locally and rebuilt the docs, it seems to work without problems. The only issue I spotted was the final plot range is different to the non-Beam version, but I couldn't see what the problem was and I don't think it's related to Beam usage.
  • Although it contains references to the Xarray-to-Zarr Sequential Recipe: NOAA OISST notebook, I haven't ported this yet as it contains functionality which may no longer be provided by the beam-refactor branch. I'm going to look at this next. In the meantime, this current notebook may be useful as an example for porting other tutorial notebooks (@norlandrhagen).

Partly fixes #483

@derekocallaghan derekocallaghan added documentation Improvements or additions to documentation beam refactor labels Feb 2, 2023
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@derekocallaghan derekocallaghan changed the title Initial port of NOAA World Ocean Atlas notebook to use Beam Port tutorial notebook to use Beam Feb 15, 2023
@derekocallaghan derekocallaghan changed the title Port tutorial notebook to use Beam Port tutorial notebooks to use Beam Feb 15, 2023
@derekocallaghan
Copy link
Contributor Author

I'm going to use this PR for any tutorial notebooks that I port rather than a single PR per notebook. It probably makes more sense to review them together.

@derekocallaghan
Copy link
Contributor Author

derekocallaghan commented Feb 17, 2023

Hi @cisaacstern, this PR is ready for review now, involving porting three of the original notebooks, with one excluded. I removed references to XarrayZarrRecipe etc in the tutorial index page. The ported notebooks are available at https://pangeo-forge--487.org.readthedocs.build/en/487/pangeo_forge_recipes/tutorials/index.html

The HDF Reference Recipe for CMIP6 notebook hasn't been ported yet as I assume it's dependent on #486. @norlandrhagen is this okay with you?

The TerraClimate notebook port is covered by #490.

I'll create separate PRs for the remaining tasklist in #483.

@derekocallaghan
Copy link
Contributor Author

derekocallaghan commented Feb 20, 2023

Hi @cisaacstern, if the notebook changes are okay with you, it might be easier to merge this before merging the latest store_name PR from @yuvipanda (#495). I can update the notebooks afterwards to include the required store_name in each StoreToZarr usage.

@derekocallaghan
Copy link
Contributor Author

Hi @rabernat and @cisaacstern, if you're free, can you take a look at these notebook changes? If you're happy with them, I'll work on the remaining documentation tasks in #483. Thanks.

Copy link
Member

@cisaacstern cisaacstern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@derekocallaghan these look great to me! The illustrations of how to add preprocessors are very clear, and helped me understand how easy that is do with beam-refactor. Thanks so much, good to merge IMO!

@derekocallaghan derekocallaghan merged commit 08364b5 into pangeo-forge:beam-refactor Mar 3, 2023
@derekocallaghan derekocallaghan deleted the beam-tutorial-notebooks branch March 3, 2023 09:59
@rabernat
Copy link
Contributor

rabernat commented Mar 3, 2023

Psyched to see this PR land! Thanks @derekocallaghan!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beam refactor documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants