Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync HAFS submodules and bring in HAFSv1 baseline configurations #166

Merged
merged 54 commits into from
Nov 30, 2022

Conversation

BinLiu-NOAA
Copy link
Collaborator

@BinLiu-NOAA BinLiu-NOAA commented Nov 28, 2022

Description of changes

Sync HAFS submodules and bring in HAFSv1a/a baseline configurations back to HAFS develop branch.

  • Update HAFS submodules hafs_forecast.fd, hafs_gsi.fd, hafs_utils.fd, which were synced to their latest authoritative branches as of 11/13/2022. And hafs_post.fd is consistent with the version used by ufs-weather-model (hafs_forecast.fd).
  • Bring in HAFSv1a/b baseline configurations back to HAFS develop branch.
  • Enable generating regional ESG grid within HAFS application workflow (supporting both single domain and moving-nesting configurations) from (@BinLiu-NOAA and @BantwaleEnyew-NOAA).
  • Modernize HAFS vortex initialization related source code (from @JunghoonShin-NOAA and @ChuankaiWang-NOAA)
  • Update hafs_datool source code with MPI and OMP parallelization (from @yonghuiweng)
  • Switch to use Intel 2022 compiler version on NOAA RDHPCS (Jet, Hera, Orion), switch to use Lua format modulefiles, remove application/workflow level support for wcoss_cray and wcoss_dell_p2 (from @BijuThomas-NOAA)
  • Add upoff as a namelist option (from a working session among a few EMC/HRD colleagues).
  • Fix the time information column in FV3 dycore internal tracker output partial atcf track file.
  • Add the application/workflow capability of using orog_gsl and new UGWP scheme, which is not yet used in the HAFSv1a/b baseline configurations (from @XuLi-NOAA and @BinLiu-NOAA)

Issues addressed

Contributors

Lots of UFS HAFS application team members contributed to this PR. Details can be found in the Git commit log messages.

Tests conducted

  • Technical tests for HAFSv1a/b baseline configurations were conducted on multiple platforms by @yonghuiweng.
  • HAFSv1a/b baseline experiments have been running on all four HAFS supported platforms (wcoss2, jet, hera, orion).
  • HAFS application/workflow level regression tests succeeded on wcoss2.

Application-level regression test status

Running the HAFS application-level regression tests is currently performed by code reviewers after the developer creates the initial PR. As regression tests are conducted, the testers should use the checklist below to indicate successful regression tests. You may add other tests as needed. If a test fails, do not check the box. Instead, describe the failure in the PR comments, noting the platform where the test failed.

  • Jet
  • Hera
  • Orion
  • WCOSS2

As usual, please help to conduct the HAFS application/workflow level regression tests for this PR.
@panll (Hera)
@mrinalbiswas (Orion)
@JunghoonShin-NOAA (Jet)
@BijuThomas-NOAA (wcoss2)

BinLiu-NOAA and others added 30 commits September 11, 2022 15:24
…rog files

in HAFS workflow (Bin Liu and Xu Li).
feature/hafs_sync_202210, which has FV3ATM-HYCOM coupling momentum flux
related bug fixed in HYCOM-src.
*And submodules hafs_forecast.fd and hafs_utils.fd were synced with
their authoritative branches as of 10/22/2022.
static/moving nesting configuration in HAFS application/workflow.
Majority of them are default values, but could be changed/tested in
future experiments. Meanwhile, change to use upoff of 2 (instead of 1)
to make the forecast model more stable.
Note: The sync of GSI develop was done through a series discussion/working
session among Ting Lei, Xu Lu (OU), Li Bi, Yonghui Weng, Biju Thomas, Bin Liu.
FV3dycore internal tracker output phtcf file.
which were synced to their latest authoritative branches as of
11/13/2022.
domain center, when config.domlon/lat are not set. storm, parent domain
centered at storm center; auto, parent domain center determined based on
storm center and 72-h forecasted storm location (method used in HWRF).
*Enable automatically setting the moving nest domain location when
i/jstart_nest, i/jend_nest are set as -999.
*Updated and improved hafs_workflow.xml.in and site entity files.
Note: Need python3 version with numpy and xarray.
Todo: Update to use proper python3 version on RDHPCS.
*Update NEXRAD related gsi_parm namelist options (according to Jason Sippel).
*Fix a typo in ush/hafs/launcher.py related auto detection of moving nest domain location.
*Added hafsv1a baseline conf file and cronjob driver.
@BinLiu-NOAA BinLiu-NOAA self-assigned this Nov 28, 2022
@BinLiu-NOAA BinLiu-NOAA added Ready for Review The PR is considered complete by the author(s) Ready for RTs The PR is ready to conduct Regression Tests labels Nov 28, 2022
@BinLiu-NOAA BinLiu-NOAA marked this pull request as ready for review November 28, 2022 15:45
@BinLiu-NOAA BinLiu-NOAA marked this pull request as draft November 28, 2022 15:54
@BinLiu-NOAA BinLiu-NOAA marked this pull request as ready for review November 28, 2022 18:41
@BinLiu-NOAA
Copy link
Collaborator Author

@panll, @mrinalbiswas, @JunghoonShin-NOAA, @BijuThomas-NOAA, all the submodules are up to date now, please go ahead to start the application/workflow level regression tests when you get a chance. Thanks!

@BijuThomas-NOAA
Copy link
Contributor

RT passed on WCOSS2

@panll
Copy link
Contributor

panll commented Nov 29, 2022

Regression tests passed on Hera except fails at Analysis (e.g. rt_regional_mvnest_vida) with error message /apps/lmod/8.5.2/libexec/lmod bash load gsi_hera.intel
Lmod has detected the following error: The following module(s) are unknown:
"mkl".
@BinLiu-NOAA

@JunghoonShin-NOAA
Copy link
Contributor

Hello Bin, Regression test is completed successfully on Jet.

@BinLiu-NOAA
Copy link
Collaborator Author

BinLiu-NOAA commented Nov 30, 2022

Regression tests passed on Hera except fails at Analysis (e.g. rt_regional_mvnest_vida) with error message /apps/lmod/8.5.2/libexec/lmod bash load gsi_hera.intel Lmod has detected the following error: The following module(s) are unknown: "mkl". @BinLiu-NOAA

@panll, Thanks for testing! This is somewhat strange. A few of us are actually running the HAFSv1 baseline configurations on Hera (with DA turned on) and we did not encounter this gsi_hera.intel or mkl module load issue. @BijuThomas-NOAA or @lbi2018, could you please help to take a look at this. Thanks!

@yonghuiweng
Copy link
Contributor

@BinLiu-NOAA , I had the same issue on Jet. I added "module purge" into both JHAFS_ANALYSIS and JHAFS_ATM_POST to fix this issue.

@BinLiu-NOAA BinLiu-NOAA reopened this Nov 30, 2022
@panll
Copy link
Contributor

panll commented Nov 30, 2022

The "module purge" fixes the problem. Thanks! @BinLiu-NOAA @yonghuiweng

Copy link
Contributor

@mrinalbiswas mrinalbiswas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RTs passed on Orion

@BinLiu-NOAA
Copy link
Collaborator Author

Thanks to @BijuThomas-NOAA, @JunghoonShin-NOAA, @panll and @mrinalbiswas for conducting the application level regression tests promptly for this PR! And @panll, for the gsi analysis job module purge issue on Hera, we will ignore it for now, since others did not have this issue. If it keeps coming next time, we will try to address this next time. Thanks!

@BinLiu-NOAA BinLiu-NOAA added the Ready to Merge Review and RTs are all finished. The PR is ready to merge. label Nov 30, 2022
@BinLiu-NOAA BinLiu-NOAA merged commit 9f81aa4 into develop Nov 30, 2022
@BinLiu-NOAA BinLiu-NOAA deleted the feature/hafs_sync_202210 branch January 31, 2023 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ready for Review The PR is considered complete by the author(s) Ready for RTs The PR is ready to conduct Regression Tests Ready to Merge Review and RTs are all finished. The PR is ready to merge.
Projects
None yet
7 participants