Introduce UFS-Aerosols component #591

rmontuoro · 2021-05-24T07:43:58Z

PR Checklist

Ths PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR. Please consult the ufs-weather-model wiki if you are unsure how to do this.
This PR has been tested using a branch which is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR
An Issue describing the work contained in this PR has been created either in the subcomponent(s) or in the ufs-weather-model. The Issue should be created in the repository that is most relevant to the changes in contained in the PR. The Issue and the dependent sub-component PR
are specified below.
If new or updated input data is required by this PR, it is clearly stated in the text of the PR.

Description

This PR introduces a new UFS component that will enable prognostic aerosols in P7 (#590).

Changes include:

Add GOCART submodule pointing to UFS-Aerosols authoritative repository (https://github.com/GEOS-ESM/GOCART)
Update CMake build system to:
- Enable building GOCART as a subproject
- Add new ATMAERO APP to configure and build the coupled FV3-GOCART system
- Add new ufs_aerosols_hera.intel modulefile containing additional GOCART dependencies
Updated FV3 submodule (Enable GOCART integration in UFS NOAA-EMC/fv3atm#310)
Update NEMS submodule (Replace GSDCHEM with GOCART NOAA-EMC/NEMS#102)

Note: additional configuration files are required to run GOCART. Coupled FV3-GOCART test configurations are available in the global workflow, branch: coupled/crow.

This PR does not introduce baseline changes.

Issue(s) addressed

fixes Add UFS-Aerosols component to include prognostic aerosols #590

Testing

Regression tests performed for the following platforms:

Log files included.

Dependencies

waiting on Enable GOCART integration in UFS NOAA-EMC/fv3atm#310
waiting on Replace GSDCHEM with GOCART NOAA-EMC/NEMS#102

Update NEMS submodule to point to a development fork.

Add the 'ATMAERO' app configuration option to build the UFS weather model as a coupled atmosphere-aerosol system.

for the UFS-Aerosols (GOCART) component on Hera.

replacing GSDCHEM settings with GOCART. Update CMakeLists.txt to reflect changes in NEMS.

GOCART coupling fields to YAML dictionary.

climbfuji · 2021-06-30T18:08:52Z

@rmontuoro @junwang-noaa @DusanJovic-NOAA @BrianCurtis-NOAA @MinsukJi-NOAA @DeniseWorthen At this point, all issues have been fixed I believe. I will run a final auto-rt run on hera.intel and rerun the CI tests, too. We then need to decide what to do with the wcoss systems.

rmontuoro · 2021-06-30T18:45:31Z

Should we add a GOCART R/T baseline for Hera/GNU, too?

climbfuji · 2021-06-30T19:49:42Z

Should we add a GOCART R/T baseline for Hera/GNU, too?

Not as part of this PR anymore as far as I am concerned, we need to move on.

climbfuji · 2021-06-30T19:52:00Z

No need to rerun the tests on all platforms after the latest RAS bugfix, since this code isn't executed at all. I am going to rerun on cheyenne with gnu to be 100% sure that everything still compiles.

climbfuji · 2021-06-30T20:08:20Z

ci tests passed again for b5d56b2

on-behalf-of @ufs-community

DusanJovic-NOAA · 2021-06-30T21:05:37Z

WCOSS Cray log file is here:

$ ls /scratch1/NCEPDEV/stmp2/Dusan.Jovic/RegressionTests_wcoss_cray.log
/scratch1/NCEPDEV/stmp2/Dusan.Jovic/RegressionTests_wcoss_cray.log

…ther-model into feature/gocart

climbfuji · 2021-06-30T21:12:02Z

hera.intel progress is extremely slow for the nems account. I started the tests manually with one of my other projects with a larger fairshare.

climbfuji · 2021-06-30T22:21:10Z

Regression test successful on hera.intel. We decided to ignore wcoss_dell_p3 for this PR, since there is a problem with the existing baselines (they are not even b4b identical with the head of develop for the 20210615 and 20210616 regression test baseline dates).

climbfuji

Changes are ok with me, approving based on .gitmodules and FV3 being updated correctly.

climbfuji · 2021-07-01T12:48:48Z

build.sh


 cd ${BUILD_DIR}
 cmake ${UFS_MODEL_DIR} ${CMAKE_FLAGS}
-make -j ${BUILD_JOBS:-4} VERBOSE=${BUILD_VERBOSE:-}
+# Turn off OpenMP threading for parallel builds


This is required unless we decide to go back and build OpenBlas without OpenMP support (not sure if we even wanted to do that, threading can be beneficial for linear algebra applications).

junwang-noaa · 2021-07-01T13:29:51Z

tests/tests/control_atm_aerosols

+export NSTF_NAME='2,0,0,0,0'
+export WRITE_DOPOST=.true.
+export IAER=5111
+export FSCAV_AERO='"*:0.6", "seas1:1.0", "seas2:1.0", "seas3:1.0", "seas4:1.0", "seas5:1.0"'


@rmontuoro I thought the test is going to use RAS scavenging from GOCART, I do not see RAS is specified, so we are using SAS scavenging for aerosol particles?

@junwang-noaa - The target physics suite for prognostic aerosols is FV3_GFS_v16, which includes SAS. This suite is used in the included regression test. I was not aware that RAS will be used with prognostic aerosols. However, we can update to RAS later on.

junwang-noaa

The submodules need to point back to official repos.

* fixes for land coupling * work to support external land component side-by-side config * fix issue with ort dbg test

rmontuoro added 30 commits April 15, 2021 02:58

Add GOCART submodule for prognostic aerosols.

b924f45

Update NEMS submodule to point to a development fork.

Update build system to include the GOCART component.

240bd25

Add the 'ATMAERO' app configuration option to build the UFS weather model as a coupled atmosphere-aerosol system.

Add modulefile including additional dependencies

a87bb49

for the UFS-Aerosols (GOCART) component on Hera.

Merge branch 'develop' into feature/gocart

afe202c

Update NEMS submodule.

a4e60b7

Properly add GOCART dependency to CMakeLists.txt.

3c74d37

Update FV3 submodule to point to forked repository.

a553906

Add default values for environment variables to build script.

cb0ec0b

Remove unnecessary module for aerosol component on Hera.

b3a646d

Merge branch 'develop' into feature/gocart

6354603

Update FV3 submodule.

e434544

Update FV3 submodule.

c524f22

Merge branch 'develop' into feature/gocart

5c19aac

Update FV3 submodule.

52ee7ae

Use proper CMake version when GOCART is enabled.

c238543

Remove CMAKE_FLAGS settings for APP in build script.

d6ca95d

Merge branch 'develop' into feature/gocart

fdd95d2

Revert NEMS submodule to point to authoritative repository.

ffa23a4

Update FV3 submodule.

04876cd

Merge branch 'develop' into feature/gocart

c4c946c

Update FV3 submodule.

9dec5ae

Update FV3 submodule.

1f136f4

Update MAPL to custom version 2.6.8-patch1 on Hera.

7319476

Update GOCART submodule.

65eb15d

Merge branch 'develop' into feature/gocart

9687696

Update CMake module to version 3.20.1 on Hera.

15f09d4

Update NEMS submodule to point to forked repository,

44d6867

replacing GSDCHEM settings with GOCART. Update CMakeLists.txt to reflect changes in NEMS.

Update FV3 submodule.

063e53b

Point CMEPS submodule to forked repository to add

bc2a7c1

GOCART coupling fields to YAML dictionary.

Fix .gitmodules

4e2a7c6

climbfuji added hera-intel-RT labels Jun 30, 2021

github-actions bot removed the run-ci label Jun 30, 2021

BrianCurtis-NOAA removed the hera-intel-RT label Jun 30, 2021

Update submodule pointer for fv3atm

b5d56b2

climbfuji added the cheyenne-gnu-RT label Jun 30, 2021

BrianCurtis-NOAA removed the cheyenne-gnu-RT label Jun 30, 2021

RT JOBS PASSED: cheyenne.gnu. Log file uploaded.

a47e546

on-behalf-of @ufs-community

climbfuji added 2 commits June 30, 2021 15:07

Regression test log for wcoss_cray

ea2c92d

Merge branch 'feature/gocart' of https://github.com/rmontuoro/ufs-wea…

ed9bb35

…ther-model into feature/gocart

Regression test logs for hera.intel

5c854bb

climbfuji approved these changes Jul 1, 2021

View reviewed changes

junwang-noaa reviewed Jul 1, 2021

View reviewed changes

junwang-noaa approved these changes Jul 1, 2021

View reviewed changes

rmontuoro added 2 commits July 1, 2021 15:16

Update FV3 submodule.

b06c715

Update NEMS submodule.

77a7ce3

DusanJovic-NOAA approved these changes Jul 1, 2021

View reviewed changes

DusanJovic-NOAA merged commit a842d57 into ufs-community:develop Jul 1, 2021

DeniseWorthen mentioned this pull request Jul 1, 2021

missing baseline data on orion #651

Closed

rmontuoro mentioned this pull request Jul 26, 2021

Unrealistically large aerosol mixing ratios #713

Closed

junwang-noaa mentioned this pull request Sep 12, 2021

GOCART configuration documentation #795

Closed

pjpegion pushed a commit to NOAA-PSL/ufs-weather-model that referenced this pull request Apr 4, 2023

Add support for external land component (ufs-community#591)

324591b

* fixes for land coupling * work to support external land component side-by-side config * fix issue with ort dbg test

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce UFS-Aerosols component #591

Introduce UFS-Aerosols component #591

rmontuoro commented May 24, 2021 •

edited by DusanJovic-NOAA

Loading

climbfuji commented Jun 30, 2021

rmontuoro commented Jun 30, 2021

climbfuji commented Jun 30, 2021

climbfuji commented Jun 30, 2021

climbfuji commented Jun 30, 2021

DusanJovic-NOAA commented Jun 30, 2021

climbfuji commented Jun 30, 2021

climbfuji commented Jun 30, 2021

climbfuji left a comment

climbfuji Jul 1, 2021

junwang-noaa Jul 1, 2021

rmontuoro Jul 1, 2021

junwang-noaa left a comment

Introduce UFS-Aerosols component #591

Introduce UFS-Aerosols component #591

Conversation

rmontuoro commented May 24, 2021 • edited by DusanJovic-NOAA Loading

PR Checklist

Description

Issue(s) addressed

Testing

Dependencies

climbfuji commented Jun 30, 2021

rmontuoro commented Jun 30, 2021

climbfuji commented Jun 30, 2021

climbfuji commented Jun 30, 2021

climbfuji commented Jun 30, 2021

DusanJovic-NOAA commented Jun 30, 2021

climbfuji commented Jun 30, 2021

climbfuji commented Jun 30, 2021

climbfuji left a comment

Choose a reason for hiding this comment

climbfuji Jul 1, 2021

Choose a reason for hiding this comment

junwang-noaa Jul 1, 2021

Choose a reason for hiding this comment

rmontuoro Jul 1, 2021

Choose a reason for hiding this comment

junwang-noaa left a comment

Choose a reason for hiding this comment

rmontuoro commented May 24, 2021 •

edited by DusanJovic-NOAA

Loading