Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #3 - Introduce manage_externals as replacement for checkout.sh #29

Merged
merged 14 commits into from
Mar 11, 2020
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Ignore all compiled files
*.pyc
*.o
*.mod

# Ignore exec folder
exec/

# Ignore sorc folders from externals
sorc/logs/
sorc/fv3gfs.fd/
sorc/gfs_post.fd/
sorc/gsi.fd/
sorc/ufs_utils.fd/
sorc/gfs_wafs.fd/
sorc/verif-global.fd/
46 changes: 46 additions & 0 deletions Externals.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# External sub-modules of global-workflow

[NEMSfv3gfs]
tag = gfs.v16_PhysicsUpdate
local_path = sorc/fv3gfs.fd
repo_url = ssh://vlab.ncep.noaa.gov:29418/NEMSfv3gfs
protocol = git
required = True

[GSI]
hash = cb8f69d8
local_path = sorc/gsi.fd
repo_url = ssh://vlab.ncep.noaa.gov:29418/ProdGSI
protocol = git
required = True

[EMC_post]
hash = ba7e59b2
local_path = sorc/gfs_post.fd
repo_url = https://github.com/NOAA-EMC/EMC_post.git
protocol = git
required = True

[UFS_UTILS]
tag = v1.1.0
local_path = sorc/ufs_utils.fd
repo_url = https://github.com/NOAA-EMC/UFS_UTILS.git
protocol = git
required = True

[EMC_verif-global]
tag = verif_global_v1.5.0
local_path = sorc/verif-global.fd
repo_url = ssh://vlab.ncep.noaa.gov:29418/EMC_verif-global
protocol = git
required = True

[EMC_gfs_wafs]
tag = gfs_wafs.v5.0.11
local_path = sorc/gfs_wafs.fd
repo_url = https://github.com/NOAA-EMC/EMC_gfs_wafs.git
protocol = git
required = False

[externals_description]
schema_version = 1.0.0
49 changes: 49 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# global-workflow
Global Superstructure/Workflow currently supporting the Finite-Volume on a Cubed-Sphere Global Forecast System (FV3GFS)

The global-workflow depends on the following prerequisities to be available on the system:

* workload management platform / scheduler - LSF or SLURM
* workflow manager - ROCOTO (https://github.com/christopherwharrop/rocoto)
* modules - NCEPLIBS (various), esmf v8.0.0bs48, hdf5, intel/ips v18, impi v18, wgrib2, netcdf v4.7.0, hpss, gempak (see module files under /modulefiles for additional details)
* manage_externals - A utility from ESMCI to checkout external dependencies. Manage_externals can be obtained at the following address and should be in the users PATH: https://github.com/ESMCI/manage_externals
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to add a blurb on what manage_externals is replacing (checkout.sh).
I also suggest either removing checkout.sh or put in big bold letters that support for checkout.sh is being deprecated, future PR's will be denied if an inconsistencies are found between checkout.sh and Externals.cfg and checkout.sh will be removed on a not-so-distant future date.


The global-workflow current supports the following machines:

* WCOSS-Dell
* WCOSS-Cray
* Hera

## Build global-workflow:

### 1. Check out components

The global-workflow uses the manage_externals utility to handle checking out its components. The manic-v1.1.8 manage_externals tag is supported.

Run manage_externals (checkout_externals) while at top of clone:

```
$ checkout_externals
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest replacing:
$ checkout_externals
with
$ checkout_externals -e Externals.cfg

This makes it explicit that is being checked out and from where.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it, I'll make the change, thanks!

```

If checkout_externals is not in your $PATH then use full path to it:

* WCOSS-Dell: /gpfs/dell2/emc/modeling/noscrub/emc.glopara/git/manage_externals/manic-v1.1.8/checkout_externals
* WCOSS-Cray: /gpfs/hps3/emc/global/noscrub/emc.glopara/git/manage_externals/manic-v1.1.8/checkout_externals
* Hera: /scratch1/NCEPDEV/global/glopara/git/manage_externals/manic-v1.1.8/checkout_externals

### 2. Build components

While in /sorc folder:
```
$ sh build_all.sh
```

### 3. Link components

While in /sorc folder:
```
$ sh link_fv3gfs.sh emc $MACHINE
```

...where $MACHINE is "dell", "cray", or "hera".
10 changes: 0 additions & 10 deletions modulefiles/module_nemsutil.hera

This file was deleted.

13 changes: 0 additions & 13 deletions modulefiles/module_nemsutil.wcoss

This file was deleted.

17 changes: 0 additions & 17 deletions modulefiles/module_nemsutil.wcoss_cray

This file was deleted.

19 changes: 0 additions & 19 deletions modulefiles/module_nemsutil.wcoss_cray_userlib

This file was deleted.

12 changes: 0 additions & 12 deletions modulefiles/module_nemsutil.wcoss_dell_p3

This file was deleted.

32 changes: 0 additions & 32 deletions modulefiles/modulefile.grib_util.wcoss

This file was deleted.

22 changes: 0 additions & 22 deletions modulefiles/modulefile.grib_util.wcoss_cray

This file was deleted.

22 changes: 0 additions & 22 deletions modulefiles/modulefile.grib_util.wcoss_cray_userlib

This file was deleted.

15 changes: 0 additions & 15 deletions modulefiles/modulefile.grib_util.wcoss_dell_p3

This file was deleted.

11 changes: 0 additions & 11 deletions modulefiles/modulefile.prod_util.wcoss_cray

This file was deleted.

13 changes: 0 additions & 13 deletions modulefiles/modulefile.prod_util.wcoss_cray_userlib

This file was deleted.

6 changes: 0 additions & 6 deletions modulefiles/modulefile.prod_util.wcoss_dell_p3

This file was deleted.

24 changes: 0 additions & 24 deletions modulefiles/modulefile.wgrib2.wcoss

This file was deleted.

13 changes: 0 additions & 13 deletions modulefiles/modulefile.wgrib2.wcoss_cray

This file was deleted.

15 changes: 0 additions & 15 deletions modulefiles/modulefile.wgrib2.wcoss_cray_userlib

This file was deleted.

11 changes: 0 additions & 11 deletions modulefiles/modulefile.wgrib2.wcoss_dell_p3

This file was deleted.

Loading