Skip to content

Commit

Permalink
ecflow updates after testing suite (#41)
Browse files Browse the repository at this point in the history
* consolidate gitignore and update resources in ecf to match dev/drivers

* minor updates to ecflow parts

* export model=wafs in all ecf scripts

* minor tweaks to build

* use envir-p1.h from NCO

* use head.h from ops

* suppress verbose echos when loading or listing modules

* use envir-p1.h from NCO, allow over-riding for devs in head.h

* update wafs.def after testing wafs_00.def

* remove hardwired paths to COMROOT and DATAROOT from envir-p1.h

* fix errors in suite def for blending family

* ensure the name of the job has a f in front of the forecast hour

* COM needs a %ENVIR%

* remove wafs_00.def and add COMPATH

* update FHR for grib jobs, use DCOMROOT for testing

* update suite def and create a setup script for developers

* add README.md for ecflow

* typo fix

* added 000 for grib2_1p25

* fix typo for upp f001
  • Loading branch information
aerorahul authored Aug 27, 2024
1 parent ca2a7f1 commit ff9c795
Show file tree
Hide file tree
Showing 19 changed files with 740 additions and 586 deletions.
15 changes: 15 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,18 @@ install*/
log*

*.sw[a-p]

# Ignore the upp parm files so they do not get added to this repo
parm/upp

# Ignore the symlinked files in the ecf/scripts directory
ecf/scripts/upp/jwafs_upp_f*.ecf
ecf/scripts/upp/jwafs_upp_anl.ecf
ecf/scripts/grib2/1p25/jwafs_grib2_1p25_f*.ecf
ecf/scripts/grib2/0p25/jwafs_grib2_0p25_f*.ecf
ecf/scripts/grib2/0p25/blending/jwafs_grib2_0p25_blending_f*.ecf
ecf/scripts/gcip/jwafs_gcip_f*.ecf
ecf/scripts/grib/jwafs_grib_f*.ecf

# Ignore wafs.def
ecf/def/wafs.def
66 changes: 66 additions & 0 deletions dev/ecf/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Getting Started with ecflow

This README provides a basic guide on how to start using ecflow and load a suite definition file.

## Create the `ecflow` suite
The `ecflow` suite is a collection of tasks that can be run in a specific order. The suite is defined in a suite definition file, which is a text file that contains the tasks and dependencies between them.
```bash
cd dev/ecf
./setup_ecf.sh
```
This will create a suite definition file called `wafs.def` in the `ecf/def` directory as well as links to the individual forecast hour ecf scripts.

## Loading `ecflow`
`ecflow` can be loaded using the following command:
```bash
module load ecflow
```
This will load `ecflow` in your environment and setup the necessary value for `ECF_PORT`. It will also add `ecflow` calls to your `PATH`.

## Starting `ecflow_server`
`ecflow_server` can only be started on dedicated ecflow server nodes. On WCOSS2, the ecflow server nodes for development are:
- `cdecflow01`, `cdecflow02` (cactus)
- `ddcflow01`, `ddcflow02` (dogwood)

Before starting the `ecflow_server`, one has to set the following variables. This only needs to be set once before starting the `ecflow_server`.
```bash
export ECF_ROOT=${HOME}/ecflow
export ECF_OUTPUTDIR=${ECF_ROOT}/output
export LFS_OUTPUTDIR=${ECF_ROOT}/submit
export ECF_COMDIR=${ECF_ROOT}/com
mkdir -p ${ECF_ROOT}
```

You are now ready to start the `ecflow_server`.
```bash
server_check.sh ${ECF_ROOT}
```
This will start the `ecflow_server` and print out the port number that the server is running on.

You can now exit the host where you started the `ecflow_server` and return to the usual WCOSS2 login nodes.

## Loading a Suite Definition File
Load the `ecflow` module on the WCOSS2 login node where you want to load the suite definition file.
Check to ensure the `ecflow_client` can ping the `ecflow_server`:
```bash
`ecflow_client --ping`
```

If this is successful, one can launch the `ecflow_ui` and place it in the background.
```bash
ecflow_ui &
```

Navigate to the directory where the suite definition file is located (typically `ecf/def`).
```bash
ecflow_client --load $PWD/wafs.def
```

## Begin Running the Suite
To begin running the suite, use the following command:
```bash
ecflow_client --begin wafs
```

## Additional Resources
For more information on using ecflow, refer to the official [documentation](https://ecflow.readthedocs.io/en/latest/overview.html)
28 changes: 28 additions & 0 deletions dev/ecf/setup_ecf.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/bin/bash

set -eu


# Get the root of the cloned WAFS directory
readonly DIR_ROOT=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}")")/../.." && pwd -P)

ECF_DIR="${DIR_ROOT}/ecf"

model="wafs"
modelver=$(echo ${DIR_ROOT} | perl -pe "s:.*?/${model}\.(v[\d\.a-z]+).*:\1:")
packageroot=$(dirname ${DIR_ROOT})

# Replace @VARIABLE@ names in suite definition files with values
echo "Create ecflow suite definition file in ... ecf/def/wafs.def"
sed -e "s|@MACHINE_SITE@|${MACHINE_SITE:-development}|g" \
-e "s|@USER@|${USER}|g" \
-e "s|@MODELVER@|${modelver}|g" \
-e "s|@PACKAGEROOT@|${packageroot}|g" \
"${ECF_DIR}/def/wafs.def.tmpl" > "${ECF_DIR}/def/wafs.def"

# Link ecflow scripts
echo "Link ecflow scripts in ... ecf/scripts"
cd "${ECF_DIR}" || exit 1
./setup_ecf_links.sh

echo "... done"
Loading

0 comments on commit ff9c795

Please sign in to comment.