Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update Hera setup scripts #299

Merged
merged 1 commit into from
Feb 11, 2022
Merged

update Hera setup scripts #299

merged 1 commit into from
Feb 11, 2022

Conversation

grantfirl
Copy link
Collaborator

Fixes #298

Hera sysadmins changed how python is managed on Hera: https://rdhpcs-common-docs.rdhpcs.noaa.gov/wiki/index.php/Anaconda#Migrating_from_the_old_.22RDHPCS_mirror.22_installation_to_the_Anaconda_installation

Specifically, installing shapely, configobj, and netCDF4 using the method of grabbing the modules from the RDHPCS mirror does not work any more. According to their documentation, the preferred method for managing python is for users to install their own version of Miniconda and use that, which is what I did on Hera.

  • I installed miniconda in /scratch1/BMC/gmtb/SCM_anaconda on Hera
  • create new "pyccpp" environment to maintain for CCPP SCM users (we can install whatever packages that we want in that environment in the future); this eliminates the need to check for installed packages and install them if missing -- if future users have a python package problem on Hera, it will be on SCM maintainers to update the pyccpp environment.

Note that UFS is going to use the hpc-stack repo for managing dependencies on supported platforms, which should also affect these files. I tried to follow their instructions for using hpc-stack modules on Hera and ran into a runtime error when running the SCM executable (something about not being able to find the sz.o dynamic library, potentially related to szip, hdf5, and/or use of netCDF in the code).

Copy link
Collaborator

@climbfuji climbfuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good and fix the problem reported in #298.

@grantfirl grantfirl merged commit 2708b38 into NCAR:main Feb 11, 2022
@siwei-noaa
Copy link

siwei-noaa commented Feb 14, 2022

May I know if only one step is required to use this new Hera setup script: source ../etc/Hera_setup_intel.csh? When I tried to use it, I got the message:
Setting environment variables for CCPP-SCM on Hera with icc/ifort
Loading intel and netcdf modules...
Setting CC/CXX/FC environment variables
Setting NCEPLIBS environment variables
Loading cmake
Loading the SCM python environment
usage: conda [-h] [-V] command ...

conda is a tool for managing and deploying applications, environments and packages.

Options:
positional arguments:

...

conda commands available from other packages:
env
Could not find conda environment: pyccpp
You can list all discoverable environments with conda info --envs.

With this, I can compile the SCM successfully, but I could not run it. The error message is:
Traceback (most recent call last):
File "/scratch2/BMC/gsd-fv3-test/she/ccpp-scm/scm/bin/./run_scm.py", line 5, in
import f90nml
ModuleNotFoundError: No module named 'f90nml'

Thank you,
Siwei

@siwei-noaa
Copy link

Please forget my previous message. I figured out the reason.
I have installed my Anaconda, and it loads automatically when I log in Hera. In this way, I cannot load your SCM python environment from your Anaconda (source Hera_setup_intel.csh could not be finished correctly). After unloading my Anaconda environment, I can source Hera_setup_intel.csh without any issue.

Thank you,
Siwei

mkavulich pushed a commit that referenced this pull request May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Problem to install a shapely python module
3 participants