[develop] Doc test bug fixes (#1206)

* Refactors doc tests to separate doc build and linkcheck
* Adds --keep-going flag so that tests continue on failure (so that developers can see all sources of failure, not just the first)
* Fixes SD-related documentation warnings
* Removes systems no longer supported by the UFS WM (Cheyenne, Odin, Stampede)
* Updates warning about cron to mention Derecho & Gaea
* Misc minor updates/formatting

---------

Co-authored-by: Brandon Selbig <156852197+selbigmtnwx23@users.noreply.github.com>
Co-authored-by: Michael Lueken <63728921+MichaelLueken@users.noreply.github.com>

.github/scripts/check_tech_doc.sh

@@ -5,7 +5,7 @@
 set -eo pipefail
 
 # Install prerequisites
-pip install sphinx
+pip install Sphinx==7.4.7
 pip install sphinx-rtd-theme
 pip install sphinxcontrib-bibtex
 

.github/workflows/doc_tests.yaml

@@ -22,4 +22,9 @@ jobs:
       - name: Build documentation
         run: |
           cd doc
-          make doc
+          make clean && make html
+      - name: Check links
+        if: ${{ !cancelled() }}
+        run: |
+          cd doc
+          make linkcheck

doc/Makefile

@@ -1,6 +1,6 @@
 # Makefile for Sphinx documentation
 
-SPHINXOPTS    = -a -n -W
+SPHINXOPTS    = -a -n -W --keep-going
 SPHINXBUILD   = sphinx-build
 SOURCEDIR     = .
 BUILDDIR      = build

doc/UsersGuide/BuildingRunningTesting/RunSRW.rst

@@ -310,9 +310,9 @@ On platforms where Rocoto and :term:`cron` are available, users can automate res
    USE_CRON_TO_RELAUNCH: true
    CRON_RELAUNCH_INTVL_MNTS: 3
 
-.. note::
+.. attention::
 
-   On Orion, *cron* is only available on the orion-login-1 node, so users will need to work on that node when running *cron* jobs on Orion.
+   Cron is not available on Derecho or Gaea. On Orion/Hercules, users must be logged into the [hercules/orion]-login-1 node to use cron.
 
 When running with GNU compilers (i.e., if the modulefile used to set up the build environment in :numref:`Section %s <BuildExecutables>` uses a GNU compiler), users must also set ``COMPILER: "gnu"`` in the ``workflow:`` section of the ``config.yaml`` file.
 
@@ -1389,6 +1389,10 @@ where ``/path/to/experiment/directory`` is changed to correspond to the user's `
    * On NOAA Cloud instances, ``*/1 * * * *`` (or ``CRON_RELAUNCH_INTVL_MNTS: 1``) is the preferred option for cron jobs because compute nodes will shut down if they remain idle too long. If the compute node shuts down, it can take 15-20 minutes to start up a new one. 
    * On other NOAA HPC systems, administrators discourage using ``*/1 * * * *`` due to load problems. ``*/3 * * * *`` (or ``CRON_RELAUNCH_INTVL_MNTS: 3``) is the preferred option for cron jobs on other Level 1 systems.
 
+.. attention::
+
+   Cron is not available on Derecho or Gaea. On Orion/Hercules, users must be logged into the [hercules/orion]-login-1 node to use cron.
+
 To check the experiment progress:
 
 .. code-block:: console

doc/UsersGuide/CustomizingTheWorkflow/ConfigWorkflow.rst

@@ -30,7 +30,7 @@ If non-default parameters are selected for the variables in this section, they s
    Setting ``RUN_ENVIR`` to "community" is recommended in most cases for users who are not running in NCO's production environment. Valid values: ``"nco"`` | ``"community"``
 
 ``MACHINE``: (Default: "BIG_COMPUTER")
-   The machine (a.k.a. platform or system) on which the workflow will run. Currently supported platforms are listed on the :srw-wiki:`SRW App Wiki page <Supported-Platforms-and-Compilers>`. When running the SRW App on any ParallelWorks/NOAA Cloud system, use "NOAACLOUD" regardless of the underlying system (AWS, GCP, or Azure). Valid values: ``"HERA"`` | ``"ORION"`` | ``"HERCULES"`` | ``"JET"`` | ``"CHEYENNE"`` | ``"DERECHO"`` | ``"GAEA"`` | ``"GAEA-C6"`` |  ``"NOAACLOUD"`` | ``"STAMPEDE"`` | ``"ODIN"`` | ``"MACOS"`` | ``"LINUX"`` | ``"SINGULARITY"`` | ``"WCOSS2"`` (Check ``ufs-srweather-app/ush/valid_param_vals.yaml`` for the most up-to-date list of supported platforms.)
+   The machine (a.k.a. platform or system) on which the workflow will run. Currently supported platforms are listed on the :srw-wiki:`SRW App Wiki page <Supported-Platforms-and-Compilers>`. When running the SRW App on any ParallelWorks/NOAA Cloud system, use "NOAACLOUD" regardless of the underlying system (AWS, GCP, or Azure). Valid values: ``"HERA"`` | ``"ORION"`` | ``"HERCULES"`` | ``"JET"`` | ``"DERECHO"`` | ``"GAEA"`` | ``"GAEA-C6"`` |  ``"NOAACLOUD"`` | ``"MACOS"`` | ``"LINUX"`` | ``"SINGULARITY"`` | ``"WCOSS2"`` (Check ``ufs-srweather-app/ush/valid_param_vals.yaml`` for the most up-to-date list of supported platforms.)
 
    .. hint::
       Users who are NOT on a named, supported Level 1 or 2 platform will need to set the ``MACHINE`` variable to ``LINUX`` or ``MACOS``. To combine use of a Linux or MacOS platform with the Rocoto workflow manager, users will also need to set ``WORKFLOW_MANAGER: "rocoto"`` in the ``platform:`` section of ``config.yaml``. This combination will assume a Slurm batch manager when generating the XML. 
@@ -2103,7 +2103,7 @@ Non-default parameters for coupled Air Quality Modeling (AQM) tasks are set in t
 
 
 Smoke and Dust Configuration Parameters
-=====================================
+=========================================
 
 Non-default parameters for Smoke and Dust tasks are set in the ``smoke_dust_parm:`` section of the ``config.yaml`` file.
 

doc/UsersGuide/Reference/Glossary.rst

@@ -106,10 +106,10 @@ Glossary
 
    FV3
       The Finite-Volume Cubed-Sphere :term:`dynamical core` (dycore). Developed at NOAA's `Geophysical 
-      Fluid Dynamics Laboratory <https://www.gfdl.noaa.gov/>`__ (GFDL), it is a scalable and flexible dycore capable of both hydrostatic and non-hydrostatic atmospheric simulations. It is the dycore used in the UFS Weather Model.
+      Fluid Dynamics Laboratory <https://www.gfdl.noaa.gov/>`_ (GFDL), it is a scalable and flexible dycore capable of both hydrostatic and non-hydrostatic atmospheric simulations. It is the dycore used in the UFS Weather Model.
 
    FVCOM
-      `Finite Volume Community Ocean Model <https://www.fvcom.org/>`__. FVCOM is used in modeling work for the `Great Lakes Coastal Forecasting System (next-gen FVCOM) <https://www.glerl.noaa.gov/res/glcfs/>`__ conducted by the `Great Lakes Environmental Research Laboratory <https://www.glerl.noaa.gov/>`__.
+      `Finite Volume Community Ocean Model <http://www.fvcom.org/>`_. FVCOM is used in modeling work for the `Great Lakes Coastal Forecasting System (next-gen FVCOM) <https://www.glerl.noaa.gov/res/glcfs/>`_ conducted by the `Great Lakes Environmental Research Laboratory <https://www.glerl.noaa.gov/>`_.
 
    GFS
       `Global Forecast System <https://www.ncei.noaa.gov/products/weather-climate-models/global-forecast>`_. The GFS is a National Centers for Environmental Prediction (:term:`NCEP`) weather forecast model that generates data for dozens of atmospheric and land-soil variables, including temperatures, winds, precipitation, soil moisture, and atmospheric ozone concentration. The system couples four separate models (atmosphere, ocean, land/soil, and sea ice) that work together to accurately depict weather conditions.
@@ -172,7 +172,7 @@ Glossary
       MPI stands for Message Passing Interface. An MPI is a standardized communication system used in parallel programming. It establishes portable and efficient syntax for the exchange of messages and data between multiple processors that are used by a single computer program. An MPI is required for high-performance computing (HPC) systems.
 
    MRMS
-      Multi-Radar/Multi-Sensor (MRMS) System Analysis data. This data is required for METplus composite reflectivity or :term:`echo top` verification tasks within the SRW App. A two-day archive of precipitation, radar, and aviation and severe weather fields is publicly available and can be accessed `here <https://mrms.ncep.noaa.gov/data/>`__.
+      Multi-Radar/Multi-Sensor (MRMS) System Analysis data. This data is required for METplus composite reflectivity or :term:`echo top` verification tasks within the SRW App. A two-day archive of precipitation, radar, and aviation and severe weather fields is publicly available and can be accessed `here <https://mrms.ncep.noaa.gov/>`_.
 
    NAM
       `North American Mesoscale Forecast System <https://www.ncei.noaa.gov/products/weather-climate-models/north-american-mesoscale>`_. NAM generates multiple grids (or domains) of weather forecasts over the North American continent at various horizontal resolutions. Each grid contains data for dozens of weather parameters, including temperature, precipitation, lightning, and turbulent kinetic energy. NAM uses additional numerical weather models to generate high-resolution forecasts over fixed regions, and occasionally to follow significant weather events like hurricanes.

doc/conf.py

@@ -36,9 +36,18 @@
 
 numfig = True
 
-nitpick_ignore = [('py:class', 'obj'),('py:class', 
-                   'yaml.dumper.Dumper'),('py:class', 
-                   'xml.etree.ElementTree'),('py:class', 'Basemap'),]
+nitpick_ignore = [('py:class', 'obj'),
+                  ('py:class', 'yaml.dumper.Dumper'),
+                   ('py:class', 'xml.etree.ElementTree'),
+                   ('py:class', 'Basemap'),
+                   ('py:class', 'pandas.DataFrame'),
+                   ('py:class', 'numpy.ndarray'),
+                   ('py:class', 'pandas.Index'),
+                   ('py:class', 'xarray.DataArray'),
+                   ('py:class', 'netCDF4.Dataset'),
+                   ('py:class', 'ESMF.Field'),
+                   ('py:class', 'ESMF.Grid'),
+                   ]
 
 # -- General configuration ---------------------------------------------------
 
@@ -255,8 +264,19 @@ def setup(app):
 
 # -- Options for autodoc extension ---------------------------------------
 
-autodoc_mock_imports = ["f90nml","cartopy","mpl_toolkits.basemap","fill_jinja_template",
-   "matplotlib","numpy","uwtools","mpl_toolkits","metplus",
+autodoc_mock_imports = ["f90nml",
+                        "cartopy",
+                        "mpl_toolkits.basemap",
+                        "fill_jinja_template",
+                        "matplotlib",
+                        "numpy",
+                        "uwtools",
+                        "mpl_toolkits",
+                        "metplus",
+                        "netCDF4",
+                        "pandas",
+                        "xarray",
+                        "ESMF",
    ]
 
 logger = logging.getLogger(__name__)

doc/requirements.in

@@ -1,3 +1,3 @@
-sphinx>=7.4.0
+sphinx==7.4.7
 sphinx_rtd_theme
 sphinxcontrib-bibtex

ush/smoke_dust_add_smoke.py

@@ -23,6 +23,9 @@ def populate_data(data: np.ndarray, target_shape: Tuple) -> np.ndarray:
 
 
 def main() -> None:
+    """
+    Main entrypoint for generating smoke/dust initial conditions.
+    """
     # File paths
     source_file = "fv_tracer.res.tile1.nc"
     target_file = "gfs_data.tile7.halo0.nc"

ush/valid_param_vals.yaml

@@ -4,7 +4,7 @@
 valid_vals_RUN_ENVIR: ["nco", "community"]
 valid_vals_VERBOSE: [True, False]
 valid_vals_DEBUG: [True, False]
-valid_vals_MACHINE: ["HERA", "WCOSS2", "ORION", "HERCULES", "JET", "ODIN", "CHEYENNE", "DERECHO", "STAMPEDE", "LINUX", "MACOS", "NOAACLOUD", "SINGULARITY", "GAEA", "GAEA-C6"]
+valid_vals_MACHINE: ["HERA", "WCOSS2", "ORION", "HERCULES", "JET", "DERECHO", "LINUX", "MACOS", "NOAACLOUD", "SINGULARITY", "GAEA", "GAEA-C6"]
 valid_vals_SCHED: ["slurm", "pbspro", "lsf", "lsfcray", "none"]
 valid_vals_FCST_MODEL: ["ufs-weather-model"]
 valid_vals_WORKFLOW_MANAGER: ["rocoto", "ecflow", "none"]