Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CIME to ESMCI cime5.8.1 #2842

Closed
wants to merge 3,671 commits into from
Closed

Conversation

jgfouca
Copy link
Member

@jgfouca jgfouca commented Apr 8, 2019

Update CIME to ESMCI cime5.8.1

Squash merge of jgfouca/branch-for-to-acme-2019-04-08

Features:

  • Support optional components by filling in stub models for any missing component class
  • cprnc: Make FIELDLIST message more informative
  • cprnc: allow differences in field lists for time-constant fields

Bug fixes:

  • The check for an rpointer.drv file did not consider multidriver mode.
  • Fix ./case.build --clean
  • PET and ERP tests were not setting compile_threaded correctly.
  • Fixes inconsistency between MPASCICE and MPASSI, change all occurances of the former to the latter

This is non-BFB for A cases due to:
ESMCI/cime#3053

[non-BFB]

brhillman and others added 30 commits February 13, 2019 15:08
Add performance archiving support for cli133 at OLCF
Add the NETCDF_PATH env for Summit. 
The MPAS components use the NETCDF_PATH env to find the NetCDF package
and the env has not been defined in the machine and compiler xml files.

[BFB] - Bit-For-Bit
Require a command line flag (set-omask) to enable inferring ocean mask
when it is not present in the input mapping file. By default, if mask_a
is not present in the input mapping file, the code will throw and error
and exit. However, if --set-omask is passed, then in the absence of
mask_a, it will be inferred by setting the mask to a vector of 1s, which
are then remapped onto the land grid. This option is only appropriate
for unstructured grids which cover only the area in which the ocean is
active. I.e., this is NOT an appropriate thing to do for regular lat-lon
grids, but should be fine for MPAS-O grids. Tested with a mapping file
from oRRS30to10 to ne30np4 and confirmed these two modes of operation
(using provided mask, and omitting the mask and passing --set-omask to
calculate it) give the same answers.
Update xml for bluewaters, fix a python3 compatibility issue

Test suite: PFS.f09_g17.BHIST.bluewaters_intel
Test baseline:
Test namelist changes:
Test status: bit for bit

Fixes

User interface changes?:

Update gh-pages html (Y/N)?:

Code review:
Allow specifying directives to use for an unknown queue
Previously, when using an unknown queue, the batch directives would come
from the default queue for the machine. This is a problem on cheyenne,
for which the default queue is the share queue. It looks like we need to
keep the share queue as the default queue for other purposes, though (in
particular: if the default is the regular queue, then the share queue
won't get selected for small - e.g., single-processor - jobs, since
small jobs also satisfy the requirements of the regular queue), so the
solution is to provide a different mechanism for specifying the queue to
use for the sake of choosing directives.

For machines that don't use this new mechanism, we maintain the previous
behavior of using the directives for the machine's default queue.

Note: I chose this mechanism of pointing to a different queue's
directives section rather than explicitly listing the directives for an
unknown queue, partly because this was easy to implement and partly
because I thought the latter would likely lead to the two getting
accidentally out-of-sync (because people may not think to update the
directives for an unknown queue, and this would likely go untested and
undiscovered for a long time).

Test suite: scripts_regression_tests on cheyenne;
also manual testing on a version off of master,
and on this version off of maint-5.6 (all tests were
with compset A, res f45_g37 on cheyenne, with NTASKS changed to 144):

* Checked .case.run with regular queue: same as before
* Checked .case.run with share queue: same as before
* Checked .case.run with an unknown queue: now same as regular queue
* Checked .case.run with an unknown queue, but with the new
  <unknown_queue_directives> block temporarily removed: same as share
  queue, as before (demonstrating that this change is backwards
  compatible)

Test baseline: N/A
Test namelist changes: none
Test status: bit for bit

Fixes #2994

User interface changes?: no

Update gh-pages html (Y/N)?: N
Port to new cgd system izumi

Test suite: scripts_regression_tests.py
Test baseline:
Test namelist changes:
Test status:
Fixes
User interface changes?:

Update gh-pages html (Y/N)?:

Code review:
Port to new cgd system izumi

Test suite: scripts_regression_tests.py
Test baseline:
Test namelist changes:
Test status:

Fixes:
User interface changes?:

Update gh-pages html (Y/N)?:

Code review:
The use of tuples is not scalable as we increase the
configurability of test suites.

[BFB]
It's harmless and prevents missing commas when things
are added to the end.
Port to tacc system lonestar5

Test suite: scripts_regression_tests.py
Test baseline:
Test namelist changes:
Test status: bit for bit,

Fixes

User interface changes?:

Update gh-pages html (Y/N)?:

Code review:
Tests have been failing for a while since the cdash update. The
cdash page is dynamically generated, so we need to use the JSON
interface to it to get the actual data.
Fix CIME dashboad regression tests

Tests have been failing for a while since the cdash update. The
cdash page is dynamically generated, so we need to use the JSON
interface to it to get the actual data.

Test suite: scripts_regression_tests WaitForTests Jenkins
Test baseline:
Test namelist changes:
Test status: bit for bit

Fixes [CIME Github issue #]

User interface changes?: N

Update gh-pages html (Y/N)?:

Code review:
update and consolidate inputdata for stamepede2 and lonestar5

Test suite: scripts_regression_tests.py
Test baseline:
Test namelist changes:
Test status: [bit for bit, roundoff, climate changing]

Fixes [CIME Github issue #]

User interface changes?:

Update gh-pages html (Y/N)?:

Code review:
update for stand-alone CMIP6 query outside of a caseroot
update to archive_metadata stand-alone functionality so that a Case object is not incorrectly instantiated when the --query_cmip6 command line option is called outside of a caseroot.

Test suite: manual
./archive_metadata --user [aliceb@ucar.edu](mailto:aliceb@ucar.edu) --password --expType CMIP6 --query_cmip6 b.e21.B1PCT.f09_g17.CMIP6-1pctCO2-bgc.001 db.json --workdir $tests/b.e21.B1PCT.f09_g17.CMIP6-1pctCO2-bgc.001_am

Test baseline: N/A
Test namelist changes: N/A
Test status: bit for bit

Fixes [CIME Github issue #] N/A

User interface changes?: None

Update gh-pages html (Y/N)?:N

Code review:jedwards
fischer-ncar and others added 6 commits April 8, 2019 10:52
Support optional components
Support optional components by filling in stub models for any missing component class

Test suite: scripts_regression_tests.py + many tests of running create_newcase with mangled compset names (both on command line and by modifying <lname> field in <compset> definition). E.g.:

2000_DATM%NYF_SLND_DICE%SSMI_DOCN%DOM_DROF%NYF_SGLC_SWAV
2000_DATM%NYF_SLND_DICE%SSMI_DOCN%DOM_DROF%NYF_SGLC_SWAV_SESP
2000_DATM%NYF_SLND_DICE%SSMI_DOCN%DOM_DROF%NYF_SGLC
2000_DATM%NYF_SLND_DICE%SSMI_DOCN%DOM_DROF%NYF_SWAV
2000_DATM%NYF_DICE%SSMI_DOCN%DOM_DROF%NYF
2000_DICE%SSMI_DOCN%DOM_DATM%NYF_DROF%NYF
2000_DROF%NYF_DICE%SSMI_DATM%NYF_DOCN%DOM
Test baseline: NA
Test namelist changes: NA
Test status: bit for bit

Fixes #3067

User interface changes?: Stub components are now optional in compset long names. Also there is less order dependency.

Update gh-pages html (Y/N)?: N

Code review:sacks
…branch-for-to-acme-2019-04-08

* commit '444b2f49da269275d21a4abbf1aa66308b23370a': (2695 commits)
  Refactor code to make testable code a separate method
  Update for cime5.6.14
  Added some tests to _valid_compset and improved documentation.
  Update cori modules
  Update cori modules
  Forgot no xrange in Python 3
  Add PR #3053 back in
  Merge pull request #3057 from jedwards4b/maint-5.6-xml_fixes
  Support optional components
  fix issue with xmllint
  Make FIELDLIST message more informative
  add pio_version to FakeCase for configure
  remove whitespace
  undo this change
  add comment
  use installed pio if it exists
  minor bug fixes for xml interface
  add PIO_VERSION as a module load conditional
  build changes to optionally use installed pio library
  Fix setting fatal_local in check_ret
  ...
@jgfouca jgfouca added Scripts BFB PR leaves answers BFB CIME labels Apr 8, 2019
@jgfouca jgfouca self-assigned this Apr 8, 2019
@jgfouca jgfouca requested a review from rljacob April 8, 2019 19:13
@rljacob
Copy link
Member

rljacob commented Apr 8, 2019

I think you should wait on a merge until ESMCI/cime#3055 has been added to CIME.

@rljacob
Copy link
Member

rljacob commented Apr 8, 2019

Also they just tagged 5.8.1

@jgfouca
Copy link
Member Author

jgfouca commented Apr 8, 2019

I can do another merge when 3055 is ready. I'll update the PR name to reflect that this is 5.8.1.

@jgfouca jgfouca changed the title Update CIME to ESMCI cime5.8.0-2 Update CIME to ESMCI cime5.8.1 Apr 8, 2019
@jgfouca
Copy link
Member Author

jgfouca commented Apr 10, 2019

@jonbob , note the MPASCICE -> MPASSI change.

@jgfouca
Copy link
Member Author

jgfouca commented Apr 10, 2019

This seems to be working now, at least on melvin with e3sm_developer. Merging to next.

jgfouca added a commit that referenced this pull request Apr 10, 2019
Update CIME to ESMCI cime5.8.1

Squash merge of jgfouca/branch-for-to-acme-2019-04-08

Features:
* Support optional components by filling in stub models for any missing component class
* cprnc: Make FIELDLIST message more informative
* cprnc: allow differences in field lists for time-constant fields

Bug fixes:
* The check for an rpointer.drv file did not consider multidriver mode.
* Fix ./case.build --clean
* PET and ERP tests were not setting compile_threaded correctly.
* Fixes inconsistency between MPASCICE and MPASSI, change all occurances of the former to the latter

This is non-BFB for A cases due to:
ESMCI/cime#3053

[non-BFB]
jgfouca added a commit that referenced this pull request Apr 11, 2019
Update CIME to ESMCI cime5.8.1

Squash merge of jgfouca/branch-for-to-acme-2019-04-08

Features:
* Support optional components by filling in stub models for any missing component class
* cprnc: Make FIELDLIST message more informative
* cprnc: allow differences in field lists for time-constant fields

Bug fixes:
* The check for an rpointer.drv file did not consider multidriver mode.
* Fix ./case.build --clean
* PET and ERP tests were not setting compile_threaded correctly.
* Fixes inconsistency between MPASCICE and MPASSI, change all occurances of the former to the latter

This is non-BFB for A cases due to:
ESMCI/cime#3053

[non-BFB]
@jgfouca
Copy link
Member Author

jgfouca commented Apr 11, 2019

Squash-merged.

@jgfouca jgfouca closed this Apr 11, 2019
@jgfouca jgfouca deleted the jgfouca/branch-for-to-acme-2019-04-08 branch April 11, 2019 19:15
jgfouca added a commit that referenced this pull request Apr 11, 2019
Improve robustness of wget

Better error messages and avoid check certificates.

[BFB]

* jgfouca/cime/improve_wget_robustness:
  Improve robustness of wget
  Update CIME to ESMCI cime5.8.1 (PR #2842)
jgfouca added a commit that referenced this pull request Apr 11, 2019
…2849)

Improve robustness of wget

Better error messages and avoid check certificates.

[BFB]

* jgfouca/cime/improve_wget_robustness:
  Improve robustness of wget
  Update CIME to ESMCI cime5.8.1 (PR #2842)
jgfouca added a commit that referenced this pull request Jun 25, 2019
Update CIME to ESMCI cime5.8.1

Squash merge of jgfouca/branch-for-to-acme-2019-04-08

Features:
* Support optional components by filling in stub models for any missing component class
* cprnc: Make FIELDLIST message more informative
* cprnc: allow differences in field lists for time-constant fields

Bug fixes:
* The check for an rpointer.drv file did not consider multidriver mode.
* Fix ./case.build --clean
* PET and ERP tests were not setting compile_threaded correctly.
* Fixes inconsistency between MPASCICE and MPASSI, change all occurances of the former to the latter

This is non-BFB for A cases due to:
ESMCI/cime#3053

[non-BFB]
rljacob pushed a commit that referenced this pull request Apr 12, 2021
Update CIME to ESMCI cime5.8.1

Squash merge of jgfouca/branch-for-to-acme-2019-04-08

Features:
* Support optional components by filling in stub models for any missing component class
* cprnc: Make FIELDLIST message more informative
* cprnc: allow differences in field lists for time-constant fields

Bug fixes:
* The check for an rpointer.drv file did not consider multidriver mode.
* Fix ./case.build --clean
* PET and ERP tests were not setting compile_threaded correctly.
* Fixes inconsistency between MPASCICE and MPASSI, change all occurances of the former to the latter

This is non-BFB for A cases due to:
ESMCI/cime#3053

[non-BFB]
rljacob pushed a commit that referenced this pull request Apr 21, 2021
Update CIME to ESMCI cime5.8.1

Squash merge of jgfouca/branch-for-to-acme-2019-04-08

Features:
* Support optional components by filling in stub models for any missing component class
* cprnc: Make FIELDLIST message more informative
* cprnc: allow differences in field lists for time-constant fields

Bug fixes:
* The check for an rpointer.drv file did not consider multidriver mode.
* Fix ./case.build --clean
* PET and ERP tests were not setting compile_threaded correctly.
* Fixes inconsistency between MPASCICE and MPASSI, change all occurances of the former to the latter

This is non-BFB for A cases due to:
ESMCI/cime#3053

[non-BFB]
jgfouca pushed a commit that referenced this pull request Jun 26, 2024
…rtmgp-gen-baselines

Automatically Merged using E3SM Pull Request AutoTester
PR Title: Fix generate_baseline test for RRTMGP
PR Author: bartgol
PR LABELS: testing, BFB, radiation, AT: AUTOMERGE, bugfix, AT: Skip v1 Testing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BFB PR leaves answers BFB CIME Scripts
Projects
None yet
Development

Successfully merging this pull request may close these issues.