-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Quantile methods #588
Quantile methods #588
Conversation
… values of the info routine.
…ensemble members are identical (sd is 0). The inflation, regression, and obs_increments are already bullet-proofed for the 0 sd case. In the long-term, need to worry about what happens if sd is very small as opposed to formally zero.
Merge branch 'main' of https://github.com/NCAR/DART
i8 to i4 conversion for gfortran in model_mod. See issue #352 for more details on various argument mismatches we will need to fix for newer compilers
cice-scm2 quantile methods
…form of current observation. This code now reproduces across processor count whereas it crashed with more than on process before. Also changed the default parameters for the tracer flow part of the model and the equation for finding the velocity.
… Fixed bug in setting the filter type in algorithm_info.
…a numerical error if the input state and the two smallest ensemble members were identical but not all ensemble members were identical. Pretty rare situation in most applications.
Moved the normal distribution routines to their own module.
Newton search in gamma_distribution_mod to match the revised version in the beta mod, and added a log_normal capability in quantile_distribution_mod
…d a beta_distribution module that was developed by Chris Riedel. Added in testing code to the gamma, beta and normal distributions modules and made the algorithms more efficient. Added a higher accuracy inverse cdf for the normal module but it is switched off for now. It is much more expensive in return for improved accuracy.
Bitwise duplicates previous version.
… by 0. This also addresses the cases where ensemble members in the RH dupliate the bounds or each other in the interior. This bitwise reproduces previous version for cases that do not have duplicates but has not been tested for duplicates.
…model to include a fixed sink so that the concentration goes to 0 remote from the source. This mimics the kind of issues that are found with sea ice freezing entirely. Tests of this have run successfully with a full suite of RH methods.
…oth upper and lower bounds but were more serious for upper bound cases. This version works for long tests with a variety of filter settings for both positive and negative tracers (bounded below and above) with l96_tracer, but still needs much more comprehensive review and error checking.
a namelist parameter to turn the use of algorithm_info_mod on or off. If use_algorithm_info_mod is false, default namelist parameters control the filter as in previous versions.
…ormal in quantile_distribution_mod to try to catch errors Molly is reporint with CICE. Added use_algorithm_info namelist into perfect_model_obs and filter_mod to control whether default namelist is used or algorithm_info_mod. Have now set the default to .true. since this branch is supposed to be testing these algorithm_info_mod capabilities.
…e that this revealed the fact that gamma gamma products are not closed. If the sum of the shape parameters of the prior and the likelihood is <= 1, the posterior is not a gamma. An error message catches this, but it makes gammas a poor choice for quantities that can get close to (or exactly to) the bounds.
…rameter to flip the tracer from positive to negative to check assimilation code for bounded above quantities.
with l96 tracer model. Removed sampling_error_correction from the input.nml to avoid the need for copying over the sampling error table file to the work directory. Sampling error correction is useful here but it's not essential. Added the readme.rst even though it's not yet rst to enable friendly testers to get started.
only typo changes, no other changes to text
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I fixed the typos mentioned in the comments of this review in PR six twelve. Approving the six twelve merge will reconcile the typos and will add substantive edits to index.rst
.
I added the substantive edits to index.rst because the quantile methods merge adds a lot of new information to the documentation that DART users may be unfamiliar with. It may be easier to read by following a standard practice in technical writing: using topic and stress positions in sentences. The topic position in a sentence contains information that a user already knows, or is presumed to know, while the stress position introduces new information.
After running through the four examples in guide/qceff-examples.rst
and plotting their output with diagnostics/matlab/plot_ens_time_series.m, my conclusion is that
- Further exposition to the QCEFF examples should be added, or
- The examples should be modified to make the benefits of the new framework more evident
|
||
* NORMAL_DISTRIBUTION (default) | ||
* BOUNDED_NORMAL_RH_DISTRIBUTION | ||
* GAMMA_DISTRIBUTION |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this still work when the sum of alpha and beta < 1?
These comments are now incorrect and should be removed since we are once again dividing by sqrt(2 PI) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The &probit_transform_nml and &algorithm_info_nml are missing from models/MITgcm_ocean/work/input.nml
.. code-block:: text | ||
|
||
git checkout quantile_methods | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These lines above (7-15) shoud be removed or edited before release. The tools will no longer be in the alpha release stage and the user will not have to checkout the quantile_methods branch
The Quantile-Conserving Ensemble Filter Framework (QCEFF) tools are in the alpha release stage.
The DART development team (dart@ucar.edu) would be happy to hear about your experiences and is
anxious to build scientific collaborations using these new capabilities.
To get started, make sure that you are on the quantile_methods branch of DART:
.. code-block:: text
git checkout quantile_methods
There is another comment like this at the very beginning of guide/qceff_probit.rst
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|
||
!!!elseif(p%distribution_type == PARTICLE_FILTER_DISTRIBUTION) then | ||
!!!call to_probit_particle(ens_size, state_ens, p, probit_ens, use_input_p, & | ||
!!!bounded_below, bounded_above, lower_bound, upper_bound) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we still planning on supporting distribution_type = PARTICLE_FILTER_DISTRIBUTION
in the future?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove PARTICLE_FILTER_DISTRIBUTION from list of available distributions
Lines 123 to 132 in d5c2164
Available distributions | |
------------------------ | |
* NORMAL_DISTRIBUTION (default) | |
* BOUNDED_NORMAL_RH_DISTRIBUTION | |
* GAMMA_DISTRIBUTION | |
* BETA_DISTRIBUTION | |
* LOG_NORMAL_DISTRIBUTION | |
* UNIFORM_DISTRIBUTION | |
* PARTICLE_FILTER_DISTRIBUTION |
since it is not available in this code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep I agree that the PARTICLE_FILTER_DISTRIBUTION can be put into a later release if we want. Good catch with removing it from this release of the documentation.
Should we also remove it from this comment in algorithm_info_mod?
! UNIFORM_DISTRIBUTION, and PARTICLE_FILTER_DISTRIBUTION. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mjs2369 it is not just the comment right? It is the code in algortihm_info_mod also:
DART/assimilation_code/modules/assimilation/algorithm_info_mod.f90
Lines 249 to 250 in 9e6b206
case ('PARTICLE_FILTER_DISTRIBUTION') | |
qceff_table_data(row)%probit_state%dist_type = PARTICLE_FILTER_DISTRIBUTION |
@jlaucar do you want particle_filter_distritbution in this release?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes we should remove that as well. I feel like we the code would potentially take a while to be ready for this release and we should probably put it in with a later one
If we do as such, should we also remove the commented out code pertaining to it in probit_transform_mod? Lines 187-9 and the subroutine to_probit_particle starting on line 378
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mjs2369 chatted with Jeff about this.
Conclusion: don't let people select PARTICLE_FILTER_DISTRIBUTION. Leave in the commented subroutine to_probit_particle (if we were being strict about code we would remove this, but this is science). I'll go ahead and remove case ('PARTICLE_FILTER_DISTRIBUTION')
algorithm_info_mod.f90. Once again thanks for the detailed review on this.
Cheers,
Helen
BKJ quantile methods edit
Co-authored-by: Marlena Smith <44214771+mjs2369@users.noreply.github.com>
doc changes merge in from #612 pull request. |
Add these in 8ed6c2e Add for any model/input.nml that was missing these.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This read and worked well overall -- I agreed with Ben's main comment that the documentation could benefit from more explicit discussion of the improvement that QCEFF can bring to bounded/non-linear quantities. I made a suggestion to add a figure to the introduction, and then maybe more discussion at the end that differentiates between the performance of example A and C vs. example B.
guide/qceff-examples.rst
Outdated
#. Examine the output with your favorite tools. Looking at the analysis ensemble | ||
for the tracer_concentration variables with indices near the source (location 1) | ||
and far downstream from the source (location 35) is interesting. Note that the | ||
source estimation capabilities of the model and filters are not being tested here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe I am too much in the weeds here, but why was the source quantity not influenced by the assimilation, and the updates restricted to the state(wind) and tracer concentration only? It seems that the source is an unobserved variable -- but maybe its not part of the Lorenz model, but a boundary condition?
…by sqrt(2PI) in obs_increment_rank_histogram removed 'under development' & revised 2008. #588 (comment)
Co-authored-by: Brett Raczka <bmraczka@ucar.edu>
Co-authored-by: Brett Raczka <bmraczka@ucar.edu>
see comment #588 (comment) for disccusion
…pdated Google Sheets template; Adjusting the format when logging the QCEFF table
Following the convenion of !!! to comment out this code see #588 (comment) for discusion
Description:
Adds a Quantile-Conserving Ensemble Filtering Framework to DART
Publications: QCEFF part1 QCEFF part 2.
The default QCEFF options are EAKF, normal distribution (no bounds).
This is a breaking change and will be v11 of DART
Namelist item removed from filter_nml
Two new required namelists (added to input.nml files):
assim_tools_mod
spread_restoration
is not supported in this versionalgorithm_info_mod
New probability distribution modules:
beta_distribution_mod
bnrh_distribution_mod (bounded normal rank histogram)
gamma_distribution_mod
normal_distribution_mod
probit_transform_mod
distribution_params_mod
Update to lorenz_96_tracer_advection:
1d forward operator updates:
Docs:
Fixes issue
review notes #458
Note this pull request includes the fix for #416
Todo:
After merge, create 'quantile_methods' tag (with a warning) so old readthedocs links point to docs.No, edit release notes not to point at quantile_methods.Types of changes
Documentation changes needed?
Tests
Please describe any tests you ran to verify your changes.
bitwise lorenz_96 with main for default qcf options
Jeff has run many algorithm tests.
Speed tests main vs. default qcf options Derecho
Timing:
Checklist for merging
Checklist for release
Testing Datasets