bug: normal CDF not enough precision

[hkershaw-brown]

🐛

Describe the bug

1. List the steps someone needs to take to reproduce the bug.
   Run filter using the RHF (lorenz_96)

2. What was the expected outcome?
   Precision of the results for norm_cdf calculation using the Abramowitz and Stegun approach matches results from Matlab.

3. What actually happened?
   Lower precision results than expected. Here is the comparison to Matlab:
   10^-8 differences
   

Error Message

No error

Which model(s) are you working with?

Lorenz_96 but will apply to any model if running with RHF.

Notes:

We believe it is only RHF that is effected.
To test: other filter options to confirm, where else is norm_cdf used.

Screenshots

Here is an improved norm_cdf method using Fortran erf and erfc intrinsics comparison to matlab
10^-16 differences

Version of DART

Which version of DART are you using?
v10.X.X

Have you modified the DART code?

No, but the improved method is available on request (from Jeff)

Build information

Please describe:

1. mac
2. gfortran

To test:

• different compilers (intel, nvhpc)
• new method effect on other filters (confirm no effect).
• unit test for norm_cdf

[hkershaw-brown]

https://github.com/NCAR/DART/tree/norm_cdf

https://github.com/hkershaw-brown/DART/tree/norm_cdf

[hkershaw-brown]

From Jeff: no rush on this to get to main, since it is need for the new filter stuff, not the current main branch.

[hkershaw-brown]

closing as this is fixed on quantile_methods since at least
f4b97de