-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move temporal 'regridding' from multi_model_statistics preprocessor to regrid_time preprocessor #744
Comments
It would be really useful if regrid_time could then also check the calendar (as does mmstats, also see #685). I keep running into incompatible time coordinates because the calendars differ. |
Yes please! I (and @hb326) encountered problems with def _unify_time_coord(cube):
"""Unify time coordinate of cube."""
if not cube.coords('time', dim_coords=True):
return
time_coord = cube.coord('time')
dates_points = time_coord.units.num2date(time_coord.points)
dates_bounds = time_coord.units.num2date(time_coord.bounds)
new_units = Unit('days since 1850-01-01 00:00:00')
new_time_coord = iris.coords.DimCoord(
new_units.date2num(dates_points),
bounds=new_units.date2num(dates_bounds),
var_name='time',
standard_name='time',
long_name='time',
units=new_units,
)
coord_dims = cube.coord_dims('time')
cube.remove_coord('time')
cube.add_dim_coord(new_time_coord, coord_dims) It think it would make sense to include this in |
The new regression test for the daily statistics of a 365-day calendar failed because some days had time points at 00:00 and others at 12:00. Since we're planning to move time handling to regrid time altogether, this behaviour is consistent with regrid_time (see issue #744).
* Fix calendar units to 'days since 1850-01-01' on a standard calendar * More thorough check on source time frequency; align behaviour with regrid time; raise for daily data * Simplify _get_overlap * Align behaviour for union (full) and intersection (overlap) of time arrays * Add function to make all cubes use the same calendar. This function also checks the frequency of the cubes (previously in _datetime_to_int_days), this seems to be a much more logical place. _datetime_to_int_days now simplifies to a one-liner, and can probably be eliminated completely. * Remove _datetime_to_int_days, as we can just use the time points * Simplify and rename _slice_cube * Align _assemble_overlap_data more with _assemble_full_data * Align _assemble_full_data more with _assemble_overlap_data * Futher align assemble full and overlap data * Merge assemble full and overlap data. * Further simplify * Remove stuff about bounds and aux coords as it is not used anyway * Remove stuff about bounds and aux coords as it is not used anyway * Clean up tests and add tests for new functions * Valeriu's suggestions * fix tests * Realize data before making time slices * Avoid codacy 'pointless-statement' message * Address Bouwe's comments * Update esmvalcore/preprocessor/_multimodel.py Co-authored-by: Bouwe Andela <b.andela@esciencecenter.nl> * Don't change the calendar if not necessary * Use template cube's calendar and don't slice by time * Use num2date rather than cells() method * Apply suggestions from code review Co-authored-by: Bouwe Andela <b.andela@esciencecenter.nl> * fix bug for monthly data in unify time * add time bounds to output cube * Add missing var_name attribute to time coordinate * Force regrid of daily data and be more consistent with regrid time. The new regression test for the daily statistics of a 365-day calendar failed because some days had time points at 00:00 and others at 12:00. Since we're planning to move time handling to regrid time altogether, this behaviour is consistent with regrid_time (see issue #744). * Add long name to new time coordinates * Temporarily bypass coordinate check The reference data contains some metadata that is redundant or inconsistent. After updating the reference data, this can be reverted. * Revert "Temporarily bypass coordinate check" This reverts commit 2bd2d62. * Update reference data * Typo Co-authored-by: Stef Smeets <stefsmeets@users.noreply.github.com> Co-authored-by: Bouwe Andela <b.andela@esciencecenter.nl> Co-authored-by: Stef Smeets <stefsmeets@users.noreply.github.com>
I support the idea, but careful with the suggested fix above. That may be appropriate in a visualization script where you know the input, but I see two problems with it:
|
The idea would be to only do this for time frequencies where it does not matter, e.g. for monthly averages or lower frequencies it does not matter whether or not your calendar includes leap days, right? |
So you are saying this would be only applicable if both source and target frequency are monthly, seasonal, annual? There may still be a (small) problem of weights, but also this could be quite annoying since going from daily to monthly is probably an important use case. |
I think so. Or maybe it would still work for daily data if you use some sort of interpolation scheme? I think there is just no way you can meaningfully regrid data with a frequency lower than daily from a 360 day calendar to a 365 day calendar.
At the moment we have the preprocessor function |
Moving this to v2.6 since there is not open PR yet. |
Any chance this can make it for 2.6? Or should it be moved? |
At the moment, the preprocessor
multi_model_statistics
will also attempt to automatically 'regrid' the time coordinate of the incoming datasets. Other coordinates, such as horizontal and vertical coordinates, need to be explicitly regridded by defining a preprocesor in the recipe. It would be good to be consistent and make the temporal regridding more visible and move the temporal regridding to theregrid_time
preprocessor. If the datasets have a time coordinate, users would then need to specify theregrid_time
preprocessor in the recipe as well as as theregrid
andextract_levels
.This would also fix the problem described in #738, that
multi_model_statistics
does not work for variables that do not have a time coordinate.The text was updated successfully, but these errors were encountered: