diff --git a/asv.conf.json b/asv.conf.json index 87682c73c7..46cd4839f2 100644 --- a/asv.conf.json +++ b/asv.conf.json @@ -2,7 +2,7 @@ // details on what can be included in this file. { "version": 1, - "project": "iris", + "project": "scitools-iris", "project_url": "https://github.com/SciTools/iris", "repo": ".", "environment_type": "conda", diff --git a/docs/iris/src/developers_guide/graphics_tests.rst b/docs/iris/src/developers_guide/graphics_tests.rst index 684ccfa4ab..2782f319ec 100644 --- a/docs/iris/src/developers_guide/graphics_tests.rst +++ b/docs/iris/src/developers_guide/graphics_tests.rst @@ -10,9 +10,10 @@ For this, a basic 'graphics test' assertion operation is provided in the method match against a stored reference. A "graphics test" is any test which employs this. -At present (Iris version 1.10), such tests include the testing for modules -`iris.tests.test_plot` and `iris.tests.test_quickplot`, and also some other -'legacy' style tests (as described in :ref:`developer_tests`). +At present, such tests include the testing for modules `iris.tests.test_plot` +and `iris.tests.test_quickplot`, all output plots from the gallery examples +(contained in `docs/iris/example_tests`), and a few other 'legacy' style tests +(as described in :ref:`developer_tests`). It is conceivable that new 'graphics tests' of this sort can still be added. However, as graphics tests are inherently "integration" style rather than true unit tests, results can differ with the installed versions of dependent @@ -38,80 +39,110 @@ Testing actual plot results introduces some significant difficulties : Graphics Testing Strategy ========================= -Prior to Iris 1.10, all graphics tests compared against a stored reference -image with a small tolerance on pixel values. +In the Iris Travis matrix, and over time, graphics tests must run with +multiple versions of Python, and of key dependencies such as matplotlib. +To make this manageable, the "check_graphic" test routine tests against +multiple alternative 'acceptable' results. It does this using an image "hash" +comparison technique which avoids storing reference images in the Iris +repository itself, to avoid space problems. -From Iris v1.11 onward, we want to support testing Iris against multiple -versions of matplotlib (and some other dependencies). -To make this manageable, we have now rewritten "check_graphic" to allow -multiple alternative 'correct' results without including many more images in -the Iris repository. This consists of : - * using a perceptual 'image hash' of the outputs (see - https://github.com/JohannesBuchner/imagehash) as the basis for checking + * The 'check_graphic' function uses a perceptual 'image hash' of the outputs + (see https://github.com/JohannesBuchner/imagehash) as the basis for checking test results. - * storing the hashes of 'known accepted results' for each test in a - database in the repo (which is actually stored in - ``lib/iris/tests/results/imagerepo.json``). - * storing associated reference images for each hash value in a separate public - repository, currently in https://github.com/SciTools/test-images-scitools , - allowing human-eye judgement of 'valid equivalent' results. - * a new version of the 'iris/tests/idiff.py' assists in comparing proposed - new 'correct' result images with the existing accepted ones. - -BRIEF... -There should be sufficient work-flow detail here to allow an iris developer to: - - * understand the new check graphic test process - * understand the steps to take and tools to use to add a new graphic test - * understand the steps to take and tools to use to diagnose and fix an graphic test failure - - -Basic workflow -============== - -If you notice that a graphics test in the Iris testing suite has failed -following changes in Iris or any of its dependencies, this is the process -you now need to follow: - -#. Create a directory in iris/lib/iris/tests called 'result_image_comparison'. -#. From your Iris root directory, run the tests by using the command: - ``python setup.py test``. -#. Navigate to iris/lib/iris/tests and run the command: ``python idiff.py``. - This will open a window for you to visually inspect the changes to the - graphic and then either accept or reject the new result. -#. Upon acceptance of a change or a new image, a copy of the output PNG file - is added to the reference image repository in - https://github.com/SciTools/test-images-scitools. The file is named - according to the image hash value, as ``.png``. -#. The hash value of the new result is added into the relevant set of 'valid - result hashes' in the image result database file, - ``tests/results/imagerepo.json``. -#. The tests must now be re-run, and the 'new' result should be accepted. - Occasionally there are several graphics checks in a single test, only the - first of which will be run should it fail. If this is the case, then you - may well encounter further graphical test failures in your next runs, and - you must repeat the process until all the graphical tests pass. -#. To add your changes to Iris, you need to make two pull requests. The first - should be made to the test-images-scitools repository, and this should - contain all the newly-generated png files copied into the folder named - 'image_files'. -#. The second pull request should be created in the Iris repository, and should - only include the change to the image results database - (``tests/results/imagerepo.json``) : - This pull request must contain a reference to the matching one in - test-images-scitools. + * The hashes of known 'acceptable' results for each test are stored in a + lookup dictionary, saved to the repo file + ``lib/iris/tests/results/imagerepo.json`` . + * An actual reference image for each hash value is stored in a *separate* + public repository : https://github.com/SciTools/test-iris-imagehash . + * The reference images allow human-eye assessment of whether a new output is + judged to be 'close enough' to the older ones, or not. + * The utility script ``iris/tests/idiff.py`` automates checking, enabling the + developer to easily compare proposed new 'acceptable' result images against the + existing accepted reference images, for each failing test. -Note: the Iris pull-request will not test out successfully in Travis until the -test-images-scitools pull request has been merged : This is because there is -an Iris test which ensures the existence of the reference images (uris) for all -the targets in the image results database. +How to Add New 'Acceptable' Result Images to Existing Tests +======================================== + +When you find that a graphics test in the Iris testing suite has failed, +following changes in Iris or the run dependencies, this is the process +you should follow: + +#. Create a new, empty directory to store temporary image results, at the path + ``lib/iris/tests/result_image_comparison`` in your Iris repository checkout. + +#. **In your Iris repo root directory**, run the relevant (failing) tests + directly as python scripts, or by using a command such as + ``python -m unittest discover paths/to/test/files``. + +#. **In the** ``iris/lib/iris/tests`` **folder**, run the command: ``python idiff.py``. + This will open a window for you to visually inspect side-by-side 'old', 'new' + and 'difference' images for each failed graphics test. + Hit a button to either "accept", "reject" or "skip" each new result ... + + * If the change is *"accepted"* : + + * the imagehash value of the new result image is added into the relevant + set of 'valid result hashes' in the image result database file, + ``tests/results/imagerepo.json`` ; + + * the relevant output file in ``tests/result_image_comparison`` is + renamed according to the image hash value, as ``.png``. + A copy of this new PNG file must then be added into the reference image + repository at https://github.com/SciTools/test-iris-imagehash. + (See below). + + * If a change is *"skipped"* : + + * no further changes are made in the repo. + + * when you run idiff again, the skipped choice will be presented again. -Fixing a failing graphics test -============================== + * If a change is *"rejected"* : + * the output image is deleted from ``result_image_comparison``. -Adding a new graphics test -========================== + * when you run idiff again, the skipped choice will not appear, unless + and until the relevant failing test is re-run. + +#. Now re-run the tests. The 'new' result should now be recognised and the + relevant test should pass. However, some tests can perform *multiple* graphics + checks within a single testcase function : In those cases, any failing + check will prevent the following ones from being run, so a test re-run may + encounter further (new) graphical test failures. If that happens, simply + repeat the check-and-accept process until all tests pass. + +#. To add your changes to Iris, you need to make two pull requests : + + * (1) The first PR is made in the test-iris-imagehash repository, at + https://github.com/SciTools/test-iris-imagehash. + + * First, add all the newly-generated referenced PNG files into the + ``images/v4`` directory. In your Iris repo, these files are to be found + in the temporary results folder ``iris/tests/result_image_comparison``. + + .. Note:: + + The ``result_image_comparison`` folder is covered by a project + ``.gitignore`` setting, so those files *will not show up* in a + ``git status`` check. + + * Then, run ``python recreate_v4_files_listing.py``, to update the file + which lists available images, ``v4_files_listing.txt``. + + * Create a PR proposing these changes, in the usual way. + + * (2) The second PR is created in the Iris repository, and + should only include the change to the image results database, + ``tests/results/imagerepo.json`` : + The description box of this pull request should contain a reference to + the matching one in test-iris-imagehash. + +Note: the Iris pull-request will not test out successfully in Travis until the +test-iris-imagehash pull request has been merged : This is because there is +an Iris test which ensures the existence of the reference images (uris) for all +the targets in the image results database. N.B. likewise, it will *also* fail +if you forgot to run ``recreate_v4_files_listing.py`` to update the image-listing +file in test-iris-imagehash. diff --git a/docs/iris/src/developers_guide/tests.rst b/docs/iris/src/developers_guide/tests.rst index 929073b569..417db96f32 100644 --- a/docs/iris/src/developers_guide/tests.rst +++ b/docs/iris/src/developers_guide/tests.rst @@ -139,16 +139,5 @@ This the only way of testing the modules :mod:`iris.plot` and :mod:`iris.quickplot`, but is also used for some other legacy and integration- style testcases. -Prior to Iris version 1.10, a single reference image for each testcase was -stored in the main Iris repository, and a 'tolerant' comparison was performed -against this. - -From version 1.11 onwards, graphics testcase outputs are compared against -possibly *multiple* known-good images, of which only the signature is stored. -This uses a sophisticated perceptual "image hashing" scheme (see: -). -Only imagehash signatures are stored in the Iris repo itself, thus freeing up -valuable space. Meanwhile, the actual reference *images* -- which are required -for human-eyes evaluation of proposed new "good results" -- are all stored -elsewhere in a separate public repository. +There are specific mechanisms for handling this. See :ref:`developer_graphics_tests`. diff --git a/docs/iris/src/userguide/subsetting_a_cube.rst b/docs/iris/src/userguide/subsetting_a_cube.rst index b61f16a043..5864de531a 100644 --- a/docs/iris/src/userguide/subsetting_a_cube.rst +++ b/docs/iris/src/userguide/subsetting_a_cube.rst @@ -103,6 +103,9 @@ same way as loading with constraints: Cube iteration ^^^^^^^^^^^^^^^ +It is not possible to directly iterate over an Iris cube. That is, you cannot use code such as +``for x in cube:``. However, you can iterate over cube slices, as this section details. + A useful way of dealing with a Cube in its **entirety** is by iterating over its layers or slices. For example, to deal with a 3 dimensional cube (z,y,x) you could iterate over all 2 dimensional slices in y and x which make up the full 3d cube.:: diff --git a/docs/iris/src/whatsnew/2.4.rst b/docs/iris/src/whatsnew/2.4.rst new file mode 100644 index 0000000000..2facb97a7a --- /dev/null +++ b/docs/iris/src/whatsnew/2.4.rst @@ -0,0 +1,59 @@ +What's New in Iris 2.4.0 +************************ + +:Release: 2.4.0 +:Date: 2020-02-20 + +This document explains the new/changed features of Iris in version 2.4.0 +(:doc:`View all changes `.) + + +Iris 2.4.0 Features +=================== + +.. admonition:: Last python 2 version of Iris + + Iris 2.4 is a final extra release of Iris 2, which back-ports specific desired features from + Iris 3 (not yet released). + + The purpose of this is both to support early adoption of certain newer features, + and to provide a final release for Python 2. + + The next release of Iris will be version 3.0 : a major-version release which + introduces breaking API and behavioural changes, and only supports Python 3. + +* :class:`iris.coord_systems.Geostationary` can now accept creation arguments of + `false_easting=None` or `false_northing=None`, equivalent to values of 0. + Previously these kwargs could be omitted, but could not be set to `None`. + This also enables loading of netcdf data on a Geostationary grid, where either of these + keys is not present as a grid-mapping variable property : Previously, loading any + such data caused an exception. +* The area weights used when performing area weighted regridding with :class:`iris.analysis.AreaWeighted` + are now cached. + This allows a significant speedup when regridding multiple similar cubes, by repeatedly using + a `'regridder' object <../iris/iris/analysis.html?highlight=regridder#iris.analysis.AreaWeighted.regridder>`_ + which you created first. +* Name constraint matching against cubes during loading or extracting has been relaxed from strictly matching + against the :meth:`~iris.cube.Cube.name`, to matching against either the + ``standard_name``, ``long_name``, NetCDF ``var_name``, or ``STASH`` attributes metadata of a cube. +* Cubes and coordinates now have a new ``names`` property that contains a tuple of the + ``standard_name``, ``long_name``, NetCDF ``var_name``, and ``STASH`` attributes metadata. +* The :class:`~iris.NameConstraint` provides richer name constraint matching when loading or extracting + against cubes, by supporting a constraint against any combination of + ``standard_name``, ``long_name``, NetCDF ``var_name`` and ``STASH`` + from the attributes dictionary of a :class:`~iris.cube.Cube`. + + +Iris 2.4.0 Dependency Updates +============================= +* Iris is now able to use the latest version of matplotlib. + + +Bugs Fixed +========== +* Fixed a problem which was causing file loads to fetch *all* field data + whenever UM files (PP or Fieldsfiles) were loaded. + With large sourcefiles, initial file loads are slow, with large memory usage + before any cube data is even fetched. Large enough files will cause a crash. + The problem occurs only with Dask versions >= 2.0. + diff --git a/docs/iris/src/whatsnew/contributions_3.0.0/bugfix_2020-Feb-13_cube_iter_remove.txt b/docs/iris/src/whatsnew/contributions_3.0.0/bugfix_2020-Feb-13_cube_iter_remove.txt new file mode 100644 index 0000000000..082cd8acc8 --- /dev/null +++ b/docs/iris/src/whatsnew/contributions_3.0.0/bugfix_2020-Feb-13_cube_iter_remove.txt @@ -0,0 +1,3 @@ +* The `__iter__()` method in class:`iris.cube.Cube` was set to `None`. + `TypeError` is still raised if a `Cube` is iterated over but + `isinstance(cube, collections.Iterable)` now behaves as expected. \ No newline at end of file diff --git a/docs/iris/src/whatsnew/index.rst b/docs/iris/src/whatsnew/index.rst index 179216ccb5..03834a43a7 100644 --- a/docs/iris/src/whatsnew/index.rst +++ b/docs/iris/src/whatsnew/index.rst @@ -11,6 +11,7 @@ Iris versions. latest.rst 3.0.rst + 2.4.rst 2.3.rst 2.2.rst 2.1.rst diff --git a/lib/iris/coord_categorisation.py b/lib/iris/coord_categorisation.py index 838874b5f9..d299cbeaa4 100644 --- a/lib/iris/coord_categorisation.py +++ b/lib/iris/coord_categorisation.py @@ -182,7 +182,7 @@ def add_day_of_year(cube, coord, name="day_of_year"): def add_weekday_number(cube, coord, name="weekday_number"): """Add a categorical weekday coordinate, values 0..6 [0=Monday].""" add_categorised_coord( - cube, name, coord, lambda coord, x: _pt_date(coord, x).weekday() + cube, name, coord, lambda coord, x: _pt_date(coord, x).dayofwk ) @@ -192,7 +192,7 @@ def add_weekday_fullname(cube, coord, name="weekday_fullname"): cube, name, coord, - lambda coord, x: calendar.day_name[_pt_date(coord, x).weekday()], + lambda coord, x: calendar.day_name[_pt_date(coord, x).dayofwk], units="no_unit", ) @@ -203,7 +203,7 @@ def add_weekday(cube, coord, name="weekday"): cube, name, coord, - lambda coord, x: calendar.day_abbr[_pt_date(coord, x).weekday()], + lambda coord, x: calendar.day_abbr[_pt_date(coord, x).dayofwk], units="no_unit", ) diff --git a/lib/iris/cube.py b/lib/iris/cube.py index a246b97e1c..0fc381725d 100644 --- a/lib/iris/cube.py +++ b/lib/iris/cube.py @@ -2622,8 +2622,9 @@ def _repr_html_(self): representer = CubeRepresentation(self) return representer.repr_html() - def __iter__(self): - raise TypeError("Cube is not iterable") + # Indicate that the iter option is not available. Python will raise + # TypeError with a useful message if a Cube is iterated over. + __iter__ = None def __getitem__(self, keys): """ diff --git a/lib/iris/fileformats/pp.py b/lib/iris/fileformats/pp.py index a57d5b5116..f1bd51f645 100644 --- a/lib/iris/fileformats/pp.py +++ b/lib/iris/fileformats/pp.py @@ -38,7 +38,7 @@ ) import iris.fileformats.rules import iris.coord_systems - +from iris.util import _array_slice_ifempty try: import mo_pack @@ -594,19 +594,25 @@ def ndim(self): return len(self.shape) def __getitem__(self, keys): - with open(self.path, "rb") as pp_file: - pp_file.seek(self.offset, os.SEEK_SET) - data_bytes = pp_file.read(self.data_len) - data = _data_bytes_to_shaped_array( - data_bytes, - self.lbpack, - self.boundary_packing, - self.shape, - self.src_dtype, - self.mdi, - ) - data = data.__getitem__(keys) - return np.asanyarray(data, dtype=self.dtype) + # Check for 'empty' slicings, in which case don't fetch the data. + # Because, since Dask v2, 'dask.array.from_array' performs an empty + # slicing and we must not fetch the data at that time. + result = _array_slice_ifempty(keys, self.shape, self.dtype) + if result is None: + with open(self.path, "rb") as pp_file: + pp_file.seek(self.offset, os.SEEK_SET) + data_bytes = pp_file.read(self.data_len) + data = _data_bytes_to_shaped_array( + data_bytes, + self.lbpack, + self.boundary_packing, + self.shape, + self.src_dtype, + self.mdi, + ) + result = data.__getitem__(keys) + + return np.asanyarray(result, dtype=self.dtype) def __repr__(self): fmt = ( diff --git a/lib/iris/tests/__init__.py b/lib/iris/tests/__init__.py index d689738008..66966daaf4 100644 --- a/lib/iris/tests/__init__.py +++ b/lib/iris/tests/__init__.py @@ -1121,116 +1121,6 @@ class GraphicsTest_nometa(GraphicsTestMixin, IrisTest_nometa): pass -class TestGribMessage(IrisTest): - def assertGribMessageContents(self, filename, contents): - """ - Evaluate whether all messages in a GRIB2 file contain the provided - contents. - - * filename (string) - The path on disk of an existing GRIB file - - * contents - An iterable of GRIB message keys and expected values. - - """ - messages = GribMessage.messages_from_filename(filename) - for message in messages: - for element in contents: - section, key, val = element - self.assertEqual(message.sections[section][key], val) - - def assertGribMessageDifference( - self, filename1, filename2, diffs, skip_keys=(), skip_sections=() - ): - """ - Evaluate that the two messages only differ in the ways specified. - - * filename[0|1] (string) - The path on disk of existing GRIB files - - * diffs - An dictionary of GRIB message keys and expected diff values: - {key: (m1val, m2val),...} . - - * skip_keys - An iterable of key names to ignore during comparison. - - * skip_sections - An iterable of section numbers to ignore during comparison. - - """ - messages1 = list(GribMessage.messages_from_filename(filename1)) - messages2 = list(GribMessage.messages_from_filename(filename2)) - self.assertEqual(len(messages1), len(messages2)) - for m1, m2 in zip(messages1, messages2): - m1_sect = set(m1.sections.keys()) - m2_sect = set(m2.sections.keys()) - - for missing_section in m1_sect ^ m2_sect: - what = ( - "introduced" if missing_section in m1_sect else "removed" - ) - # Assert that an introduced section is in the diffs. - self.assertIn( - missing_section, - skip_sections, - msg="Section {} {}".format(missing_section, what), - ) - - for section in m1_sect & m2_sect: - # For each section, check that the differences are - # known diffs. - m1_keys = set(m1.sections[section]._keys) - m2_keys = set(m2.sections[section]._keys) - - difference = m1_keys ^ m2_keys - unexpected_differences = difference - set(skip_keys) - if unexpected_differences: - self.fail( - "There were keys in section {} which \n" - "weren't in both messages and which weren't " - "skipped.\n{}" - "".format(section, ", ".join(unexpected_differences)) - ) - - keys_to_compare = m1_keys & m2_keys - set(skip_keys) - - for key in keys_to_compare: - m1_value = m1.sections[section][key] - m2_value = m2.sections[section][key] - msg = "{} {} != {}" - if key not in diffs: - # We have a key which we expect to be the same for - # both messages. - if isinstance(m1_value, np.ndarray): - # A large tolerance appears to be required for - # gribapi 1.12, but not for 1.14. - self.assertArrayAlmostEqual( - m1_value, m2_value, decimal=2 - ) - else: - self.assertEqual( - m1_value, - m2_value, - msg=msg.format(key, m1_value, m2_value), - ) - else: - # We have a key which we expect to be different - # for each message. - self.assertEqual( - m1_value, - diffs[key][0], - msg=msg.format(key, m1_value, diffs[key][0]), - ) - - self.assertEqual( - m2_value, - diffs[key][1], - msg=msg.format(key, m2_value, diffs[key][1]), - ) - - def skip_data(fn): """ Decorator to choose whether to run tests, based on the availability of diff --git a/lib/iris/tests/experimental/regrid/test_regrid_area_weighted_rectilinear_src_and_grid.py b/lib/iris/tests/experimental/regrid/test_regrid_area_weighted_rectilinear_src_and_grid.py index 4721aa44ee..7f8ad4b551 100644 --- a/lib/iris/tests/experimental/regrid/test_regrid_area_weighted_rectilinear_src_and_grid.py +++ b/lib/iris/tests/experimental/regrid/test_regrid_area_weighted_rectilinear_src_and_grid.py @@ -108,8 +108,9 @@ def _resampled_coord(coord, samplefactor): delta = 0.00001 * np.sign(upper - lower) * abs(bounds[0, 1] - bounds[0, 0]) lower = lower + delta upper = upper - delta + samples = int(len(bounds) * samplefactor) new_points, step = np.linspace( - lower, upper, len(bounds) * samplefactor, endpoint=False, retstep=True + lower, upper, samples, endpoint=False, retstep=True ) new_points += step * 0.5 new_coord = coord.copy(points=new_points) diff --git a/lib/iris/tests/integration/test_grib2.py b/lib/iris/tests/integration/test_grib2.py deleted file mode 100644 index 691c4469d3..0000000000 --- a/lib/iris/tests/integration/test_grib2.py +++ /dev/null @@ -1,364 +0,0 @@ -# Copyright Iris contributors -# -# This file is part of Iris and is released under the LGPL license. -# See COPYING and COPYING.LESSER in the root of the repository for full -# licensing details. -"""Integration tests for loading and saving GRIB2 files.""" - -# Import iris.tests first so that some things can be initialised before -# importing anything else. -import iris.tests as tests - -from cf_units import Unit -import numpy.ma as ma - -import iris -from iris import load_cube, save -from iris.coords import DimCoord -from iris.coord_systems import RotatedGeogCS -from iris.fileformats.pp import EARTH_RADIUS as UM_DEFAULT_EARTH_RADIUS -import iris.tests.stock as stock -from iris.util import is_regular - -# Grib support is optional. -if tests.GRIB_AVAILABLE: - from iris_grib import load_pairs_from_fields - from iris_grib.message import GribMessage - from iris_grib.grib_phenom_translation import GRIBCode - - -@tests.skip_data -@tests.skip_grib -class TestImport(tests.IrisTest): - def test_gdt1(self): - path = tests.get_data_path( - ("GRIB", "rotated_nae_t", "sensible_pole.grib2") - ) - cube = load_cube(path) - self.assertCMLApproxData(cube) - - def test_gdt90_with_bitmap(self): - path = tests.get_data_path(("GRIB", "umukv", "ukv_chan9.grib2")) - cube = load_cube(path) - # Pay particular attention to the orientation. - self.assertIsNot(cube.data[0, 0], ma.masked) - self.assertIs(cube.data[-1, 0], ma.masked) - self.assertIs(cube.data[0, -1], ma.masked) - self.assertIs(cube.data[-1, -1], ma.masked) - x = cube.coord("projection_x_coordinate").points - y = cube.coord("projection_y_coordinate").points - self.assertGreater(x[0], x[-1]) # Decreasing X coordinate - self.assertLess(y[0], y[-1]) # Increasing Y coordinate - # Check everything else. - self.assertCMLApproxData(cube) - - -@tests.skip_data -@tests.skip_grib -class TestPDT8(tests.IrisTest): - def setUp(self): - # Load from the test file. - file_path = tests.get_data_path( - ("GRIB", "time_processed", "time_bound.grib2") - ) - self.cube = load_cube(file_path) - - def test_coords(self): - # Check the result has main coordinates as expected. - for name, shape, is_bounded in [ - ("forecast_reference_time", (1,), False), - ("time", (1,), True), - ("forecast_period", (1,), True), - ("pressure", (1,), False), - ("latitude", (73,), False), - ("longitude", (96,), False), - ]: - coords = self.cube.coords(name) - self.assertEqual( - len(coords), - 1, - "expected one {!r} coord, found {}".format(name, len(coords)), - ) - (coord,) = coords - self.assertEqual( - coord.shape, - shape, - "coord {!r} shape is {} instead of {!r}.".format( - name, coord.shape, shape - ), - ) - self.assertEqual( - coord.has_bounds(), - is_bounded, - "coord {!r} has_bounds={}, expected {}.".format( - name, coord.has_bounds(), is_bounded - ), - ) - - def test_cell_method(self): - # Check the result has the expected cell method. - cell_methods = self.cube.cell_methods - self.assertEqual( - len(cell_methods), - 1, - "result has {} cell methods, expected one.".format( - len(cell_methods) - ), - ) - (cell_method,) = cell_methods - self.assertEqual(cell_method.coord_names, ("time",)) - - -@tests.skip_data -@tests.skip_grib -class TestPDT11(tests.TestGribMessage): - def test_perturbation(self): - path = tests.get_data_path( - ("NetCDF", "global", "xyt", "SMALL_hires_wind_u_for_ipcc4.nc") - ) - cube = load_cube(path) - # trim to 1 time and regular lats - cube = cube[0, 12:144, :] - crs = iris.coord_systems.GeogCS(6371229) - cube.coord("latitude").coord_system = crs - cube.coord("longitude").coord_system = crs - # add a realization coordinate - cube.add_aux_coord( - iris.coords.DimCoord( - points=1, standard_name="realization", units="1" - ) - ) - with self.temp_filename("testPDT11.GRIB2") as temp_file_path: - iris.save(cube, temp_file_path) - - # Check that various aspects of the saved file are as expected. - expect_values = ( - (0, "editionNumber", 2), - (3, "gridDefinitionTemplateNumber", 0), - (4, "productDefinitionTemplateNumber", 11), - (4, "perturbationNumber", 1), - (4, "typeOfStatisticalProcessing", 0), - (4, "numberOfForecastsInEnsemble", 255), - ) - self.assertGribMessageContents(temp_file_path, expect_values) - - -@tests.skip_grib -class TestPDT40(tests.IrisTest): - def test_save_load(self): - cube = stock.lat_lon_cube() - cube.rename("atmosphere_mole_content_of_ozone") - cube.units = Unit("Dobson") - tcoord = DimCoord( - 23, "time", units=Unit("days since epoch", calendar="standard") - ) - fpcoord = DimCoord(24, "forecast_period", units=Unit("hours")) - cube.add_aux_coord(tcoord) - cube.add_aux_coord(fpcoord) - cube.attributes["WMO_constituent_type"] = 0 - cube.attributes["GRIB_PARAM"] = GRIBCode("GRIB2:d000c014n000") - - with self.temp_filename("test_grib_pdt40.grib2") as temp_file_path: - save(cube, temp_file_path) - loaded = load_cube(temp_file_path) - self.assertEqual(loaded.attributes, cube.attributes) - - -@tests.skip_data -@tests.skip_grib -class TestGDT5(tests.TestGribMessage): - def test_save_load(self): - # Load sample UKV data (variable-resolution rotated grid). - path = tests.get_data_path(("PP", "ukV1", "ukVpmslont.pp")) - cube = load_cube(path) - - # Extract a single 2D field, for simplicity. - self.assertEqual(cube.ndim, 3) - self.assertEqual(cube.coord_dims("time"), (0,)) - cube = cube[0] - - # FOR NOW: **also** fix the data so that it is square, i.e. nx=ny. - # This is needed because of a bug in the gribapi. - # See : https://software.ecmwf.int/issues/browse/SUP-1096 - ny, nx = cube.shape - nn = min(nx, ny) - cube = cube[:nn, :nn] - - # Check that it has a rotated-pole variable-spaced grid, as expected. - x_coord = cube.coord(axis="x") - self.assertIsInstance(x_coord.coord_system, RotatedGeogCS) - self.assertFalse(is_regular(x_coord)) - - # Write to temporary file, check that key contents are in the file, - # then load back in. - with self.temp_filename("ukv_sample.grib2") as temp_file_path: - save(cube, temp_file_path) - - # Check that various aspects of the saved file are as expected. - expect_values = ( - (0, "editionNumber", 2), - (3, "gridDefinitionTemplateNumber", 5), - (3, "Ni", cube.shape[-1]), - (3, "Nj", cube.shape[-2]), - (3, "shapeOfTheEarth", 1), - ( - 3, - "scaledValueOfRadiusOfSphericalEarth", - int(UM_DEFAULT_EARTH_RADIUS), - ), - (3, "resolutionAndComponentFlags", 0), - (3, "latitudeOfSouthernPole", -37500000), - (3, "longitudeOfSouthernPole", 357500000), - (3, "angleOfRotation", 0), - ) - self.assertGribMessageContents(temp_file_path, expect_values) - - # Load the Grib file back into a new cube. - cube_loaded_from_saved = load_cube(temp_file_path) - # Also load data, before the temporary file gets deleted. - cube_loaded_from_saved.data - - # The re-loaded result will not match the original in every respect: - # * cube attributes are discarded - # * horizontal coordinates are rounded to an integer representation - # * bounds on horizontal coords are lost - # Thus the following "equivalence tests" are rather piecemeal.. - - # Check those re-loaded properties which should match the original. - for test_cube in (cube, cube_loaded_from_saved): - self.assertEqual( - test_cube.standard_name, "air_pressure_at_sea_level" - ) - self.assertEqual(test_cube.units, "Pa") - self.assertEqual(test_cube.shape, (744, 744)) - self.assertEqual(test_cube.cell_methods, ()) - - # Check only the GRIB_PARAM attribute exists on the re-loaded cube. - # Note: this does *not* match the original, but is as expected. - self.assertEqual( - cube_loaded_from_saved.attributes, - {"GRIB_PARAM": GRIBCode("GRIB2:d000c003n001")}, - ) - - # Now remaining to check: coordinates + data... - - # Check they have all the same coordinates. - co_names = [coord.name() for coord in cube.coords()] - co_names_reload = [ - coord.name() for coord in cube_loaded_from_saved.coords() - ] - self.assertEqual(sorted(co_names_reload), sorted(co_names)) - - # Check all the coordinates. - for coord_name in co_names: - try: - co_orig = cube.coord(coord_name) - co_load = cube_loaded_from_saved.coord(coord_name) - - # Check shape. - self.assertEqual( - co_load.shape, - co_orig.shape, - 'Shape of re-loaded "{}" coord is {} ' - "instead of {}".format( - coord_name, co_load.shape, co_orig.shape - ), - ) - - # Check coordinate points equal, within a tolerance. - self.assertArrayAllClose( - co_load.points, co_orig.points, rtol=1.0e-6 - ) - - # Check all coords are unbounded. - # (NOTE: this is not so for the original X and Y coordinates, - # but Grib does not store those bounds). - self.assertIsNone(co_load.bounds) - - except AssertionError as err: - self.assertTrue( - False, - 'Failed on coordinate "{}" : {}'.format( - coord_name, str(err) - ), - ) - - # Check that main data array also matches. - self.assertArrayAllClose(cube.data, cube_loaded_from_saved.data) - - -@tests.skip_data -@tests.skip_grib -class TestGDT30(tests.IrisTest): - def test_lambert(self): - path = tests.get_data_path(("GRIB", "lambert", "lambert.grib2")) - cube = load_cube(path) - self.assertCMLApproxData(cube) - - -@tests.skip_data -@tests.skip_grib -class TestGDT40(tests.IrisTest): - def test_regular(self): - path = tests.get_data_path(("GRIB", "gaussian", "regular_gg.grib2")) - cube = load_cube(path) - self.assertCMLApproxData(cube) - - def test_reduced(self): - path = tests.get_data_path(("GRIB", "reduced", "reduced_gg.grib2")) - cube = load_cube(path) - self.assertCMLApproxData(cube) - - -@tests.skip_data -@tests.skip_grib -class TestDRT3(tests.IrisTest): - def test_grid_complex_spatial_differencing(self): - path = tests.get_data_path( - ("GRIB", "missing_values", "missing_values.grib2") - ) - cube = load_cube(path) - self.assertCMLApproxData(cube) - - -@tests.skip_data -@tests.skip_grib -class TestAsCubes(tests.IrisTest): - def setUp(self): - # Load from the test file. - self.file_path = tests.get_data_path( - ("GRIB", "time_processed", "time_bound.grib2") - ) - - def test_year_filter(self): - msgs = GribMessage.messages_from_filename(self.file_path) - chosen_messages = [] - for gmsg in msgs: - if gmsg.sections[1]["year"] == 1998: - chosen_messages.append(gmsg) - cubes_msgs = list(load_pairs_from_fields(chosen_messages)) - self.assertEqual(len(cubes_msgs), 1) - - def test_year_filter_none(self): - msgs = GribMessage.messages_from_filename(self.file_path) - chosen_messages = [] - for gmsg in msgs: - if gmsg.sections[1]["year"] == 1958: - chosen_messages.append(gmsg) - cubes_msgs = list(load_pairs_from_fields(chosen_messages)) - self.assertEqual(len(cubes_msgs), 0) - - def test_as_pairs(self): - messages = GribMessage.messages_from_filename(self.file_path) - cubes = [] - cube_msg_pairs = load_pairs_from_fields(messages) - for cube, gmsg in cube_msg_pairs: - if gmsg.sections[1]["year"] == 1998: - cube.attributes["the year is"] = gmsg.sections[1]["year"] - cubes.append(cube) - self.assertEqual(len(cubes), 1) - self.assertEqual(cubes[0].attributes["the year is"], 1998) - - -if __name__ == "__main__": - tests.main() diff --git a/lib/iris/tests/integration/test_grib_load.py b/lib/iris/tests/integration/test_grib_load.py deleted file mode 100644 index 0e7548ee34..0000000000 --- a/lib/iris/tests/integration/test_grib_load.py +++ /dev/null @@ -1,230 +0,0 @@ -# Copyright Iris contributors -# -# This file is part of Iris and is released under the LGPL license. -# See COPYING and COPYING.LESSER in the root of the repository for full -# licensing details. -""" -Integration tests for grib2 file loading. - -This code used to be part of 'tests/test_grib_load.py', but these integration- -style tests have been split out of there. - -The remainder of the old 'tests/test_grib_load.py' is now renamed as -'tests/test_grib_load_translations.py'. Those tests are implementation- -specific, and target the module 'iris_grib'. - -""" - -# Import iris tests first so that some things can be initialised before -# importing anything else -import iris.tests as tests - -import iris -import iris.exceptions -import iris.tests.stock -import iris.util -from unittest import skipIf - -# Skip out some tests that fail now that grib edition 2 files no longer use -# the GribWrapper. -# TODO: either fix these problems, or remove the tests. -skip_irisgrib_fails = skipIf( - True, "Test(s) are not currently usable with the new " "grib 2 loader." -) - - -@tests.skip_data -@tests.skip_grib -class TestBasicLoad(tests.GraphicsTest): - def test_load_rotated(self): - cubes = iris.load( - tests.get_data_path(("GRIB", "rotated_uk", "uk_wrongparam.grib1")) - ) - self.assertCML(cubes, ("grib_load", "rotated.cml")) - - def test_load_time_bound(self): - cubes = iris.load( - tests.get_data_path(("GRIB", "time_processed", "time_bound.grib1")) - ) - self.assertCML(cubes, ("grib_load", "time_bound_grib1.cml")) - - def test_load_time_processed(self): - cubes = iris.load( - tests.get_data_path(("GRIB", "time_processed", "time_bound.grib2")) - ) - self.assertCML(cubes, ("grib_load", "time_bound_grib2.cml")) - - def test_load_3_layer(self): - cubes = iris.load( - tests.get_data_path(("GRIB", "3_layer_viz", "3_layer.grib2")) - ) - cubes = iris.cube.CubeList([cubes[1], cubes[0], cubes[2]]) - self.assertCML(cubes, ("grib_load", "3_layer.cml")) - - def test_load_masked(self): - gribfile = tests.get_data_path( - ("GRIB", "missing_values", "missing_values.grib2") - ) - cubes = iris.load(gribfile) - self.assertCML(cubes, ("grib_load", "missing_values_grib2.cml")) - - @skip_irisgrib_fails - def test_y_fastest(self): - cubes = iris.load( - tests.get_data_path(("GRIB", "y_fastest", "y_fast.grib2")) - ) - self.assertCML(cubes, ("grib_load", "y_fastest.cml")) - - def test_polar_stereo_grib1(self): - cube = iris.load_cube( - tests.get_data_path(("GRIB", "polar_stereo", "ST4.2013052210.01h")) - ) - self.assertCML(cube, ("grib_load", "polar_stereo_grib1.cml")) - - def test_polar_stereo_grib2_grid_definition(self): - cube = iris.load_cube( - tests.get_data_path( - ( - "GRIB", - "polar_stereo", - "CMC_glb_TMP_ISBL_1015_ps30km_2013052000_P006.grib2", - ) - ) - ) - self.assertEqual(cube.shape, (200, 247)) - pxc = cube.coord("projection_x_coordinate") - self.assertAlmostEqual(pxc.points.max(), 4769905.5125, places=4) - self.assertAlmostEqual(pxc.points.min(), -2610094.4875, places=4) - pyc = cube.coord("projection_y_coordinate") - self.assertAlmostEqual(pyc.points.max(), -216.1459, places=4) - self.assertAlmostEqual(pyc.points.min(), -5970216.1459, places=4) - self.assertEqual(pyc.coord_system, pxc.coord_system) - self.assertEqual(pyc.coord_system.grid_mapping_name, "stereographic") - self.assertEqual(pyc.coord_system.central_lat, 90.0) - self.assertEqual(pyc.coord_system.central_lon, 249.0) - self.assertEqual(pyc.coord_system.false_easting, 0.0) - self.assertEqual(pyc.coord_system.false_northing, 0.0) - self.assertEqual(pyc.coord_system.true_scale_lat, 60.0) - - def test_lambert_grib1(self): - cube = iris.load_cube( - tests.get_data_path(("GRIB", "lambert", "lambert.grib1")) - ) - self.assertCML(cube, ("grib_load", "lambert_grib1.cml")) - - def test_lambert_grib2(self): - cube = iris.load_cube( - tests.get_data_path(("GRIB", "lambert", "lambert.grib2")) - ) - self.assertCML(cube, ("grib_load", "lambert_grib2.cml")) - - def test_regular_gg_grib1(self): - cube = iris.load_cube( - tests.get_data_path(("GRIB", "gaussian", "regular_gg.grib1")) - ) - self.assertCML(cube, ("grib_load", "regular_gg_grib1.cml")) - - def test_regular_gg_grib2(self): - cube = iris.load_cube( - tests.get_data_path(("GRIB", "gaussian", "regular_gg.grib2")) - ) - self.assertCML(cube, ("grib_load", "regular_gg_grib2.cml")) - - def test_reduced_ll(self): - cube = iris.load_cube( - tests.get_data_path(("GRIB", "reduced", "reduced_ll.grib1")) - ) - self.assertCML(cube, ("grib_load", "reduced_ll_grib1.cml")) - - def test_reduced_gg(self): - cube = iris.load_cube( - tests.get_data_path(("GRIB", "reduced", "reduced_gg.grib2")) - ) - self.assertCML(cube, ("grib_load", "reduced_gg_grib2.cml")) - - -@tests.skip_data -@tests.skip_grib -class TestIjDirections(tests.GraphicsTest): - @staticmethod - def _old_compat_load(name): - cube = iris.load(tests.get_data_path(("GRIB", "ij_directions", name)))[ - 0 - ] - return [cube] - - def test_ij_directions_ipos_jpos(self): - cubes = self._old_compat_load("ipos_jpos.grib2") - self.assertCML(cubes, ("grib_load", "ipos_jpos.cml")) - - def test_ij_directions_ipos_jneg(self): - cubes = self._old_compat_load("ipos_jneg.grib2") - self.assertCML(cubes, ("grib_load", "ipos_jneg.cml")) - - def test_ij_directions_ineg_jneg(self): - cubes = self._old_compat_load("ineg_jneg.grib2") - self.assertCML(cubes, ("grib_load", "ineg_jneg.cml")) - - def test_ij_directions_ineg_jpos(self): - cubes = self._old_compat_load("ineg_jpos.grib2") - self.assertCML(cubes, ("grib_load", "ineg_jpos.cml")) - - -@tests.skip_data -@tests.skip_grib -class TestShapeOfEarth(tests.GraphicsTest): - @staticmethod - def _old_compat_load(name): - cube = iris.load( - tests.get_data_path(("GRIB", "shape_of_earth", name)) - )[0] - return cube - - def test_shape_of_earth_basic(self): - # pre-defined sphere - cube = self._old_compat_load("0.grib2") - self.assertCML(cube, ("grib_load", "earth_shape_0.cml")) - - def test_shape_of_earth_custom_1(self): - # custom sphere - cube = self._old_compat_load("1.grib2") - self.assertCML(cube, ("grib_load", "earth_shape_1.cml")) - - def test_shape_of_earth_IAU65(self): - # IAU65 oblate sphere - cube = self._old_compat_load("2.grib2") - self.assertCML(cube, ("grib_load", "earth_shape_2.cml")) - - def test_shape_of_earth_custom_3(self): - # custom oblate spheroid (km) - cube = self._old_compat_load("3.grib2") - self.assertCML(cube, ("grib_load", "earth_shape_3.cml")) - - def test_shape_of_earth_IAG_GRS80(self): - # IAG-GRS80 oblate spheroid - cube = self._old_compat_load("4.grib2") - self.assertCML(cube, ("grib_load", "earth_shape_4.cml")) - - def test_shape_of_earth_WGS84(self): - # WGS84 - cube = self._old_compat_load("5.grib2") - self.assertCML(cube, ("grib_load", "earth_shape_5.cml")) - - def test_shape_of_earth_pre_6(self): - # pre-defined sphere - cube = self._old_compat_load("6.grib2") - self.assertCML(cube, ("grib_load", "earth_shape_6.cml")) - - def test_shape_of_earth_custom_7(self): - # custom oblate spheroid (m) - cube = self._old_compat_load("7.grib2") - self.assertCML(cube, ("grib_load", "earth_shape_7.cml")) - - def test_shape_of_earth_grib1(self): - # grib1 - same as grib2 shape 6, above - cube = self._old_compat_load("global.grib1") - self.assertCML(cube, ("grib_load", "earth_shape_grib1.cml")) - - -if __name__ == "__main__": - tests.main() diff --git a/lib/iris/tests/integration/test_pickle.py b/lib/iris/tests/integration/test_pickle.py index 5de0d1444c..e2a863610f 100644 --- a/lib/iris/tests/integration/test_pickle.py +++ b/lib/iris/tests/integration/test_pickle.py @@ -10,13 +10,9 @@ import iris.tests as tests import pickle -import unittest import iris -if tests.GRIB_AVAILABLE: - from iris_grib.message import GribMessage - class Common: def pickle_cube(self, protocol): @@ -39,44 +35,6 @@ def test_protocol_2(self): self.pickle_cube(2) -@tests.skip_data -@tests.skip_grib -class TestGribMessage(Common, tests.IrisTest): - def setUp(self): - self.path = tests.get_data_path(("GRIB", "fp_units", "hours.grib2")) - - def pickle_obj(self, obj): - with self.temp_filename(".pkl") as filename: - with open(filename, "wb") as f: - pickle.dump(obj, f) - - # These probably "ought" to work, but currently fail. - # see https://github.com/SciTools/iris/pull/2608 - @unittest.expectedFailure - def test_protocol_0(self): - super().test_protocol_0() - - @unittest.expectedFailure - def test_protocol_1(self): - super().test_protocol_1() - - @unittest.expectedFailure - def test_protocol_2(self): - super().test_protocol_2() - - def test(self): - # Check that a GribMessage pickles without errors. - messages = GribMessage.messages_from_filename(self.path) - obj = next(messages) - self.pickle_obj(obj) - - def test_data(self): - # Check that GribMessage.data pickles without errors. - messages = GribMessage.messages_from_filename(self.path) - obj = next(messages).data - self.pickle_obj(obj) - - @tests.skip_data class test_netcdf(Common, tests.IrisTest): def setUp(self): diff --git a/lib/iris/tests/results/grib_load/3_layer.cml b/lib/iris/tests/results/grib_load/3_layer.cml deleted file mode 100644 index 76cc41a04a..0000000000 --- a/lib/iris/tests/results/grib_load/3_layer.cml +++ /dev/null @@ -1,127 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_0.cml b/lib/iris/tests/results/grib_load/earth_shape_0.cml deleted file mode 100644 index bb51db3201..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_0.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_1.cml b/lib/iris/tests/results/grib_load/earth_shape_1.cml deleted file mode 100644 index 774e9921b5..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_1.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_2.cml b/lib/iris/tests/results/grib_load/earth_shape_2.cml deleted file mode 100644 index 3ff9ccccb5..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_2.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_3.cml b/lib/iris/tests/results/grib_load/earth_shape_3.cml deleted file mode 100644 index 47d11467ee..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_3.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_4.cml b/lib/iris/tests/results/grib_load/earth_shape_4.cml deleted file mode 100644 index e6aa14e45a..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_4.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_5.cml b/lib/iris/tests/results/grib_load/earth_shape_5.cml deleted file mode 100644 index 1257c9c2ad..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_5.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_6.cml b/lib/iris/tests/results/grib_load/earth_shape_6.cml deleted file mode 100644 index eb96657104..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_6.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_7.cml b/lib/iris/tests/results/grib_load/earth_shape_7.cml deleted file mode 100644 index d27ce04a4c..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_7.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/earth_shape_grib1.cml b/lib/iris/tests/results/grib_load/earth_shape_grib1.cml deleted file mode 100644 index 7ee99f8d74..0000000000 --- a/lib/iris/tests/results/grib_load/earth_shape_grib1.cml +++ /dev/null @@ -1,32 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/ineg_jneg.cml b/lib/iris/tests/results/grib_load/ineg_jneg.cml deleted file mode 100644 index a7d7741092..0000000000 --- a/lib/iris/tests/results/grib_load/ineg_jneg.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/ineg_jpos.cml b/lib/iris/tests/results/grib_load/ineg_jpos.cml deleted file mode 100644 index f578fceadb..0000000000 --- a/lib/iris/tests/results/grib_load/ineg_jpos.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/ipos_jneg.cml b/lib/iris/tests/results/grib_load/ipos_jneg.cml deleted file mode 100644 index bb51db3201..0000000000 --- a/lib/iris/tests/results/grib_load/ipos_jneg.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/ipos_jpos.cml b/lib/iris/tests/results/grib_load/ipos_jpos.cml deleted file mode 100644 index 4dc6d7f980..0000000000 --- a/lib/iris/tests/results/grib_load/ipos_jpos.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/lambert_grib1.cml b/lib/iris/tests/results/grib_load/lambert_grib1.cml deleted file mode 100644 index 74fe0a27fb..0000000000 --- a/lib/iris/tests/results/grib_load/lambert_grib1.cml +++ /dev/null @@ -1,38 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/lambert_grib2.cml b/lib/iris/tests/results/grib_load/lambert_grib2.cml deleted file mode 100644 index dc938f0aca..0000000000 --- a/lib/iris/tests/results/grib_load/lambert_grib2.cml +++ /dev/null @@ -1,37 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/missing_values_grib2.cml b/lib/iris/tests/results/grib_load/missing_values_grib2.cml deleted file mode 100644 index c4c0d81915..0000000000 --- a/lib/iris/tests/results/grib_load/missing_values_grib2.cml +++ /dev/null @@ -1,40 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/polar_stereo_grib1.cml b/lib/iris/tests/results/grib_load/polar_stereo_grib1.cml deleted file mode 100644 index f8e03e6d18..0000000000 --- a/lib/iris/tests/results/grib_load/polar_stereo_grib1.cml +++ /dev/null @@ -1,34 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/reduced_gg_grib2.cml b/lib/iris/tests/results/grib_load/reduced_gg_grib2.cml deleted file mode 100644 index fa3ba45e3d..0000000000 --- a/lib/iris/tests/results/grib_load/reduced_gg_grib2.cml +++ /dev/null @@ -1,46 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/reduced_ll_grib1.cml b/lib/iris/tests/results/grib_load/reduced_ll_grib1.cml deleted file mode 100644 index b1d68014fd..0000000000 --- a/lib/iris/tests/results/grib_load/reduced_ll_grib1.cml +++ /dev/null @@ -1,21 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/regular_gg_grib1.cml b/lib/iris/tests/results/grib_load/regular_gg_grib1.cml deleted file mode 100644 index b1dc1f6c81..0000000000 --- a/lib/iris/tests/results/grib_load/regular_gg_grib1.cml +++ /dev/null @@ -1,32 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/regular_gg_grib2.cml b/lib/iris/tests/results/grib_load/regular_gg_grib2.cml deleted file mode 100644 index 14213c1602..0000000000 --- a/lib/iris/tests/results/grib_load/regular_gg_grib2.cml +++ /dev/null @@ -1,35 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/rotated.cml b/lib/iris/tests/results/grib_load/rotated.cml deleted file mode 100644 index 06e2b517e1..0000000000 --- a/lib/iris/tests/results/grib_load/rotated.cml +++ /dev/null @@ -1,31 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/time_bound_grib1.cml b/lib/iris/tests/results/grib_load/time_bound_grib1.cml deleted file mode 100644 index 89902729b5..0000000000 --- a/lib/iris/tests/results/grib_load/time_bound_grib1.cml +++ /dev/null @@ -1,33 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/grib_load/time_bound_grib2.cml b/lib/iris/tests/results/grib_load/time_bound_grib2.cml deleted file mode 100644 index bb51db3201..0000000000 --- a/lib/iris/tests/results/grib_load/time_bound_grib2.cml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/imagerepo.json b/lib/iris/tests/results/imagerepo.json index e22b147de6..884e2b875f 100644 --- a/lib/iris/tests/results/imagerepo.json +++ b/lib/iris/tests/results/imagerepo.json @@ -141,7 +141,8 @@ "example_tests.test_rotated_pole_mapping.TestRotatedPoleMapping.test_rotated_pole_mapping.2": [ "https://scitools.github.io/test-iris-imagehash/images/v4/ba1e605ec7a191a1b85e9e81c4da58909996b37e3a65e16f7b817939e57a1e01.png", "https://scitools.github.io/test-iris-imagehash/images/v4/ba1e605ec7a193a1b85e9e81c4da58909996b3763a65e16f7b816939ed7a1e01.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/e85a697e97a18681c6da9f8190bf3e263624c1ef3b48c17a2b223c47c0ff3f81.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/e85a697e97a18681c6da9f8190bf3e263624c1ef3b48c17a2b223c47c0ff3f81.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/ea57685f95a886a1c0de9da090be3e2497e1c0ef3f01c17e6b366c17c07b3f01.png" ], "example_tests.test_rotated_pole_mapping.TestRotatedPoleMapping.test_rotated_pole_mapping.3": [ "https://scitools.github.io/test-iris-imagehash/images/v4/fa8172d0847ecd2bc913939c36846c714933799cc3cc8727e67639f939996a58.png", @@ -326,7 +327,8 @@ "iris.tests.test_plot.Test1dQuickplotPlotMultiArgs.test_cube_coord.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/83fec1ff7f90987720029f1ef458cd43811cdb60d647de609485ddb899215f62.png", "https://scitools.github.io/test-iris-imagehash/images/v4/83fec1ff7f94987720009f1ef458cd43810cdb60d647de609485ddb89921df62.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/a3ffc1de7e009c7030019786f438cde3810fd97c93734a778ce07c9f99b02731.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/a3ffc1de7e009c7030019786f438cde3810fd97c93734a778ce07c9f99b02731.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/a3ffc1de7e009c7030019786f438cde3810fd93c9b734a778ce47c9799b02731.png" ], "iris.tests.test_plot.Test1dQuickplotPlotMultiArgs.test_cube_cube.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/83ffc8967e0098a6241f9d26e34b8e42f4d20bb4942759e9941f78f8d7867a39.png", @@ -470,12 +472,14 @@ "iris.tests.test_plot.TestHybridHeight.test_orography.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/fa17291f95e895e8645e7a95c17a6eece4b4e1333b01c07e1bb13909914b9ec1.png", "https://scitools.github.io/test-iris-imagehash/images/v4/fa17291f95e895e8645e7a91c17a6ee464f4e1333b01c17e1bb1390d914b9ec1.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/fa817a91957a857ac4fe268cc07f6e846e05d9373b81d17b1b6a1b41c4fa2cc4.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/fa817a91957a857ac4fe268cc07f6e846e05d9373b81d17b1b6a1b41c4fa2cc4.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/fa817a91917a957ac4ff248cc07f6ea466a5c03f3b81c17f1b321b01935b3fc0.png" ], "iris.tests.test_plot.TestHybridHeight.test_orography.1": [ "https://scitools.github.io/test-iris-imagehash/images/v4/bb07314fc4e0c6b4c31e9ee1847939a1c116c15e7b94e57e1ea9391de16e1ac3.png", "https://scitools.github.io/test-iris-imagehash/images/v4/bb07314fc6e1c6b4c31e9ee1846939a1c116c15e7b14e17e1ea9393de16e1ac3.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/af0b690f96f0d2d4c25e94a194ad3da19a52c25e3f02c07f3fa52d03c16a3fcb.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/af0b690f96f0d2d4c25e94a194ad3da19a52c25e3f02c07f3fa52d03c16a3fcb.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/ea07695f95e0d2b4c09d95e0956a3da99294c2be3e85c07f3fa92b05c15e3f42.png" ], "iris.tests.test_plot.TestHybridHeight.test_points.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/ea953bfb956ac4f4649f1a05c56e6ca45a53945e6ea5c13f1b498542c13f1b41.png", @@ -694,12 +698,14 @@ "https://scitools.github.io/test-iris-imagehash/images/v4/aa953d0f85fab50fd0f2956a7a1785fafa176877d00f68f1d02c60f2f008d0f0.png", "https://scitools.github.io/test-iris-imagehash/images/v4/ebeaa5419e94b5019e97950d685395bee05361fad05560fad01570fef001dabe.png", "https://scitools.github.io/test-iris-imagehash/images/v4/ebeaa5419e95b5419e97950d6853953ee053617ad05560fad01570fef001dabe.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/ebfaa56f96a1856cd681a56ee8162d52e8467e12c50c7e8095ad7e0095ad03ff.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/ebfaa56f96a1856cd681a56ee8162d52e8467e12c50c7e8095ad7e0095ad03ff.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/eaa9b5699556854e9456854ed05625f9c0a92bfdc0a90afd81f97e00857e6af6.png" ], "iris.tests.test_plot.TestPlotCoordinatesGiven.test_tx.5": [ "https://scitools.github.io/test-iris-imagehash/images/v4/ebfaaf439e87b5019687b5019687b56ac05561fae07103fe6079687a607178f8.png", "https://scitools.github.io/test-iris-imagehash/images/v4/ebfa2d4b968795059e87970f6854697ae055697ac08561fad041d7aef001d6ae.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/eb7a3e0c978187a4950190bc6856687a607e687bc0fcc1e394acfc0197fc2bfb.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/eb7a3e0c978187a4950190bc6856687a607e687bc0fcc1e394acfc0197fc2bfb.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/eaf73e0d9503852c950395ac9528c1fad16cc0f2d1ec6af2c0ec6a536a1797f3.png" ], "iris.tests.test_plot.TestPlotCoordinatesGiven.test_x.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/aeb8b5095a87cd60386592d9ec97ad6dd23ca4f6d0797827f0096216c1f878e6.png", @@ -736,12 +742,14 @@ "https://scitools.github.io/test-iris-imagehash/images/v4/ad2f6d2fd2d09295c3c0c7d13c1bc6d23d2c696ce0e53c3ac393dbf6d205c2c0.png", "https://scitools.github.io/test-iris-imagehash/images/v4/ad2f6d2f92d09295c3d0c7d13c1bc6d23d2c696cf0e53c3ac2b3d9f6d201c2c4.png", "https://scitools.github.io/test-iris-imagehash/images/v4/e85e3e2f97a1c19996a1c8f26c1e360f684a3c2c6913dca497b9d38097a903ff.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/e85e3e3f96a1c3e197a169f1785e3b0e68523e1c398bc58687b1d86096e1039f.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/e85e3e3f96a1c3e197a169f1785e3b0e68523e1c398bc58687b1d86096e1039f.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/ea153e0395aac1f895eac0f8940e69e56a743e5f7a432787691ef860c3c1938f.png" ], "iris.tests.test_plot.TestPlotCoordinatesGiven.test_yx.5": [ "https://scitools.github.io/test-iris-imagehash/images/v4/e9686d8c9696924797879e3b86929e58696d69cc6869659379626133398d9ccd.png", "https://scitools.github.io/test-iris-imagehash/images/v4/e961658f961e92469e1e1c7966f36cd86165618c70e166b39b9698719e1e9ec8.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/e1a530e29e5ecf199a5acd8f64f1326161a530e265999cd29e52cf199a5e6669.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/e1a530e29e5ecf199a5acd8f64f1326161a530e265999cd29e52cf199a5e6669.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/e96930749696cb9d9697cdc39692671b696c306969eb3c76697319942a0d8699.png" ], "iris.tests.test_plot.TestPlotCoordinatesGiven.test_zx.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/bf803f00c05fc4bfc07ec15dc05fd8bbc07cc96c333a32113bd02dd27ced3ec0.png", @@ -765,12 +773,14 @@ "iris.tests.test_plot.TestPlotCoordinatesGiven.test_zx.4": [ "https://scitools.github.io/test-iris-imagehash/images/v4/ee953f0591ea3f07914a95fa7e07d1fa68156a15d07c6a3dd038c0fef000d0fa.png", "https://scitools.github.io/test-iris-imagehash/images/v4/ae953f0591ea3f07914a95fa7e07d1fa68156a15d07c6a7dd068c0fef000d0fa.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/bec11ab5c1be857ac13e7ae53c422d423e017a85b542fc00c1fefe0091fe03ff.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/bec11ab5c1be857ac13e7ae53c422d423e017a85b542fc00c1fefe0091fe03ff.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/bec13a81c13ec56ac13e5afdd11e256a3e412afd3e4002ff2ee0fe0035fa817a.png" ], "iris.tests.test_plot.TestPlotCoordinatesGiven.test_zx.5": [ "https://scitools.github.io/test-iris-imagehash/images/v4/e87a973d96a56953968769439685a54ae05117eae0511fba60513bba69717aba.png", "https://scitools.github.io/test-iris-imagehash/images/v4/e87a952d96a56953968769439685a54ae85197eae0511fba60513bba69717aba.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/e85a96ac97a16c5897a1791e95a53b0b913c6953687c4ec3685cc6c36e7c87c3.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/e85a96ac97a16c5897a1791e95a53b0b913c6953687c4ec3685cc6c36e7c87c3.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/ea1595ec95ea681d95ea7b0595ab3b13950d7a536a1cc6f26a0cc4f26e0c85f2.png" ], "iris.tests.test_plot.TestPlotDimAndAuxCoordsKwarg.test_coord_names.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/f9789b388786678686966c9093879ce592c79bc94d19929b6939cf66316c672c.png", @@ -842,12 +852,14 @@ ], "iris.tests.test_plot.TestSymbols.test_cloud_cover.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/e95a330c96a5ccf2695a330c96a5ccf2695a330c96b5ccf3694a330c96b5ccf3.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/eb52916494ad6e1b6b5291e494ad6e1b6b5291e494ad6e1b6b5291e494ad6e1b.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/eb52916494ad6e1b6b5291e494ad6e1b6b5291e494ad6e1b6b5291e494ad6e1b.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/eb5291e494ad6e136b5291ec94ad6e136b5291ec94ad6e136b5291ec94ad6e13.png" ], "iris.tests.test_quickplot.TestLabels.test_alignment.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/fa95350f952ad2f0c1f66ac1c55a4af4e550a52b3e05905e1e419e6f937e3b21.png", "https://scitools.github.io/test-iris-imagehash/images/v4/fa95350f952ad3f0c1f66a81e55a4af4e550a52b3e05905e1e419e6f937e1b21.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/be8137f4954ac03fc0ff3e81d03f496a6d00b4af3ea0c07f6fa232c0db7f2d00.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/be8137f4954ac03fc0ff3e81d03f496a6d00b4af3ea0c07f6fa232c0db7f2d00.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/be813fe0954ac07fc0ff3e81c03fc97a6d0094af3f80c17f36a53240d97f2d82.png" ], "iris.tests.test_quickplot.TestLabels.test_contour.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/a3fd956a7a01a5ee321fc96666919b6ec15fdca593600d2586785a259dfa5a01.png", @@ -921,18 +933,21 @@ "iris.tests.test_quickplot.TestQuickplotCoordinatesGiven.test_tx.4": [ "https://scitools.github.io/test-iris-imagehash/images/v4/aa97b70ff5f0970f20b2956a6a17957af805da71d06f5a75d02cd870d800d8f2.png", "https://scitools.github.io/test-iris-imagehash/images/v4/e1faa549de9497090697971d60539f3ef171c87ac075487ad025d87ed801da3e.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/eadab54fd7a1856d90819d6df8169962e946d862802ed8809ded7e809d2d03ff.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/eadab54fd7a1856d90819d6df8169962e946d862802ed8809ded7e809d2d03ff.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/eaa9b549f756854ea0169d6ad5568969d9a909ed80290afdd9e97e008d6e6a96.png" ], "iris.tests.test_quickplot.TestQuickplotCoordinatesGiven.test_tx.5": [ "https://scitools.github.io/test-iris-imagehash/images/v4/e8faad47f784bd0596859d03969f9962c05dc96ee07189fe6870c862687178f8.png", "https://scitools.github.io/test-iris-imagehash/images/v4/a8fa2d4797859585b6959d07605f896ee051697ad061d9fad0619aaed801deae.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/aa5b3c0c978187a4b60199bc605f6976687e6873d07c99e390acdc0391fc2f7b.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/aa5b3c0c978187a4b60199bc605f6976687e6873d07c99e390acdc0391fc2f7b.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/aad73e0df78085acb50195ac8029d9f2d16cd8f2d1ec48f280ec6a536a17b7f3.png" ], "iris.tests.test_quickplot.TestQuickplotCoordinatesGiven.test_x.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/a6ffb5097e84cde2224598d1649f8d6cd2388c76d0799867d009da76c9f8d866.png", "https://scitools.github.io/test-iris-imagehash/images/v4/a6bfb5097f84cde2224599d1649f8d6cd2388c76d0799867d009da76c1f8d866.png", "https://scitools.github.io/test-iris-imagehash/images/v4/a6fbb50cfbd0c036203598dce4c88d26d32f8cf3886e1df3dc047b4289ec6e72.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/a6fb958dfb50c03e203598dca4c9cd26933f9cb3886e1df1dc047b4289ec2e72.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/a6fb958dfb50c03e203598dca4c9cd26933f9cb3886e1df1dc047b4289ec2e72.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/a6fb958dff50c03e203598dca4c9cd26933f9cf3886e1de1dc047b4289ec2672.png" ], "iris.tests.test_quickplot.TestQuickplotCoordinatesGiven.test_y.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/a7ff978b7f00c9c830d7992166179e969509d866c478d964d079c876d869da26.png", @@ -962,12 +977,14 @@ "https://scitools.github.io/test-iris-imagehash/images/v4/ad2f6d2fd2d09295c2d1c3d33c1bc2d67d2c696ce0653c3ac2b1d976da05c2c4.png", "https://scitools.github.io/test-iris-imagehash/images/v4/ad2f6d2fd2d09295c2d1c3d33c1bc2d27d2c696ce0e53c3ad2b1d976da01c2c4.png", "https://scitools.github.io/test-iris-imagehash/images/v4/e85e3e2f97a1c19996a1c8f26d1e3a0f684a3c2c6913dc2497b9db8095e502ff.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/e85e3c1f97a1c3e197a1c9f37c5e390668521e0c390bdd8685b1d86096e5279f.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/e85e3c1f97a1c3e197a1c9f37c5e390668521e0c390bdd8685b1d86096e5279f.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/ea153f0395eac1f895eac9fa941c79e56a741e4f68430f876916f860c9c1938d.png" ], "iris.tests.test_quickplot.TestQuickplotCoordinatesGiven.test_yx.5": [ "https://scitools.github.io/test-iris-imagehash/images/v4/e968658e969692c797879e3b86929e58696d49cd6869c9a37962c923990d9c6d.png", "https://scitools.github.io/test-iris-imagehash/images/v4/e9e1658e961e92569e9e3c7966d36c586165698c70e1ce739b3698619e1e984c.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/e1a530e29e5ecf199a5acd8f64f1326161a538e665a198d29e52cb1d9a5e6669.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/e1a530e29e5ecf199a5acd8f64f1326161a538e665a198d29e52cb1d9a5e6669.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/e96930749696cf9d9697cdc39692670b696c386969eb3866696399a41a0d8e99.png" ], "iris.tests.test_quickplot.TestQuickplotCoordinatesGiven.test_zx.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/bf813f80c156c05dc0fec29dc17f1a6dd05fc0ff1aa1c57e3b243b20375a1e81.png", @@ -993,19 +1010,22 @@ "iris.tests.test_quickplot.TestQuickplotCoordinatesGiven.test_zx.4": [ "https://scitools.github.io/test-iris-imagehash/images/v4/ae953f87d5e82d86801f91ee6e1591fe7e117876c07d6877d068d878d800d07a.png", "https://scitools.github.io/test-iris-imagehash/images/v4/ae953f87d5e82d87801b91ee6e1599fe7e117874c07d6877d068d878d800d07a.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/bec1329dc5be85dac01d58d73e419d423e41daa59822dc00c5fefe0091fe03ff.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/bec1329dc5be85dac01d58d73e419d423e41daa59822dc00c5fefe0091fe03ff.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/bec13e81c5bec55ac03dd896d17e8d6a1e410af7380008ff1de6fe0099ea237b.png" ], "iris.tests.test_quickplot.TestQuickplotCoordinatesGiven.test_zx.5": [ "https://scitools.github.io/test-iris-imagehash/images/v4/e87a952d96856943969f694696858d4ee0519d6ee07f9b6a78619b2a79711a2a.png", "https://scitools.github.io/test-iris-imagehash/images/v4/e87a952d96856943969f694696858d4ae0519d6ee07f996a78719b2a79711a3a.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/e85e96ac97a168d897a5791695a19927913c3953687ecce3687c86e3487cc6c3.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/e85e96ac97a168d897a5791695a19927913c3953687ecce3687c86e3487cc6c3.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/ea1595ac95e8689d95fb7b0595291963916f3b73487fccf2680484f2486ec6f0.png" ], "iris.tests.test_quickplot.TestTimeReferenceUnitsLabels.test_not_reference_time_units.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/82faa1977fdf89976200ddf6e000d9e7f75f9866d560dae4dc00d966dc005e20.png", "https://scitools.github.io/test-iris-imagehash/images/v4/82b8a1977fdf89876200dde6e000d9e7f77f9866d560dfe4dc00d966fc005e20.png", "https://scitools.github.io/test-iris-imagehash/images/v4/82f8a1977fdf89876200ddf6e000d9e7f77f9866d560dee4dc00d966dc005e20.png", "https://scitools.github.io/test-iris-imagehash/images/v4/82f8a1977fdf89876200dde6e000d9e7f77f9866d560dfe4dc00dd64dc005e20.png", - "https://scitools.github.io/test-iris-imagehash/images/v4/82faa19e7f51898c6001dd86845fd9a2dd7f996281ee19f389ef03ffdc007e00.png" + "https://scitools.github.io/test-iris-imagehash/images/v4/82faa19e7f51898c6001dd86845fd9a2dd7f996281ee19f389ef03ffdc007e00.png", + "https://scitools.github.io/test-iris-imagehash/images/v4/82f8a19e7f51888c6001dda6855fd9a2dd7f986281ee19f389ff03ffdc007e00.png" ], "iris.tests.test_quickplot.TestTimeReferenceUnitsLabels.test_reference_time_units.0": [ "https://scitools.github.io/test-iris-imagehash/images/v4/82fe81987fd777ffe0002addd4002805dda8de65dde9d4625bfddc209841de20.png", diff --git a/lib/iris/tests/results/integration/grib2/TestDRT3/grid_complex_spatial_differencing.cml b/lib/iris/tests/results/integration/grib2/TestDRT3/grid_complex_spatial_differencing.cml deleted file mode 100644 index 2cfe06f8f6..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestDRT3/grid_complex_spatial_differencing.cml +++ /dev/null @@ -1,40 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/integration/grib2/TestDRT3/grid_complex_spatial_differencing.data.0.json b/lib/iris/tests/results/integration/grib2/TestDRT3/grid_complex_spatial_differencing.data.0.json deleted file mode 100644 index f42d355d31..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestDRT3/grid_complex_spatial_differencing.data.0.json +++ /dev/null @@ -1 +0,0 @@ -{"std": 7.798695691713748, "min": -34.43, "max": 33.009999999999998, "shape": [73, 144], "masked": true, "mean": 2.3147813807531383} \ No newline at end of file diff --git a/lib/iris/tests/results/integration/grib2/TestGDT30/lambert.cml b/lib/iris/tests/results/integration/grib2/TestGDT30/lambert.cml deleted file mode 100644 index 215a0de88d..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestGDT30/lambert.cml +++ /dev/null @@ -1,37 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/integration/grib2/TestGDT30/lambert.data.0.json b/lib/iris/tests/results/integration/grib2/TestGDT30/lambert.data.0.json deleted file mode 100644 index 99d58c5e34..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestGDT30/lambert.data.0.json +++ /dev/null @@ -1 +0,0 @@ -{"std": 5.3916288115779398, "min": 265.550048828125, "max": 300.862548828125, "shape": [799, 1199], "masked": false, "mean": 287.6306666328037} \ No newline at end of file diff --git a/lib/iris/tests/results/integration/grib2/TestGDT40/reduced.cml b/lib/iris/tests/results/integration/grib2/TestGDT40/reduced.cml deleted file mode 100644 index 3a963b3203..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestGDT40/reduced.cml +++ /dev/null @@ -1,46 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/integration/grib2/TestGDT40/reduced.data.0.json b/lib/iris/tests/results/integration/grib2/TestGDT40/reduced.data.0.json deleted file mode 100644 index 4cd44b531e..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestGDT40/reduced.data.0.json +++ /dev/null @@ -1 +0,0 @@ -{"std": 6295.5250434859099, "min": -6419.0146484375, "max": 55403.9853515625, "shape": [13280], "masked": false, "mean": 2446.3044780685241} \ No newline at end of file diff --git a/lib/iris/tests/results/integration/grib2/TestGDT40/regular.cml b/lib/iris/tests/results/integration/grib2/TestGDT40/regular.cml deleted file mode 100644 index e5eea0fc7c..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestGDT40/regular.cml +++ /dev/null @@ -1,35 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/integration/grib2/TestGDT40/regular.data.0.json b/lib/iris/tests/results/integration/grib2/TestGDT40/regular.data.0.json deleted file mode 100644 index 9595f90e89..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestGDT40/regular.data.0.json +++ /dev/null @@ -1 +0,0 @@ -{"std": 290.49181302067751, "min": 4388.16162109375, "max": 5576.53662109375, "shape": [96, 192], "masked": false, "mean": 5210.564598931207} \ No newline at end of file diff --git a/lib/iris/tests/results/integration/grib2/TestImport/gdt1.cml b/lib/iris/tests/results/integration/grib2/TestImport/gdt1.cml deleted file mode 100644 index d304d8a843..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestImport/gdt1.cml +++ /dev/null @@ -1,36 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/integration/grib2/TestImport/gdt1.data.0.json b/lib/iris/tests/results/integration/grib2/TestImport/gdt1.data.0.json deleted file mode 100644 index 2b416f6a93..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestImport/gdt1.data.0.json +++ /dev/null @@ -1 +0,0 @@ -{"std": 6.1629553729365423, "min": 266.625, "max": 302.25, "shape": [360, 600], "masked": false, "mean": 284.43164236111113} \ No newline at end of file diff --git a/lib/iris/tests/results/integration/grib2/TestImport/gdt90_with_bitmap.cml b/lib/iris/tests/results/integration/grib2/TestImport/gdt90_with_bitmap.cml deleted file mode 100644 index 3118f86823..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestImport/gdt90_with_bitmap.cml +++ /dev/null @@ -1,40 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/results/integration/grib2/TestImport/gdt90_with_bitmap.data.0.json b/lib/iris/tests/results/integration/grib2/TestImport/gdt90_with_bitmap.data.0.json deleted file mode 100644 index 08c3cb82cb..0000000000 --- a/lib/iris/tests/results/integration/grib2/TestImport/gdt90_with_bitmap.data.0.json +++ /dev/null @@ -1 +0,0 @@ -{"std": 13.780125280995835, "min": 208.90541992187502, "max": 287.25541992187499, "shape": [227, 390], "masked": true, "mean": 266.3984425053925} \ No newline at end of file diff --git a/lib/iris/tests/results/uri_callback/grib_global.cml b/lib/iris/tests/results/uri_callback/grib_global.cml deleted file mode 100644 index aef0310a96..0000000000 --- a/lib/iris/tests/results/uri_callback/grib_global.cml +++ /dev/null @@ -1,60 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/lib/iris/tests/system_test.py b/lib/iris/tests/system_test.py index bc97e4fc00..207bd700a3 100644 --- a/lib/iris/tests/system_test.py +++ b/lib/iris/tests/system_test.py @@ -76,19 +76,6 @@ def horiz_cs(): new_cube, ("system", "supported_filetype_%s.cml" % filetype) ) - @tests.skip_grib - def system_test_grib_patch(self): - import gribapi - - gm = gribapi.grib_new_from_samples("GRIB2") - _ = gribapi.grib_get_double(gm, "missingValue") - - new_missing_value = 123456.0 - gribapi.grib_set_double(gm, "missingValue", new_missing_value) - new_result = gribapi.grib_get_double(gm, "missingValue") - - self.assertEqual(new_result, new_missing_value) - def system_test_imports_general(self): if tests.MPL_AVAILABLE: import matplotlib # noqa diff --git a/lib/iris/tests/test_cdm.py b/lib/iris/tests/test_cdm.py index 2c006e0e4f..ab27ad6040 100644 --- a/lib/iris/tests/test_cdm.py +++ b/lib/iris/tests/test_cdm.py @@ -11,6 +11,7 @@ # import iris tests first so that some things can be initialised before importing anything else import iris.tests as tests +import collections import os import re @@ -690,6 +691,9 @@ def test_cube_iteration(self): for subcube in self.t: pass + def test_not_iterable(self): + self.assertFalse(isinstance(self.t, collections.Iterable)) + class Test2dSlicing(TestCube2d): def test_cube_slice_all_dimensions(self): diff --git a/lib/iris/tests/test_grib_load_translations.py b/lib/iris/tests/test_grib_load_translations.py deleted file mode 100644 index 692601d533..0000000000 --- a/lib/iris/tests/test_grib_load_translations.py +++ /dev/null @@ -1,415 +0,0 @@ -# Copyright Iris contributors -# -# This file is part of Iris and is released under the LGPL license. -# See COPYING and COPYING.LESSER in the root of the repository for full -# licensing details. -""" -Tests for specific implementation aspects of the grib loaders. - -""" - -# Import iris.tests first so that some things can be initialised before -# importing anything else -import iris.tests as tests - -import datetime -from unittest import mock - -import cf_units -import numpy as np - -import iris -import iris.exceptions -import iris.tests.stock -import iris.util - -if tests.GRIB_AVAILABLE: - import gribapi - import iris.fileformats - import iris_grib - - -def _mock_gribapi_fetch(message, key): - """ - Fake the gribapi key-fetch. - - Fetch key-value from the fake message (dictionary). - If the key is not present, raise the diagnostic exception. - - """ - if key in message: - return message[key] - else: - raise _mock_gribapi.errors.GribInternalError - - -def _mock_gribapi__grib_is_missing(grib_message, keyname): - """ - Fake the gribapi key-existence enquiry. - - Return whether the key exists in the fake message (dictionary). - - """ - return keyname not in grib_message - - -def _mock_gribapi__grib_get_native_type(grib_message, keyname): - """ - Fake the gribapi type-discovery operation. - - Return type of key-value in the fake message (dictionary). - If the key is not present, raise the diagnostic exception. - - """ - if keyname in grib_message: - return type(grib_message[keyname]) - raise _mock_gribapi.errors.GribInternalError(keyname) - - -if tests.GRIB_AVAILABLE: - # Construct a mock object to mimic the gribapi for GribWrapper testing. - _mock_gribapi = mock.Mock(spec=gribapi) - _mock_gribapi.errors.GribInternalError = Exception - - _mock_gribapi.grib_get_long = mock.Mock(side_effect=_mock_gribapi_fetch) - _mock_gribapi.grib_get_string = mock.Mock(side_effect=_mock_gribapi_fetch) - _mock_gribapi.grib_get_double = mock.Mock(side_effect=_mock_gribapi_fetch) - _mock_gribapi.grib_get_double_array = mock.Mock( - side_effect=_mock_gribapi_fetch - ) - _mock_gribapi.grib_is_missing = mock.Mock( - side_effect=_mock_gribapi__grib_is_missing - ) - _mock_gribapi.grib_get_native_type = mock.Mock( - side_effect=_mock_gribapi__grib_get_native_type - ) - -# define seconds in an hour, for general test usage -_hour_secs = 3600.0 - - -class FakeGribMessage(dict): - """ - A 'fake grib message' object, for testing GribWrapper construction. - - Behaves as a dictionary, containing key-values for message keys. - - """ - - def __init__(self, **kwargs): - """ - Create a fake message object. - - General keys can be set/add as required via **kwargs. - The keys 'edition' and 'time_code' are specially managed. - - """ - # Start with a bare dictionary - dict.__init__(self) - # Extract specially-recognised keys. - edition = kwargs.pop("edition", 1) - time_code = kwargs.pop("time_code", None) - # Set the minimally required keys. - self._init_minimal_message(edition=edition) - # Also set a time-code, if given. - if time_code is not None: - self.set_timeunit_code(time_code) - # Finally, add any remaining passed key-values. - self.update(**kwargs) - - def _init_minimal_message(self, edition=1): - # Set values for all the required keys. - # 'edition' controls the edition-specific keys. - self.update( - { - "Ni": 1, - "Nj": 1, - "numberOfValues": 1, - "alternativeRowScanning": 0, - "centre": "ecmf", - "year": 2007, - "month": 3, - "day": 23, - "hour": 12, - "minute": 0, - "indicatorOfUnitOfTimeRange": 1, - "shapeOfTheEarth": 6, - "gridType": "rotated_ll", - "angleOfRotation": 0.0, - "iDirectionIncrementInDegrees": 0.036, - "jDirectionIncrementInDegrees": 0.036, - "iScansNegatively": 0, - "jScansPositively": 1, - "longitudeOfFirstGridPointInDegrees": -5.70, - "latitudeOfFirstGridPointInDegrees": -4.452, - "jPointsAreConsecutive": 0, - "values": np.array([[1.0]]), - "indicatorOfParameter": 9999, - "parameterNumber": 9999, - } - ) - # Add edition-dependent settings. - self["edition"] = edition - if edition == 1: - self.update( - { - "startStep": 24, - "timeRangeIndicator": 1, - "P1": 2, - "P2": 0, - # time unit - needed AS WELL as 'indicatorOfUnitOfTimeRange' - "unitOfTime": 1, - "table2Version": 9999, - } - ) - if edition == 2: - self.update( - { - "iDirectionIncrementGiven": 1, - "jDirectionIncrementGiven": 1, - "uvRelativeToGrid": 0, - "forecastTime": 24, - "productDefinitionTemplateNumber": 0, - "stepRange": 24, - "discipline": 9999, - "parameterCategory": 9999, - "tablesVersion": 4, - } - ) - - def set_timeunit_code(self, timecode): - # Do timecode setting (somewhat edition-dependent). - self["indicatorOfUnitOfTimeRange"] = timecode - if self["edition"] == 1: - # for some odd reason, GRIB1 code uses *both* of these - # NOTE kludge -- the 2 keys are really the same thing - self["unitOfTime"] = timecode - - -@tests.skip_grib -class TestGribTimecodes(tests.IrisTest): - def _run_timetests(self, test_set): - # Check the unit-handling for given units-codes and editions. - - # Operates on lists of cases for various time-units and grib-editions. - # Format: (edition, code, expected-exception, - # equivalent-seconds, description-string) - with mock.patch("iris_grib.gribapi", _mock_gribapi): - for test_controls in test_set: - ( - grib_edition, - timeunit_codenum, - expected_error, - timeunit_secs, - timeunit_str, - ) = test_controls - - # Construct a suitable fake test message. - message = FakeGribMessage( - edition=grib_edition, time_code=timeunit_codenum - ) - - if expected_error: - # Expect GribWrapper construction to fail. - with self.assertRaises(type(expected_error)) as ar_context: - _ = iris_grib.GribWrapper(message) - self.assertEqual( - ar_context.exception.args, expected_error.args - ) - continue - - # 'ELSE'... - # Expect the wrapper construction to work. - # Make a GribWrapper object and test it. - wrapped_msg = iris_grib.GribWrapper(message) - - # Check the units string. - forecast_timeunit = wrapped_msg._forecastTimeUnit - self.assertEqual( - forecast_timeunit, - timeunit_str, - "Bad unit string for edition={ed:01d}, " - "unitcode={code:01d} : " - 'expected="{wanted}" GOT="{got}"'.format( - ed=grib_edition, - code=timeunit_codenum, - wanted=timeunit_str, - got=forecast_timeunit, - ), - ) - - # Check the data-starttime calculation. - interval_start_to_end = ( - wrapped_msg._phenomenonDateTime - - wrapped_msg._referenceDateTime - ) - if grib_edition == 1: - interval_from_units = wrapped_msg.P1 - else: - interval_from_units = wrapped_msg.forecastTime - interval_from_units *= datetime.timedelta(0, timeunit_secs) - self.assertEqual( - interval_start_to_end, - interval_from_units, - "Inconsistent start time offset for edition={ed:01d}, " - "unitcode={code:01d} : " - 'from-unit="{unit_str}" ' - 'from-phenom-minus-ref="{e2e_str}"'.format( - ed=grib_edition, - code=timeunit_codenum, - unit_str=interval_from_units, - e2e_str=interval_start_to_end, - ), - ) - - # Test groups of testcases for various time-units and grib-editions. - # Format: (edition, code, expected-exception, - # equivalent-seconds, description-string) - def test_timeunits_common(self): - tests = ( - (1, 0, None, 60.0, "minutes"), - (1, 1, None, _hour_secs, "hours"), - (1, 2, None, 24.0 * _hour_secs, "days"), - (1, 10, None, 3.0 * _hour_secs, "3 hours"), - (1, 11, None, 6.0 * _hour_secs, "6 hours"), - (1, 12, None, 12.0 * _hour_secs, "12 hours"), - ) - TestGribTimecodes._run_timetests(self, tests) - - @staticmethod - def _err_bad_timeunit(code): - return iris.exceptions.NotYetImplementedError( - "Unhandled time unit for forecast " - "indicatorOfUnitOfTimeRange : {code}".format(code=code) - ) - - def test_timeunits_grib1_specific(self): - tests = ( - (1, 13, None, 0.25 * _hour_secs, "15 minutes"), - (1, 14, None, 0.5 * _hour_secs, "30 minutes"), - (1, 254, None, 1.0, "seconds"), - (1, 111, TestGribTimecodes._err_bad_timeunit(111), 1.0, "??"), - ) - TestGribTimecodes._run_timetests(self, tests) - - def test_timeunits_calendar(self): - tests = ( - (1, 3, TestGribTimecodes._err_bad_timeunit(3), 0.0, "months"), - (1, 4, TestGribTimecodes._err_bad_timeunit(4), 0.0, "years"), - (1, 5, TestGribTimecodes._err_bad_timeunit(5), 0.0, "decades"), - (1, 6, TestGribTimecodes._err_bad_timeunit(6), 0.0, "30 years"), - (1, 7, TestGribTimecodes._err_bad_timeunit(7), 0.0, "centuries"), - ) - TestGribTimecodes._run_timetests(self, tests) - - def test_timeunits_invalid(self): - tests = ( - (1, 111, TestGribTimecodes._err_bad_timeunit(111), 1.0, "??"), - ) - TestGribTimecodes._run_timetests(self, tests) - - def test_warn_unknown_pdts(self): - # Test loading of an unrecognised GRIB Product Definition Template. - - # Get a temporary file by name (deleted afterward by context). - with self.temp_filename() as temp_gribfile_path: - # Write a test grib message to the temporary file. - with open(temp_gribfile_path, "wb") as temp_gribfile: - grib_message = gribapi.grib_new_from_samples("GRIB2") - # Set the PDT to something unexpected. - gribapi.grib_set_long( - grib_message, "productDefinitionTemplateNumber", 5 - ) - gribapi.grib_write(grib_message, temp_gribfile) - - # Load the message from the file as a cube. - cube_generator = iris_grib.load_cubes(temp_gribfile_path) - with self.assertRaises(iris.exceptions.TranslationError) as te: - _ = next(cube_generator) - self.assertEqual( - "Product definition template [5]" " is not supported", - str(te.exception), - ) - - -@tests.skip_grib -class TestGribSimple(tests.IrisTest): - # A testing class that does not need the test data. - def mock_grib(self): - # A mock grib message, with attributes that can't be Mocks themselves. - grib = mock.Mock() - grib.startStep = 0 - grib.phenomenon_points = lambda unit: 3 - grib._forecastTimeUnit = "hours" - grib.productDefinitionTemplateNumber = 0 - # define a level type (NB these 2 are effectively the same) - grib.levelType = 1 - grib.typeOfFirstFixedSurface = 1 - grib.typeOfSecondFixedSurface = 1 - return grib - - def cube_from_message(self, grib): - # Parameter translation now uses the GribWrapper, so we must convert - # the Mock-based fake message to a FakeGribMessage. - with mock.patch("iris_grib.gribapi", _mock_gribapi): - grib_message = FakeGribMessage(**grib.__dict__) - wrapped_msg = iris_grib.GribWrapper(grib_message) - cube, _, _ = iris.fileformats.rules._make_cube( - wrapped_msg, iris_grib._grib1_load_rules.grib1_convert - ) - return cube - - -@tests.skip_grib -class TestGrib1LoadPhenomenon(TestGribSimple): - # Test recognition of grib phenomenon types. - def mock_grib(self): - grib = super().mock_grib() - grib.edition = 1 - return grib - - def test_grib1_unknownparam(self): - grib = self.mock_grib() - grib.table2Version = 0 - grib.indicatorOfParameter = 9999 - cube = self.cube_from_message(grib) - self.assertEqual(cube.standard_name, None) - self.assertEqual(cube.long_name, None) - self.assertEqual(cube.units, cf_units.Unit("???")) - - def test_grib1_unknown_local_param(self): - grib = self.mock_grib() - grib.table2Version = 128 - grib.indicatorOfParameter = 999 - cube = self.cube_from_message(grib) - self.assertEqual(cube.standard_name, None) - self.assertEqual(cube.long_name, "UNKNOWN LOCAL PARAM 999.128") - self.assertEqual(cube.units, cf_units.Unit("???")) - - def test_grib1_unknown_standard_param(self): - grib = self.mock_grib() - grib.table2Version = 1 - grib.indicatorOfParameter = 975 - cube = self.cube_from_message(grib) - self.assertEqual(cube.standard_name, None) - self.assertEqual(cube.long_name, "UNKNOWN LOCAL PARAM 975.1") - self.assertEqual(cube.units, cf_units.Unit("???")) - - def known_grib1(self, param, standard_str, units_str): - grib = self.mock_grib() - grib.table2Version = 1 - grib.indicatorOfParameter = param - cube = self.cube_from_message(grib) - self.assertEqual(cube.standard_name, standard_str) - self.assertEqual(cube.long_name, None) - self.assertEqual(cube.units, cf_units.Unit(units_str)) - - def test_grib1_known_standard_params(self): - # at present, there are just a very few of these - self.known_grib1(11, "air_temperature", "kelvin") - self.known_grib1(33, "x_wind", "m s-1") - self.known_grib1(34, "y_wind", "m s-1") - - -if __name__ == "__main__": - tests.main() diff --git a/lib/iris/tests/test_grib_save.py b/lib/iris/tests/test_grib_save.py deleted file mode 100644 index 7799872a9c..0000000000 --- a/lib/iris/tests/test_grib_save.py +++ /dev/null @@ -1,318 +0,0 @@ -# Copyright Iris contributors -# -# This file is part of Iris and is released under the LGPL license. -# See COPYING and COPYING.LESSER in the root of the repository for full -# licensing details. - -# import iris tests first so that some things can be initialised before importing anything else -import iris.tests as tests - -import os -import datetime - -import cf_units -import numpy as np - -import iris -import iris.cube -import iris.coord_systems -import iris.coords - -if tests.GRIB_AVAILABLE: - import gribapi - from iris_grib._load_convert import _MDI as MDI - - -@tests.skip_data -@tests.skip_grib -class TestLoadSave(tests.TestGribMessage): - def setUp(self): - self.skip_keys = [] - - def test_latlon_forecast_plev(self): - source_grib = tests.get_data_path(("GRIB", "uk_t", "uk_t.grib2")) - cubes = iris.load(source_grib) - with self.temp_filename(suffix=".grib2") as temp_file_path: - iris.save(cubes, temp_file_path) - expect_diffs = { - "totalLength": (4837, 4832), - "productionStatusOfProcessedData": (0, 255), - "scaleFactorOfRadiusOfSphericalEarth": (MDI, 0), - "shapeOfTheEarth": (0, 1), - "scaledValueOfRadiusOfSphericalEarth": (MDI, 6367470), - "scaledValueOfEarthMajorAxis": (MDI, 0), - "scaleFactorOfEarthMajorAxis": (MDI, 0), - "scaledValueOfEarthMinorAxis": (MDI, 0), - "scaleFactorOfEarthMinorAxis": (MDI, 0), - "typeOfGeneratingProcess": (0, 255), - "generatingProcessIdentifier": (128, 255), - } - self.assertGribMessageDifference( - source_grib, - temp_file_path, - expect_diffs, - self.skip_keys, - skip_sections=[2], - ) - - def test_rotated_latlon(self): - source_grib = tests.get_data_path( - ("GRIB", "rotated_nae_t", "sensible_pole.grib2") - ) - cubes = iris.load(source_grib) - with self.temp_filename(suffix=".grib2") as temp_file_path: - iris.save(cubes, temp_file_path) - expect_diffs = { - "totalLength": (648196, 648191), - "productionStatusOfProcessedData": (0, 255), - "scaleFactorOfRadiusOfSphericalEarth": (MDI, 0), - "shapeOfTheEarth": (0, 1), - "scaledValueOfRadiusOfSphericalEarth": (MDI, 6367470), - "scaledValueOfEarthMajorAxis": (MDI, 0), - "scaleFactorOfEarthMajorAxis": (MDI, 0), - "scaledValueOfEarthMinorAxis": (MDI, 0), - "scaleFactorOfEarthMinorAxis": (MDI, 0), - "longitudeOfLastGridPoint": (392109982, 32106370), - "latitudeOfLastGridPoint": (19419996, 19419285), - "typeOfGeneratingProcess": (0, 255), - "generatingProcessIdentifier": (128, 255), - } - self.assertGribMessageDifference( - source_grib, - temp_file_path, - expect_diffs, - self.skip_keys, - skip_sections=[2], - ) - - def test_time_mean(self): - # This test for time-mean fields also tests negative forecast time. - source_grib = tests.get_data_path( - ("GRIB", "time_processed", "time_bound.grib2") - ) - cubes = iris.load(source_grib) - expect_diffs = { - "totalLength": (21232, 21227), - "productionStatusOfProcessedData": (0, 255), - "scaleFactorOfRadiusOfSphericalEarth": (MDI, 0), - "shapeOfTheEarth": (0, 1), - "scaledValueOfRadiusOfSphericalEarth": (MDI, 6367470), - "scaledValueOfEarthMajorAxis": (MDI, 0), - "scaleFactorOfEarthMajorAxis": (MDI, 0), - "scaledValueOfEarthMinorAxis": (MDI, 0), - "scaleFactorOfEarthMinorAxis": (MDI, 0), - "longitudeOfLastGridPoint": (356249908, 356249809), - "latitudeOfLastGridPoint": (-89999938, -89999944), - "typeOfGeneratingProcess": (0, 255), - "generatingProcessIdentifier": (128, 255), - "typeOfTimeIncrement": (2, 255), - } - self.skip_keys.append("stepType") - self.skip_keys.append("stepTypeInternal") - with self.temp_filename(suffix=".grib2") as temp_file_path: - iris.save(cubes, temp_file_path) - self.assertGribMessageDifference( - source_grib, - temp_file_path, - expect_diffs, - self.skip_keys, - skip_sections=[2], - ) - - -@tests.skip_data -@tests.skip_grib -class TestCubeSave(tests.IrisTest): - # save fabricated cubes - - def _load_basic(self): - path = tests.get_data_path(("GRIB", "uk_t", "uk_t.grib2")) - return iris.load(path)[0] - - def test_params(self): - # TODO - pass - - def test_originating_centre(self): - # TODO - pass - - def test_irregular(self): - cube = self._load_basic() - lat_coord = cube.coord("latitude") - cube.remove_coord("latitude") - - new_lats = np.append( - lat_coord.points[:-1], lat_coord.points[0] - ) # Irregular - cube.add_aux_coord( - iris.coords.AuxCoord( - new_lats, - "latitude", - units="degrees", - coord_system=lat_coord.coord_system, - ), - 0, - ) - - saved_grib = iris.util.create_temp_filename(suffix=".grib2") - self.assertRaises( - iris.exceptions.TranslationError, iris.save, cube, saved_grib - ) - os.remove(saved_grib) - - def test_non_latlon(self): - cube = self._load_basic() - cube.coord(dimensions=[0]).coord_system = None - saved_grib = iris.util.create_temp_filename(suffix=".grib2") - self.assertRaises( - iris.exceptions.TranslationError, iris.save, cube, saved_grib - ) - os.remove(saved_grib) - - def test_forecast_period(self): - # unhandled unit - cube = self._load_basic() - cube.coord("forecast_period").units = cf_units.Unit("years") - saved_grib = iris.util.create_temp_filename(suffix=".grib2") - self.assertRaises( - iris.exceptions.TranslationError, iris.save, cube, saved_grib - ) - os.remove(saved_grib) - - def test_unhandled_vertical(self): - # unhandled level type - cube = self._load_basic() - # Adjust the 'pressure' coord to make it into an "unrecognised Z coord" - p_coord = cube.coord("pressure") - p_coord.rename("not the messiah") - p_coord.units = "K" - p_coord.attributes["positive"] = "up" - saved_grib = iris.util.create_temp_filename(suffix=".grib2") - with self.assertRaises(iris.exceptions.TranslationError): - iris.save(cube, saved_grib) - os.remove(saved_grib) - - def test_scalar_int32_pressure(self): - # Make sure we can save a scalar int32 coordinate with unit conversion. - cube = self._load_basic() - cube.coord("pressure").points = np.array([200], dtype=np.int32) - cube.coord("pressure").units = "hPa" - with self.temp_filename(".grib2") as testfile: - iris.save(cube, testfile) - - def test_bounded_level(self): - cube = iris.load_cube( - tests.get_data_path(("GRIB", "uk_t", "uk_t.grib2")) - ) - # Changing pressure to altitude due to grib api bug: - # https://github.com/SciTools/iris/pull/715#discussion_r5901538 - cube.remove_coord("pressure") - cube.add_aux_coord( - iris.coords.AuxCoord( - 1030.0, - long_name="altitude", - units="m", - bounds=np.array([111.0, 1949.0]), - ) - ) - with self.temp_filename(".grib2") as testfile: - iris.save(cube, testfile) - with open(testfile, "rb") as saved_file: - g = gribapi.grib_new_from_file(saved_file) - self.assertEqual( - gribapi.grib_get_double( - g, "scaledValueOfFirstFixedSurface" - ), - 111.0, - ) - self.assertEqual( - gribapi.grib_get_double( - g, "scaledValueOfSecondFixedSurface" - ), - 1949.0, - ) - - -@tests.skip_grib -class TestHandmade(tests.IrisTest): - def _lat_lon_cube_no_time(self): - """Returns a cube with a latitude and longitude suitable for testing saving to PP/NetCDF etc.""" - cube = iris.cube.Cube(np.arange(12, dtype=np.int32).reshape((3, 4))) - cs = iris.coord_systems.GeogCS(6371229) - cube.add_dim_coord( - iris.coords.DimCoord( - np.arange(4) * 90 + -180, - "longitude", - units="degrees", - coord_system=cs, - ), - 1, - ) - cube.add_dim_coord( - iris.coords.DimCoord( - np.arange(3) * 45 + -90, - "latitude", - units="degrees", - coord_system=cs, - ), - 0, - ) - - return cube - - def _cube_time_no_forecast(self): - cube = self._lat_lon_cube_no_time() - unit = cf_units.Unit( - "hours since epoch", calendar=cf_units.CALENDAR_GREGORIAN - ) - dt = datetime.datetime(2010, 12, 31, 12, 0) - cube.add_aux_coord( - iris.coords.AuxCoord( - np.array([unit.date2num(dt)], dtype=np.float64), - "time", - units=unit, - ) - ) - return cube - - def _cube_with_forecast(self): - cube = self._cube_time_no_forecast() - cube.add_aux_coord( - iris.coords.AuxCoord( - np.array([6], dtype=np.int32), "forecast_period", units="hours" - ) - ) - return cube - - def _cube_with_pressure(self): - cube = self._cube_with_forecast() - cube.add_aux_coord( - iris.coords.DimCoord(np.int32(10), "air_pressure", units="Pa") - ) - return cube - - def _cube_with_time_bounds(self): - cube = self._cube_with_pressure() - cube.coord("time").bounds = np.array([[0, 100]]) - return cube - - def test_no_time_cube(self): - cube = self._lat_lon_cube_no_time() - saved_grib = iris.util.create_temp_filename(suffix=".grib2") - self.assertRaises( - iris.exceptions.TranslationError, iris.save, cube, saved_grib - ) - os.remove(saved_grib) - - def test_cube_with_time_bounds(self): - cube = self._cube_with_time_bounds() - saved_grib = iris.util.create_temp_filename(suffix=".grib2") - self.assertRaises( - iris.exceptions.TranslationError, iris.save, cube, saved_grib - ) - os.remove(saved_grib) - - -if __name__ == "__main__": - tests.main() diff --git a/lib/iris/tests/test_grib_save_rules.py b/lib/iris/tests/test_grib_save_rules.py deleted file mode 100644 index e9f8e6e392..0000000000 --- a/lib/iris/tests/test_grib_save_rules.py +++ /dev/null @@ -1,148 +0,0 @@ -# Copyright Iris contributors -# -# This file is part of Iris and is released under the LGPL license. -# See COPYING and COPYING.LESSER in the root of the repository for full -# licensing details. -"""Integration tests for :mod:`iris_grib._save_rules`.""" - -# import iris tests first so that some things can be initialised before importing anything else -import iris.tests as tests - -from unittest import mock -import warnings - -import cf_units -import numpy as np - -import iris.cube -import iris.coords - -if tests.GRIB_AVAILABLE: - import gribapi - import iris_grib._save_rules as grib_save_rules -else: - gribapi = None - - -@tests.skip_grib -class Test_set_fixed_surfaces(tests.IrisTest): - @mock.patch.object(gribapi, "grib_set") - def test_altitude_point(self, mock_set): - grib = None - cube = iris.cube.Cube([1, 2, 3, 4, 5]) - cube.add_aux_coord( - iris.coords.AuxCoord([12345], "altitude", units="m") - ) - - grib_save_rules.set_fixed_surfaces(cube, grib) - - mock_set.assert_any_call(grib, "typeOfFirstFixedSurface", 102) - mock_set.assert_any_call(grib, "scaleFactorOfFirstFixedSurface", 0) - mock_set.assert_any_call(grib, "scaledValueOfFirstFixedSurface", 12345) - mock_set.assert_any_call(grib, "typeOfSecondFixedSurface", -1) - mock_set.assert_any_call(grib, "scaleFactorOfSecondFixedSurface", 255) - mock_set.assert_any_call(grib, "scaledValueOfSecondFixedSurface", -1) - - @mock.patch.object(gribapi, "grib_set") - def test_height_point(self, mock_set): - grib = None - cube = iris.cube.Cube([1, 2, 3, 4, 5]) - cube.add_aux_coord(iris.coords.AuxCoord([12345], "height", units="m")) - - grib_save_rules.set_fixed_surfaces(cube, grib) - - mock_set.assert_any_call(grib, "typeOfFirstFixedSurface", 103) - mock_set.assert_any_call(grib, "scaleFactorOfFirstFixedSurface", 0) - mock_set.assert_any_call(grib, "scaledValueOfFirstFixedSurface", 12345) - mock_set.assert_any_call(grib, "typeOfSecondFixedSurface", -1) - mock_set.assert_any_call(grib, "scaleFactorOfSecondFixedSurface", 255) - mock_set.assert_any_call(grib, "scaledValueOfSecondFixedSurface", -1) - - @mock.patch.object(gribapi, "grib_set") - def test_no_vertical(self, mock_set): - grib = None - cube = iris.cube.Cube([1, 2, 3, 4, 5]) - grib_save_rules.set_fixed_surfaces(cube, grib) - mock_set.assert_any_call(grib, "typeOfFirstFixedSurface", 1) - mock_set.assert_any_call(grib, "scaleFactorOfFirstFixedSurface", 0) - mock_set.assert_any_call(grib, "scaledValueOfFirstFixedSurface", 0) - mock_set.assert_any_call(grib, "typeOfSecondFixedSurface", -1) - mock_set.assert_any_call(grib, "scaleFactorOfSecondFixedSurface", 255) - mock_set.assert_any_call(grib, "scaledValueOfSecondFixedSurface", -1) - - -@tests.skip_grib -class Test_phenomenon(tests.IrisTest): - @mock.patch.object(gribapi, "grib_set") - def test_phenom_unknown(self, mock_set): - grib = None - cube = iris.cube.Cube(np.array([1.0])) - # Force reset of warnings registry to avoid suppression of - # repeated warnings. warnings.resetwarnings() does not do this. - if hasattr(grib_save_rules, "__warningregistry__"): - grib_save_rules.__warningregistry__.clear() - with warnings.catch_warnings(): - # This should issue a warning about unrecognised data - warnings.simplefilter("error") - with self.assertRaises(UserWarning): - grib_save_rules.set_discipline_and_parameter(cube, grib) - # do it all again, and this time check the results - grib = None - cube = iris.cube.Cube(np.array([1.0])) - grib_save_rules.set_discipline_and_parameter(cube, grib) - mock_set.assert_any_call(grib, "discipline", 255) - mock_set.assert_any_call(grib, "parameterCategory", 255) - mock_set.assert_any_call(grib, "parameterNumber", 255) - - @mock.patch.object(gribapi, "grib_set") - def test_phenom_known_standard_name(self, mock_set): - grib = None - cube = iris.cube.Cube( - np.array([1.0]), standard_name="sea_surface_temperature" - ) - grib_save_rules.set_discipline_and_parameter(cube, grib) - mock_set.assert_any_call(grib, "discipline", 10) - mock_set.assert_any_call(grib, "parameterCategory", 3) - mock_set.assert_any_call(grib, "parameterNumber", 0) - - @mock.patch.object(gribapi, "grib_set") - def test_phenom_known_long_name(self, mock_set): - grib = None - cube = iris.cube.Cube(np.array([1.0]), long_name="cloud_mixing_ratio") - grib_save_rules.set_discipline_and_parameter(cube, grib) - mock_set.assert_any_call(grib, "discipline", 0) - mock_set.assert_any_call(grib, "parameterCategory", 1) - mock_set.assert_any_call(grib, "parameterNumber", 22) - - -@tests.skip_grib -class Test_type_of_statistical_processing(tests.IrisTest): - @mock.patch.object(gribapi, "grib_set") - def test_stats_type_min(self, mock_set): - grib = None - cube = iris.cube.Cube(np.array([1.0])) - time_unit = cf_units.Unit("hours since 1970-01-01 00:00:00") - time_coord = iris.coords.DimCoord( - [0.0], bounds=[0.0, 1], standard_name="time", units=time_unit - ) - cube.add_aux_coord(time_coord, ()) - cube.add_cell_method(iris.coords.CellMethod("maximum", time_coord)) - grib_save_rules.product_definition_template_8(cube, grib) - mock_set.assert_any_call(grib, "typeOfStatisticalProcessing", 2) - - @mock.patch.object(gribapi, "grib_set") - def test_stats_type_max(self, mock_set): - grib = None - cube = iris.cube.Cube(np.array([1.0])) - time_unit = cf_units.Unit("hours since 1970-01-01 00:00:00") - time_coord = iris.coords.DimCoord( - [0.0], bounds=[0.0, 1], standard_name="time", units=time_unit - ) - cube.add_aux_coord(time_coord, ()) - cube.add_cell_method(iris.coords.CellMethod("minimum", time_coord)) - grib_save_rules.product_definition_template_8(cube, grib) - mock_set.assert_any_call(grib, "typeOfStatisticalProcessing", 3) - - -if __name__ == "__main__": - tests.main() diff --git a/lib/iris/tests/test_uri_callback.py b/lib/iris/tests/test_uri_callback.py index 16bccb7a09..b428a1b561 100644 --- a/lib/iris/tests/test_uri_callback.py +++ b/lib/iris/tests/test_uri_callback.py @@ -9,37 +9,11 @@ import os -import iris.coords +import iris @tests.skip_data class TestCallbacks(tests.IrisTest): - @tests.skip_grib - def test_grib_callback(self): - def grib_thing_getter(cube, field, filename): - if hasattr(field, "sections"): - # New-style loader callback : 'field' is a GribMessage, which has 'sections'. - cube.add_aux_coord( - iris.coords.AuxCoord( - field.sections[1]["year"], - long_name="extra_year_number_coord", - units="no_unit", - ) - ) - else: - # Old-style loader provides 'GribWrapper' type field. - cube.add_aux_coord( - iris.coords.AuxCoord( - field.extra_keys["_periodStartDateTime"], - long_name="random element", - units="no_unit", - ) - ) - - fname = tests.get_data_path(("GRIB", "global_t", "global.grib2")) - cube = iris.load_cube(fname, callback=grib_thing_getter) - self.assertCML(cube, ["uri_callback", "grib_global.cml"]) - def test_pp_callback(self): def pp_callback(cube, field, filename): cube.attributes["filename"] = os.path.basename(filename) diff --git a/lib/iris/tests/unit/analysis/__init__.py b/lib/iris/tests/unit/analysis/__init__.py index 3f6179fbf2..974b4e3584 100644 --- a/lib/iris/tests/unit/analysis/__init__.py +++ b/lib/iris/tests/unit/analysis/__init__.py @@ -4,42 +4,3 @@ # See COPYING and COPYING.LESSER in the root of the repository for full # licensing details. """Unit tests for the :mod:`iris.analysis` package.""" - -# Import iris.tests first so that some things can be initialised before -# importing anything else. -import iris.tests as tests - -from unittest import mock - -from iris.analysis import Linear - - -class Test_Linear(tests.IrisTest): - def setUp(self): - self.extrap = "some extrapolation" - - def test___init__(self): - linear = Linear(extrapolation_mode=self.extrap) - self.assertEqual( - getattr(linear, "extrapolation_mode", None), self.extrap - ) - - @mock.patch("iris.analysis.LinearInterpolator", name="LinearInterpolator") - def test_interpolator(self, linear_interp_patch): - mock_interpolator = mock.Mock(name="mocked linear interpolator") - linear_interp_patch.return_value = mock_interpolator - - linear = Linear(self.extrap) - cube = mock.Mock(name="cube") - coords = mock.Mock(name="coords") - - interpolator = linear.interpolator(cube, coords) - - self.assertIs(interpolator, mock_interpolator) - linear_interp_patch.assert_called_once_with( - cube, coords, extrapolation_mode=self.extrap - ) - - -if __name__ == "__main__": - tests.main() diff --git a/lib/iris/tests/unit/analysis/cartography/test_project.py b/lib/iris/tests/unit/analysis/cartography/test_project.py index d617e33898..33edfb5675 100644 --- a/lib/iris/tests/unit/analysis/cartography/test_project.py +++ b/lib/iris/tests/unit/analysis/cartography/test_project.py @@ -81,7 +81,7 @@ def test_bad_resolution_negative(self): @tests.skip_data def test_bad_resolution_non_numeric(self): cube = low_res_4d() - with self.assertRaises(ValueError): + with self.assertRaises(TypeError): project(cube, ROBINSON, nx=200, ny="abc") @tests.skip_data diff --git a/lib/iris/tests/unit/cube/test_CubeList.py b/lib/iris/tests/unit/cube/test_CubeList.py index 2e0fb2d12d..6870e2367f 100644 --- a/lib/iris/tests/unit/cube/test_CubeList.py +++ b/lib/iris/tests/unit/cube/test_CubeList.py @@ -7,6 +7,8 @@ # Import iris.tests first so that some things can be initialised before # importing anything else. +import collections + import iris.tests as tests import iris.tests.stock @@ -292,6 +294,22 @@ def test_scalar_cube_data_constraint(self): self.assertEqual(res, expected) +class Test_iteration(tests.IrisTest): + def setUp(self): + self.scalar_cubes = CubeList() + for i in range(5): + for letter in "abcd": + self.scalar_cubes.append(Cube(i, long_name=letter)) + + def test_iterable(self): + self.assertTrue(isinstance(self.scalar_cubes, collections.Iterable)) + + def test_iteration(self): + letters = "abcd" * 5 + for i, cube in enumerate(self.scalar_cubes): + self.assertEqual(cube.long_name, letters[i]) + + class TestPrint(tests.IrisTest): def setUp(self): self.cubes = CubeList([iris.tests.stock.lat_lon_cube()]) diff --git a/lib/iris/tests/unit/fileformats/pp/test_PPDataProxy.py b/lib/iris/tests/unit/fileformats/pp/test_PPDataProxy.py index 53fcc08b95..8a22da061c 100644 --- a/lib/iris/tests/unit/fileformats/pp/test_PPDataProxy.py +++ b/lib/iris/tests/unit/fileformats/pp/test_PPDataProxy.py @@ -10,6 +10,7 @@ import iris.tests as tests from unittest import mock +import numpy as np from iris.fileformats.pp import PPDataProxy, SplittableInt @@ -21,7 +22,7 @@ def test_lbpack_SplittableInt(self): self.assertEqual(proxy.lbpack, lbpack) self.assertIs(proxy.lbpack, lbpack) - def test_lnpack_raw(self): + def test_lbpack_raw(self): lbpack = 4321 proxy = PPDataProxy(None, None, None, None, None, lbpack, None, None) self.assertEqual(proxy.lbpack, lbpack) @@ -33,5 +34,128 @@ def test_lnpack_raw(self): self.assertEqual(proxy.lbpack.n4, lbpack // 1000 % 10) +class SliceTranslator: + """ + Class to translate an array-indexing expression into a tuple of keys. + + An instance just returns the argument of its __getitem__ call. + + """ + + def __getitem__(self, keys): + return keys + + +# A multidimensional-indexable object that returns its index keys, so we can +# use multidimensional-indexing notation to specify a slicing expression. +Slices = SliceTranslator() + + +class Test__getitem__slicing(tests.IrisTest): + def _check_slicing( + self, test_shape, indices, result_shape, data_was_fetched=True + ): + # Check behaviour of the getitem call with specific slicings. + # Especially: check cases where a fetch does *not* read from the file. + # This is necessary because, since Dask 2.0, the "from_array" function + # takes a zero-length slice of its array argument, to capture array + # metadata, and in those cases we want to avoid file access. + test_dtype = np.dtype(np.float32) + proxy = PPDataProxy( + shape=test_shape, + src_dtype=test_dtype, + path=None, + offset=None, + data_len=None, + lbpack=0, # Note: a 'real' value is needed. + boundary_packing=None, + mdi=None, + ) + + # Mock out the file-open call, to see if the file would be read. + builtin_open_func_name = "builtins.open" + mock_fileopen = self.patch(builtin_open_func_name) + + # Also mock out the 'databytes_to_shaped_array' call, to fake minimal + # operation in the cases where file-open *does* get called. + fake_data = np.zeros(test_shape, dtype=test_dtype) + self.patch( + "iris.fileformats.pp._data_bytes_to_shaped_array", + mock.MagicMock(return_value=fake_data), + ) + + # Test the requested indexing operation. + result = proxy.__getitem__(indices) + + # Check the behaviour and results were as expected. + self.assertEqual(mock_fileopen.called, data_was_fetched) + self.assertIsInstance(result, np.ndarray) + self.assertEqual(result.dtype, test_dtype) + self.assertEqual(result.shape, result_shape) + + def test_slicing_1d_normal(self): + # A 'normal' 1d testcase with no empty slices. + self._check_slicing( + test_shape=(3,), + indices=Slices[1:10], + result_shape=(2,), + data_was_fetched=True, + ) + + def test_slicing_1d_empty(self): + # A 1d testcase with an empty slicing. + self._check_slicing( + test_shape=(3,), + indices=Slices[0:0], + result_shape=(0,), + data_was_fetched=False, + ) + + def test_slicing_2d_normal(self): + # A 2d testcase with no empty slices. + self._check_slicing( + test_shape=(3, 4), + indices=Slices[2, :3], + result_shape=(3,), + data_was_fetched=True, + ) + + def test_slicing_2d_allempty(self): + # A 2d testcase with all empty slices. + self._check_slicing( + test_shape=(3, 4), + indices=Slices[0:0, 0:0], + result_shape=(0, 0), + data_was_fetched=False, + ) + + def test_slicing_2d_empty_dim0(self): + # A 2d testcase with an empty slice. + self._check_slicing( + test_shape=(3, 4), + indices=Slices[0:0], + result_shape=(0, 4), + data_was_fetched=False, + ) + + def test_slicing_2d_empty_dim1(self): + # A 2d testcase with an empty slice, and an integer index. + self._check_slicing( + test_shape=(3, 4), + indices=Slices[1, 0:0], + result_shape=(0,), + data_was_fetched=False, + ) + + def test_slicing_complex(self): + # Multiple dimensions with multiple empty slices. + self._check_slicing( + test_shape=(3, 4, 2, 5, 6, 3, 7), + indices=Slices[1:3, 2, 0:0, :, 1:1, :100], + result_shape=(2, 0, 5, 0, 3, 7), + data_was_fetched=False, + ) + + if __name__ == "__main__": tests.main() diff --git a/lib/iris/util.py b/lib/iris/util.py index 3bda110a07..3212eba4a5 100644 --- a/lib/iris/util.py +++ b/lib/iris/util.py @@ -959,6 +959,67 @@ def __lt__(self, other): return NotImplemented +def _array_slice_ifempty(keys, shape, dtype): + """ + Detect cases where an array slice will contain no data, as it contains a + zero-length dimension, and produce an equivalent result for those cases. + + The function indicates 'empty' slicing cases, by returning an array equal + to the slice result in those cases. + + Args: + + * keys (indexing key, or tuple of keys): + The argument from an array __getitem__ call. + Only tuples of integers and slices are supported, in particular no + newaxis, ellipsis or array keys. + These are the types of array access usage we expect from Dask. + * shape (tuple of int): + The shape of the array being indexed. + * dtype (numpy.dtype): + The dtype of the array being indexed. + + Returns: + result (np.ndarray or None): + If 'keys' contains a slice(0, 0), this is an ndarray of the correct + resulting shape and provided dtype. + Otherwise it is None. + + .. note:: + + This is used to prevent DataProxy arraylike objects from fetching their + file data when wrapped as Dask arrays. + This is because, for Dask >= 2.0, the "dask.array.from_array" call + performs a fetch like [0:0, 0:0, ...], to 'snapshot' array metadata. + This function enables us to avoid triggering a file data fetch in those + cases : This is consistent because the result will not contain any + actual data content. + + """ + # Convert a single key into a 1-tuple, so we always have a tuple of keys. + if isinstance(keys, tuple): + keys_tuple = keys + else: + keys_tuple = (keys,) + + if any(key == slice(0, 0) for key in keys_tuple): + # An 'empty' slice is present : Return a 'fake' array instead. + target_shape = list(shape) + for i_dim, key in enumerate(keys_tuple): + if key == slice(0, 0): + # Reduce dims with empty slicing to length 0. + target_shape[i_dim] = 0 + # Create a prototype result : no memory usage, as some dims are 0. + result = np.zeros(target_shape, dtype=dtype) + # Index with original keys to produce the desired result shape. + # Note : also ok in 0-length dims, as the slice is always '0:0'. + result = result[keys] + else: + result = None + + return result + + def create_temp_filename(suffix=""): """Return a temporary file name.