Problems with pyke when installing from source

[MHBalsmeier]

A few months ago I contributed this to the documentation.

However, this does not fully work anymore (import iris leads to an error because pyke is missing).

I think this happens because pyke was removed from the list of dependencies (#4198).

So is this a temporary problem? Or should I add scitools-pyke to the python3 packages which need to be installed manually before iris? This would make no sense I think because then it'd basically be a dependency.

[bjlittle]

Hey @MHBalsmeier,

Thanks for raising this issue and touching base to let us know.

Okay, so on the main branch of iris we've dropped the requirement for pyke (on PyPI it's called scitools-pyke, and conda is pyke)

If you're building iris from the main branch from source then you should not need to install scitools-pyke.

It might be that you've checked out the latest branch but there are still old pyke source files in the directory tree. If this is the case, then please try a complete fresh checkout.

I'm just guessing here, as you not provided any details of the failure.

Please provide further details of the failure, of that's not the case.

[MHBalsmeier]

This is easy to reproduce. I set up a totally fresh Ubuntu 20.04 VM and did the following:

sudo apt-get update
sudo apt-get install python3-pip python3-tk libudunits2-dev libproj-dev proj-bin libgeos-dev libcunit1-dev
sudo apt-get update
pip3 install numpy
pip3 install setuptools cftime==1.2.1 cf-units scitools-iris

Opening python and typing in import iris leads to

Python 3.8.10 (default, Jun  2 2021, 10:49:15) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import iris
/home/ubuntu/.local/lib/python3.8/site-packages/iris/config.py:139: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/home/h05/itwl/projects/git/iris-test-data/test_data' is not a valid directory path.
  warnings.warn(msg.format(section, option, c_path))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.8/site-packages/iris/__init__.py", line 98, in <module>
    import iris.fileformats
  File "/home/ubuntu/.local/lib/python3.8/site-packages/iris/fileformats/__init__.py", line 22, in <module>
    from . import netcdf
  File "/home/ubuntu/.local/lib/python3.8/site-packages/iris/fileformats/netcdf.py", line 28, in <module>
    from pyke import knowledge_engine
ModuleNotFoundError: No module named 'pyke'
>>>

Installing scitools-pyke with pip3 install scitools-pyke solves the problem.

[MHBalsmeier]

I also think that the cftime==1.2.1 is outdated since pip warns me that iris now wants >=1.5. I can open a pull request for this if this is indeed the case.

[rcomer]

Hi @MHBalsmeier, you are right that cftime v1.5 is required for Iris v3.0.4. Iris versions 3.0.1, 3.0.2 and 3.0.3 all require cftime v1.2.1 or older.

I'm afraid your instructions in the documentation are in a sort of limbo at the moment: since they are written in the main branch, they do not actually correspond to any released version of Iris yet. They should work for the next minor version (Iris 3.1) when it is released though. Versions 3.0.x have been developed separately on their own branch (https://github.com/SciTools/iris/tree/v3.0.x) so, although they have been released more recently, they do not come with those instructions. You can verify this by comparing the "stable" documentation, which corresponds to the most recent release (currently 3.0.4) with the "latest" documentation, which corresponds to what we are currently developing in the main branch in GitHub.
https://scitools-iris.readthedocs.io/en/stable/installing.html
https://scitools-iris.readthedocs.io/en/latest/installing.html

The v3.0.x branch should soon be merged back into main, at which point main (and therefore v3.1 when it is released) will need cftime>=1.5, and will not need pyke.

Sorry it isn't more straightforward!

[MHBalsmeier]

Okay, thanks, so I will adapt the installation instructions by dropping the ==1.2.1, but leaving scitools-pyke away,

[MHBalsmeier]

Okay, I opened a pull request. I got two more things though:

1.) Why can't the python packages setuptools cftime cf-units be included in the requirements.txt of iris, so that they do not need to be installed manually?

2.) I always get these warnings:

/home/max/.local/lib/python3.8/site-packages/iris/config.py:139: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/home/h05/itwl/projects/git/iris-test-data/test_data' is not a valid directory path.
  warnings.warn(msg.format(section, option, c_path))

when importing iris. I mentioned this before (#3937) and can solve it now manually by changing/deleting some files, but it is still confusing that a correctly-installed package throws warnings. If you tell me how, I can try to get rid of this and then open a pull request.

[MHBalsmeier]

Okay, I opened a pull request. I got two more things though:

1.) Why can't the python packages setuptools cftime cf-units be included in the requirements.txt of iris, so that they do not need to be installed manually?

2.) I always get these warnings:

/home/max/.local/lib/python3.8/site-packages/iris/config.py:139: UserWarning: Ignoring config item 'Resources':'test_data_dir' (section:option) as '/home/h05/itwl/projects/git/iris-test-data/test_data' is not a valid directory path.
  warnings.warn(msg.format(section, option, c_path))

when importing iris. I mentioned this before (#3937) and can solve it now manually by changing/deleting some files, but it is still confusing that a correctly-installed package throws warnings. If you tell me how, I can try to get rid of this and then open a pull request.

I've checked the first question, it works solely with pip3 install scitools-iris. I will include this in the pull request.

[MHBalsmeier]

The other problem does not occur in 3.1.dev0 anymore, so I am closing this.