7.2.0: pytest warnings and call stack

[kloczek]

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-client-7.2.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-client-7.2.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-7.1.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0, configfile: pyproject.toml, testpaths: jupyter_client/tests/
plugins: asyncio-0.18.2, timeout-2.1.0, forked-1.4.0, xdist-2.5.0
asyncio: mode=legacy
timeout: 300.0s
timeout method: signal
timeout func_only: False
collected 181 items

jupyter_client/tests/test_adapter.py ...........................                                                                                                     [ 14%]
jupyter_client/tests/test_client.py ........                                                                                                                         [ 19%]
jupyter_client/tests/test_connect.py ...........                                                                                                                     [ 25%]
jupyter_client/tests/test_jsonutil.py .....                                                                                                                          [ 28%]
jupyter_client/tests/test_kernelapp.py .                                                                                                                             [ 28%]
jupyter_client/tests/test_kernelmanager.py .....................s.s.s............                                                                                    [ 49%]
jupyter_client/tests/test_kernelspec.py .............                                                                                                                [ 56%]
jupyter_client/tests/test_localinterfaces.py .                                                                                                                       [ 57%]
jupyter_client/tests/test_manager.py .                                                                                                                               [ 58%]
jupyter_client/tests/test_multikernelmanager.py ..............................                                                                                       [ 74%]
jupyter_client/tests/test_provisioning.py ...........                                                                                                                [ 80%]
jupyter_client/tests/test_public_api.py ....                                                                                                                         [ 82%]
jupyter_client/tests/test_restarter.py ........                                                                                                                      [ 87%]
jupyter_client/tests/test_session.py .........s...........                                                                                                           [ 98%]
jupyter_client/tests/test_ssh.py .                                                                                                                                   [ 99%]
jupyter_client/tests/test_utils.py .                                                                                                                                 [100%]

============================================================================= warnings summary =============================================================================
../../../../../usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191
  /usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191: DeprecationWarning: The 'asyncio_mode' default value will change to 'strict' in future, please explicitly use 'asyncio_mode=strict' or 'asyncio_mode=auto' in pytest configuration file.
    config.issue_config_time_warning(LEGACY_MODE, stacklevel=2)

../../../../../usr/lib/python3.8/site-packages/ipykernel/iostream.py:9
  /usr/lib/python3.8/site-packages/ipykernel/iostream.py:9: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    from imp import lock_held as import_lock_held

../../../../../usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:226
  /usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:226: DeprecationWarning: '@pytest.fixture' is applied to <fixture start_async_kernel, file=/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/tests/test_kernelmanager.py, line=126> in 'legacy' mode, please replace it with '@pytest_asyncio.fixture' as a preparation for switching to 'strict' mode (or use 'auto' mode to seamlessly handle all these fixtures as asyncio-driven).
    warnings.warn(

jupyter_client/tests/test_session.py::TestSession::test_tracking
  /usr/lib64/python3.8/site-packages/zmq/eventloop/zmqstream.py:101: DeprecationWarning: zmq.eventloop.ioloop is deprecated in pyzmq 17. pyzmq now works with default tornado and asyncio eventloops.
    self.io_loop = io_loop or IOLoop.current()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================================================================== slowest 10 durations ===========================================================================
18.98s call     jupyter_client/tests/test_kernelmanager.py::TestParallel::test_start_sequence_kernels[tcp]
18.52s call     jupyter_client/tests/test_kernelmanager.py::TestParallel::test_start_sequence_kernels[ipc]
18.14s call     jupyter_client/tests/test_restarter.py::test_async_restarter_gives_up[tcp]
18.14s call     jupyter_client/tests/test_restarter.py::test_async_restarter_gives_up[ipc]
12.72s call     jupyter_client/tests/test_kernelmanager.py::TestParallel::test_start_parallel_process_kernels[tcp]
12.71s call     jupyter_client/tests/test_kernelmanager.py::TestParallel::test_start_parallel_thread_kernels[tcp]
12.67s call     jupyter_client/tests/test_kernelmanager.py::TestParallel::test_start_sequence_process_kernels[tcp]
9.29s call     jupyter_client/tests/test_restarter.py::test_async_restart_check[tcp]
9.24s call     jupyter_client/tests/test_restarter.py::test_async_restart_check[ipc]
9.16s call     jupyter_client/tests/test_restarter.py::test_restart_check[ipc]
========================================================================= short test summary info ==========================================================================
SKIPPED [1] jupyter_client/tests/test_kernelmanager.py:367: IPC transport is currently not working for this test!
SKIPPED [1] jupyter_client/tests/test_kernelmanager.py:383: IPC transport is currently not working for this test!
SKIPPED [1] jupyter_client/tests/test_kernelmanager.py:399: IPC transport is currently not working for this test!
SKIPPED [1] jupyter_client/tests/test_session.py:307: could not import 'msgpack': No module named 'msgpack'
========================================================== 177 passed, 4 skipped, 4 warnings in 247.37s (0:04:07) ==========================================================
Task exception was never retrieved
future: <Task finished name='Task-1675' coro=<MultiKernelManager._add_kernel_when_ready() done, defined at /home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/multikernelmanager.py:166> exception=FileNotFoundError(2, 'No such file or directory')>
Traceback (most recent call last):
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/tests/test_multikernelmanager.py", line 590, in test_bad_kernelspec_pending
    await km.get_kernel(kernel_id).ready
  File "/usr/lib64/python3.8/asyncio/tasks.py", line 280, in __step
    result = coro.send(None)
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/multikernelmanager.py", line 170, in _add_kernel_when_ready
    await kernel_awaitable
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/utils.py", line 34, in ensure_async
    return await obj
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/manager.py", line 79, in wrapper
    raise e
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/manager.py", line 71, in wrapper
    out = await method(self, *args, **kwargs)
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/manager.py", line 380, in _async_start_kernel
    await ensure_async(self._launch_kernel(kernel_cmd, **kw))
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/utils.py", line 34, in ensure_async
    return await obj
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/manager.py", line 300, in _async_launch_kernel
    connection_info = await self.provisioner.launch_kernel(kernel_cmd, **kw)
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/provisioning/local_provisioner.py", line 199, in launch_kernel
    self.process = launch_kernel(cmd, **scrubbed_kwargs)
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/launcher.py", line 169, in launch_kernel
    raise ex
  File "/home/tkloczko/rpmbuild/BUILD/jupyter_client-7.2.0/jupyter_client/launcher.py", line 157, in launch_kernel
    proc = Popen(cmd, **kwargs)
  File "/usr/lib64/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib64/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'non_existent_executable'

Despite that printed call stack pytest finishes with exit 0.
List of modules isntalled in build env:

Package                       Version
----------------------------- -----------------
alabaster                     0.7.12
asgiref                       3.4.1
async-generator               1.10
attrs                         21.4.0
Babel                         2.9.1
backcall                      0.2.0
black                         22.3.0
build                         0.7.0
charset-normalizer            2.0.12
click                         8.0.4
coverage                      6.3.2
debugpy                       1.6.0
decorator                     5.1.1
distro                        1.6.0
Django                        3.2.9
docutils                      0.17.1
dpcontracts                   0.6.0
entrypoints                   0.3
execnet                       1.9.0
extras                        1.0.0
fakeredis                     1.6.1
fixtures                      3.0.0
gpg                           1.17.1-unknown
idna                          3.3
imagesize                     1.3.0
importlib-metadata            4.11.3
iniconfig                     1.1.1
ipykernel                     6.10.0
ipython                       7.30.1
jedi                          0.18.1
Jinja2                        3.0.3
jupyter-client                7.1.2
jupyter-core                  4.9.2
lark                          1.0.0
libcomps                      0.1.18
libcst                        0.3.20
markdown-it-py                2.0.1
MarkupSafe                    2.0.1
matplotlib-inline             0.1.3
mdit-py-plugins               0.3.0
mdurl                         0.1.0
mock                          4.0.3
multipledispatch              0.6.0
mypy-extensions               0.4.3
myst-parser                   0.17.0
nest-asyncio                  1.5.4
numpy                         1.22.3
packaging                     21.3
pandas                        1.4.1
parso                         0.8.3
pathspec                      0.9.0
pbr                           5.8.0
pep517                        0.12.0
pexpect                       4.8.0
pickleshare                   0.7.5
pip                           22.0.4
platformdirs                  2.5.1
pluggy                        1.0.0
prompt-toolkit                3.0.24
psutil                        5.9.0
ptyprocess                    0.7.0
py                            1.11.0
Pygments                      2.11.2
PyGObject                     3.42.0
pyparsing                     3.0.7
pytest                        7.1.1
pytest-asyncio                0.18.2
pytest-forked                 1.4.0
pytest-timeout                2.1.0
pytest-xdist                  2.5.0
python-dateutil               2.8.2
pytz                          2022.1
PyYAML                        6.0
pyzmq                         22.3.0
redis                         3.5.3
requests                      2.27.1
rpm                           4.17.0
setuptools                    60.9.3
six                           1.16.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
Sphinx                        4.5.0.dev20220327
sphinx-hoverxref              1.0.0
sphinx-rtd-theme              1.0.0
sphinx-selective-exclude      1.0.3
sphinxcontrib-applehelp       1.0.2.dev20220108
sphinxcontrib-devhelp         1.0.2.dev20220108
sphinxcontrib_github_alt      1.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1.dev20220108
sphinxcontrib-qthelp          1.0.3.dev20220108
sphinxcontrib-serializinghtml 1.1.5
sqlparse                      0.4.2
strategies                    0.2.3
testtools                     2.5.0
tomli                         2.0.1
toolz                         0.11.2
tornado                       6.1
traitlets                     5.1.1
typing_extensions             4.0.1
typing-inspect                0.7.1
urllib3                       1.26.8
wcwidth                       0.2.5
wheel                         0.37.1
zipp                          3.7.0

[kloczek]

@zeromq could you please have look on above? (because at lease some of those warnings are related to pyzmq module)

[davidbrochart]

Thanks @kloczek for opening an issue.
It's strange because the FileNotFound exception should be captured by pytest. We don't have this traceback in our CI.
Sorry I don't have a solution right now. I don't think Mr ZeroMQ will answer here either 😄

[blink1073]

#760 should help with the above.