execnet.gateway_base.DumpError: can't serialize <enum 'ExitCode'> #464

tiffinysmile · 2019-08-23T08:35:15Z

platform win32 -- Python 3.6.4, pytest-5.1.1, py-1.5.3, pluggy-0.12.0
plugins: allure-pytest-2.7.1, forked-1.0.2, html-1.22.0, metadata-1.8.0, xdist-1
.29.0
[gw2] node down: Traceback (most recent call last):
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1379, in _s
ave
    dispatch = self._dispatch[tp]
KeyError: <enum 'ExitCode'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1068, in ex
ecutetask
    do_exec(co, loc)  # noqa
  File "c:\python36\lib\site-packages\xdist\remote.py", line 274, in <module>
    config.hook.pytest_cmdline_main(config=config)
  File "c:\python36\lib\site-packages\pluggy\hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "c:\python36\lib\site-packages\pluggy\manager.py", line 87, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\python36\lib\site-packages\pluggy\manager.py", line 81, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "c:\python36\lib\site-packages\pluggy\callers.py", line 208, in _multical
l
    return outcome.get_result()
  File "c:\python36\lib\site-packages\pluggy\callers.py", line 80, in get_result

    raise ex[1].with_traceback(ex[2])
  File "c:\python36\lib\site-packages\pluggy\callers.py", line 187, in _multical
l
    res = hook_impl.function(*args)
  File "c:\python36\lib\site-packages\_pytest\main.py", line 228, in pytest_cmdl
ine_main
    return wrap_session(config, _main)
  File "c:\python36\lib\site-packages\_pytest\main.py", line 221, in wrap_sessio
n
    session=session, exitstatus=session.exitstatus
  File "c:\python36\lib\site-packages\pluggy\hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "c:\python36\lib\site-packages\pluggy\manager.py", line 87, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\python36\lib\site-packages\pluggy\manager.py", line 81, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "c:\python36\lib\site-packages\pluggy\callers.py", line 203, in _multical
l
    gen.send(outcome)
  File "c:\python36\lib\site-packages\xdist\remote.py", line 45, in pytest_sessi
onfinish
    self.sendevent("workerfinished", workeroutput=self.config.workeroutput)
  File "c:\python36\lib\site-packages\xdist\remote.py", line 30, in sendevent
    self.channel.send((name, kwargs))
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 710, in sen
d
    self.gateway._send(Message.CHANNEL_DATA, self.id, dumps_internal(item))
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1350, in du
mps_internal
    return _Serializer().save(obj)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1368, in sa
ve
    self._save(obj)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1386, in _s
ave
    dispatch(self, obj)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1471, in sa
ve_tuple
    self._save(item)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1386, in _s
ave
    dispatch(self, obj)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1467, in sa
ve_dict
    self._write_setitem(key, value)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1461, in _w
rite_setitem
    self._save(value)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1386, in _s
ave
    dispatch(self, obj)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1467, in sa
ve_dict
    self._write_setitem(key, value)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1461, in _w
rite_setitem
    self._save(value)
  File "c:\python36\lib\site-packages\execnet\gateway_base.py", line 1384, in _s
ave
    raise DumpError("can't serialize {}".format(tp))
execnet.gateway_base.DumpError: can't serialize <enum 'ExitCode'>

Replacing crashed worker gw2

The text was updated successfully, but these errors were encountered:

nicoddemus · 2019-08-23T11:52:00Z

Hi @tiffinysmile,

Strange, version 1.29.0 was released explicitly to support the ExitCode enum.

Could you provide a minimal reproducible example? I would be glad to try it out.

Some suggestions in the future to improve bug reports:

Please include more information in the future than just a traceback dump
Formatting the post appropriately makes it easier for us to see what's going on (I've formatted yours).
Usually it is nice to provide some information about the environment you are using (although much can be gather from the traceback itself in this case).

tiffinysmile · 2019-08-26T01:52:50Z

it only happens when many UI test cases run

nicoddemus · 2019-08-26T11:46:16Z

Strange, I don't see how they could affect this...

Any chance of providing a reproducible example?

melund · 2019-09-06T06:51:54Z

I have run into this error a few times recently. It seems elusive. I always works when I restart the test.

============================= test session starts =============================
platform win32 -- Python 3.6.6, pytest-5.0.1, py-1.5.2, pluggy-0.12.0
rootdir: C:\Multi-Runner\builds\bd095854\0\anybody\beta\ammr, inifile: pytest.ini
plugins: forked-0.2, xdist-1.22.1
gw0 I / gw1 I / gw2 I / gw3 I

gw0 [233] / gw1 [233] / gw2 [233] / gw3 [233]

scheduling tests via LoadScheduling
....................................................F................... [ 30%]
........................................................................ [ 61%]
........................................................................ [ 92%]
................[gw3] node down: Traceback (most recent call last):
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1324, in _save
    dispatch = self._dispatch[tp]
KeyError: <enum 'ExitCode'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1039, in executetask
    do_exec(co, loc) # noqa
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 205, in <module>
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\manager.py", line 87, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\manager.py", line 81, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\callers.py", line 208, in _multicall
    return outcome.get_result()
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\_pytest\main.py", line 250, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\_pytest\main.py", line 243, in wrap_session
    session=session, exitstatus=session.exitstatus
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\manager.py", line 87, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\manager.py", line 81, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\pluggy\callers.py", line 203, in _multicall
    gen.send(outcome)
  File "<remote exec>", line 44, in pytest_sessionfinish
  File "<remote exec>", line 29, in sendevent
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 691, in send
    self.gateway._send(Message.CHANNEL_DATA, self.id, dumps_internal(item))
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1295, in dumps_internal
    return _Serializer().save(obj)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1313, in save
    self._save(obj)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1331, in _save
    dispatch(self, obj)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1412, in save_tuple
    self._save(item)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1331, in _save
    dispatch(self, obj)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1408, in save_dict
    self._write_setitem(key, value)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1402, in _write_setitem
    self._save(value)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1331, in _save
    dispatch(self, obj)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1408, in save_dict
    self._write_setitem(key, value)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1402, in _write_setitem
    self._save(value)
  File "C:\Users\gitlabrunner\Miniconda3\lib\site-packages\execnet\gateway_base.py", line 1329, in _save
    raise DumpError("can't serialize %s" % (tp,))
execnet.gateway_base.DumpError: can't serialize <enum 'ExitCode'>

orsinium · 2019-09-12T09:54:25Z

The same on pytest 5.0.1, 5.1.2. Version 4.6.5 is ok. I guess, there is the reason (5.0.0 release changelog):

Session.exitcode values are now coded in pytest.ExitCode, an IntEnum. This makes the exit code available for consumer code and are more explicit other than just documentation. User defined exit codes are still valid, but should be used with caution.

pytest-dev/pytest#5125

ssbarnea · 2019-11-02T14:19:16Z

I encountered a very similar error when trying to use xdist with pytest-molecule:

gw0 C / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 IINTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/ssbarnea/.pyenv/versions/3.7.4/lib/python3.7/site-packages/execnet/gateway_base.py", line 1400, in _save
INTERNALERROR>     dispatch = self._dispatch[tp]
INTERNALERROR> KeyError: <class 'molecule_azure.driver.Azure'>
INTERNALERROR>
....
INTERNALERROR>     self._save(key)
INTERNALERROR>   File "/Users/ssbarnea/.pyenv/versions/3.7.4/lib/python3.7/site-packages/execnet/gateway_base.py", line 1405, in _save
INTERNALERROR>     raise DumpError("can't serialize {}".format(tp))
INTERNALERROR> execnet.gateway_base.DumpError: can't serialize <class 'molecule_azure.driver.Azure'>

I mention that the plugin works fine when not used with xdist, only enabling xdist triggers this error.
molecule_azure happens to be a plugin of molecule tool, which also uses pluggy.

I have no idea what is confusing them because they all use different entry points, like seen on https://github.com/pycontribs/molecule-azure/blob/master/setup.cfg#L85-L87 -- they are not supposed to interfere one with another.

RonnyPfannschmidt · 2019-11-02T14:28:27Z

@ssbarnea different issue - https://github.com/pycontribs/pytest-molecule/blob/master/pytest_molecule/__init__.py#L55 bug in pytest-molecule, unserializable option setups like that arent supported by pytest

Mainly pytest options must be serializable and our code was passing Driver instances instead of strings to pytest options dictionary. This was breaking python-xdist module, even if pytest-molecule was not used. Related: pytest-dev/pytest-xdist#464

frostming · 2020-04-15T03:13:24Z

@RonnyPfannschmidt @ssbarnea Same issue here, what is the proposed solution?

Triggered by the combination of pytest-xdist + pytest-cov + MacOS. I have no idea why it will pass if any of the three conditions doesn't meet.

https://github.com/frostming/pdm/pull/109/checks?check_run_id=587701930

graingert · 2024-10-24T08:13:31Z

@RonnyPfannschmidt @ssbarnea Same issue here, what is the proposed solution?

Triggered by the combination of pytest-xdist + pytest-cov + MacOS. I have no idea why it will pass if any of the three conditions doesn't meet.

frostming/pdm#109 (checks)

for this workflow run logs have been deleted, so you still see the error on the latest pytest-xdist?

RonnyPfannschmidt · 2024-10-24T08:17:24Z

closin this one as not actionable

nicoddemus added the needs information label Aug 23, 2019

ssbarnea mentioned this issue Nov 2, 2019

Fix pytest serialization issue ansible-community/pytest-molecule#50

Merged

This was referenced Nov 12, 2019

Trying to update pytest 4 to 5 -> pytest_report_teststatus(report) Trying to update pytest 4.0 to 5 pytest-dev/pytest#6169

Closed

execnet.gateway_base.DumpError: can't serialize <enum 'ExitCode'> #487

Closed

valeriupredoi mentioned this issue Sep 18, 2020

Segmentation fault when running tests on CircleCI ESMValGroup/ESMValCore#644

Closed

RonnyPfannschmidt closed this as completed Oct 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

execnet.gateway_base.DumpError: can't serialize <enum 'ExitCode'> #464

execnet.gateway_base.DumpError: can't serialize <enum 'ExitCode'> #464

tiffinysmile commented Aug 23, 2019 •

edited by nicoddemus

Loading

nicoddemus commented Aug 23, 2019 •

edited

Loading

tiffinysmile commented Aug 26, 2019

nicoddemus commented Aug 26, 2019

melund commented Sep 6, 2019

orsinium commented Sep 12, 2019

ssbarnea commented Nov 2, 2019

RonnyPfannschmidt commented Nov 2, 2019

frostming commented Apr 15, 2020 •

edited

Loading

graingert commented Oct 24, 2024 •

edited

Loading

RonnyPfannschmidt commented Oct 24, 2024

execnet.gateway_base.DumpError: can't serialize <enum 'ExitCode'> #464

execnet.gateway_base.DumpError: can't serialize <enum 'ExitCode'> #464

Comments

tiffinysmile commented Aug 23, 2019 • edited by nicoddemus Loading

nicoddemus commented Aug 23, 2019 • edited Loading

tiffinysmile commented Aug 26, 2019

nicoddemus commented Aug 26, 2019

melund commented Sep 6, 2019

orsinium commented Sep 12, 2019

ssbarnea commented Nov 2, 2019

RonnyPfannschmidt commented Nov 2, 2019

frostming commented Apr 15, 2020 • edited Loading

graingert commented Oct 24, 2024 • edited Loading

RonnyPfannschmidt commented Oct 24, 2024

tiffinysmile commented Aug 23, 2019 •

edited by nicoddemus

Loading

nicoddemus commented Aug 23, 2019 •

edited

Loading

frostming commented Apr 15, 2020 •

edited

Loading

graingert commented Oct 24, 2024 •

edited

Loading