BUG: Fail build of docker bidsapp

[pdedumast]

Step 13/40 : RUN conda env create -f /app/environment.yml
[...]
[...]
[...]
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... 

done
Installing pip dependencies: ...working... Pip subprocess error:
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/envs/pymialsrtk-env/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-usuxg6tn/apptools_a0d019a67c224bb4bbf17cb1753d2319/setup.py'"'"'; __file__='"'"'/tmp/pip-install-usuxg6tn/apptools_a0d019a67c224bb4bbf17cb1753d2319/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gikwswr2
         cwd: /tmp/pip-install-usuxg6tn/apptools_a0d019a67c224bb4bbf17cb1753d2319/
    Complete output (1 lines):
    error in apptools setup command: use_2to3 is invalid.
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/3f/b1/d89ad215cd3ac0bea71a10e2fe46d74e2009ad7eb31980e3946c493a6c58/apptools-4.4.0.zip#sha256=f5db6af37edfb46246ba2bfd0914c8770c1feb85bc84c51c991aff90de96b015 (from https://pypi.org/simple/apptools/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/envs/pymialsrtk-env/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-usuxg6tn/apptools_679f5b8d4c7f477d9ad117db9a38dbda/setup.py'"'"'; __file__='"'"'/tmp/pip-install-usuxg6tn/apptools_679f5b8d4c7f477d9ad117db9a38dbda/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-grogxbbm
         cwd: /tmp/pip-install-usuxg6tn/apptools_679f5b8d4c7f477d9ad117db9a38dbda/
    Complete output (1 lines):
    error in apptools setup command: use_2to3 is invalid.
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/a7/9e/b866f3c170d59d39bd44560734acbf5528ba7ada2fa3dd768e46eaa59dce/apptools-4.4.0.tar.gz#sha256=73f5c2344d8d36b81f164e9e576425749e91342cf9e8615f3d72627af0de86b7 (from https://pypi.org/simple/apptools/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/envs/pymialsrtk-env/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-usuxg6tn/apptools_dec7d047677445ddb9bf28a202604d35/setup.py'"'"'; __file__='"'"'/tmp/pip-install-usuxg6tn/apptools_dec7d047677445ddb9bf28a202604d35/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-b4_v75z8
         cwd: /tmp/pip-install-usuxg6tn/apptools_dec7d047677445ddb9bf28a202604d35/
    Complete output (1 lines):
    error in apptools setup command: use_2to3 is invalid.
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/78/84/64764e2c87fcb7ad7521bdabc7c44c109b80691f94218cbddd5df2ee682c/apptools-4.4.0.tar.bz2#sha256=bb61c54a5043a8beda68129b0aab3cc47f4a5eb6bc3671cc0e14461637c398d1 (from https://pypi.org/simple/apptools/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement apptools==4.4.0
ERROR: No matching distribution found for apptools==4.4.0

Ran pip subprocess with arguments:
['/opt/conda/envs/pymialsrtk-env/bin/python', '-m', 'pip', 'install', '-U', '-r', '/app/condaenv.65mwz_gl.requirements.txt']
Pip subprocess output:
Collecting coverage==5.1
  Downloading coverage-5.1-cp37-cp37m-manylinux1_x86_64.whl (227 kB)
Collecting duecredit==0.8.0
  Downloading duecredit-0.8.0.tar.gz (68 kB)
Collecting apptools==4.4.0
  Downloading apptools-4.4.0.zip (490 kB)
  Downloading apptools-4.4.0.tar.gz (294 kB)
  Downloading apptools-4.4.0.tar.bz2 (241 kB)

failed

CondaEnvException: Pip failed

The command '/bin/sh -c conda env create -f /app/environment.yml' returned a non-zero code: 1

[pdedumast]

@sebastientourbier seems ok to remove the dependency apptools==4.4.0... Can you confirm?

[sebastientourbier]

@pdedumast I am not sure... It might be a dependency with setuptools to install the package but by looking at the setup.py and setup_pypi.py, we import only setuptools modules. By also looking of the history of docker/bidsapp/environment.yml this dependency seems to have been there since the beginning so it should really be tested before removal.

Based on a similar issue reported here, the cause of the problem would be breaking change introduced in the last 58 version of setuptools:

https://setuptools.readthedocs.io/en/latest/history.html#v58-0-0
#2086: Removed support for 2to3 during builds. Projects should port to a unified codebase or pin to an older version of Setuptools using PEP 518 build-requires.

In our case, I see two options if we need to keep apptools:

1. try to update apptools to the latest version in the environment.yml (Update line - apptools==4.4.0 to - apptools==5.1.0)
2. try to fix setuptools to 57.5.0 version in the environment.yml (Add line - setuptools==57.5.0 before - apptools==...)