[BUG] 3.8.0 writes wrong json result

[fschulze]

The json report puts the test run info into the setup section instead of the test section.

The devpi test suite contains tests with an empty package except the following tox.ini:

[testenv]
commands = python -c "print('ok')"

With 3.7.0 the following json is generated:

{
  "reportversion": "1",
  "toxversion": "3.7.0",
  "platform": "darwin",
  "host": "trubel.lan",
  "testenvs": {
    "GLOB": {
      "setup": [
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/bin/python3.6",
            "/Users/fschulze/temp/toxtemp/exa-1.0/setup.py",
            "sdist",
            "--formats=zip",
            "--dist-dir",
            "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/dist"
          ],
          "output": "actionid: tox\nmsg: packaging\ncmdargs: '/Users/fschulze/temp/toxtemp/bin/python3.6 /Users/fschulze/temp/toxtemp/exa-1.0/setup.py sdist --formats=zip --dist-dir /Users/fschulze/temp/toxtemp/exa-1.0/.tox/dist'\n\nrunning sdist\nrunning egg_info\ncreating exa.egg-info\nwriting exa.egg-info/PKG-INFO\nwriting dependency_links to exa.egg-info/dependency_links.txt\nwriting top-level names to exa.egg-info/top_level.txt\nwriting manifest file 'exa.egg-info/SOURCES.txt'\nreading manifest file 'exa.egg-info/SOURCES.txt'\nreading manifest template 'MANIFEST.in'\nwriting manifest file 'exa.egg-info/SOURCES.txt'\nwarning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md\n\nrunning check\nwarning: check: missing required meta-data: url\n\nwarning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied\n\ncreating exa-1.0\ncreating exa-1.0/exa\ncreating exa-1.0/exa.egg-info\ncopying files to exa-1.0...\ncopying MANIFEST.in -> exa-1.0\ncopying setup.cfg -> exa-1.0\ncopying setup.py -> exa-1.0\ncopying tox.ini -> exa-1.0\ncopying ./exa/__init__.py -> exa-1.0/./exa\ncopying exa.egg-info/PKG-INFO -> exa-1.0/exa.egg-info\ncopying exa.egg-info/SOURCES.txt -> exa-1.0/exa.egg-info\ncopying exa.egg-info/dependency_links.txt -> exa-1.0/exa.egg-info\ncopying exa.egg-info/top_level.txt -> exa-1.0/exa.egg-info\nWriting exa-1.0/setup.cfg\ncreating .tox/dist\ncreating '.tox/dist/exa-1.0.zip' and adding 'exa-1.0' to it\nadding 'exa-1.0'\nadding 'exa-1.0/exa.egg-info'\nadding 'exa-1.0/exa'\nadding 'exa-1.0/PKG-INFO'\nadding 'exa-1.0/MANIFEST.in'\nadding 'exa-1.0/setup.py'\nadding 'exa-1.0/tox.ini'\nadding 'exa-1.0/setup.cfg'\nadding 'exa-1.0/exa.egg-info/PKG-INFO'\nadding 'exa-1.0/exa.egg-info/SOURCES.txt'\nadding 'exa-1.0/exa.egg-info/top_level.txt'\nadding 'exa-1.0/exa.egg-info/dependency_links.txt'\nadding 'exa-1.0/exa/__init__.py'\nremoving 'exa-1.0' (and everything under it)\n",
          "retcode": "0"
        }
      ]
    },
    "python": {
      "setup": [
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/bin/python3.6",
            "-m",
            "virtualenv",
            "--python",
            "/Users/fschulze/temp/toxtemp/bin/python3.6",
            "python"
          ],
          "output": "actionid: python\nmsg: getenv\ncmdargs: '/Users/fschulze/temp/toxtemp/bin/python3.6 -m virtualenv --python /Users/fschulze/temp/toxtemp/bin/python3.6 python'\n\nAlready using interpreter /Users/fschulze/temp/toxtemp/bin/python3.6\nUsing real prefix '/Users/fschulze/.pyenv/versions/3.6.8'\nNew python executable in /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python3.6\nAlso creating executable in /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python\nPlease make sure you remove any previous custom paths from your /Users/fschulze/.pydistutils.cfg file.\nInstalling setuptools, pip, wheel...\ndone.\n",
          "retcode": "0"
        },
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python",
            "-m",
            "pip",
            "install",
            "--exists-action",
            "w",
            "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/.tmp/package/1/exa-1.0.zip"
          ],
          "output": "actionid: python\nmsg: installpkg\ncmdargs: '/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python -m pip install --exists-action w /Users/fschulze/temp/toxtemp/exa-1.0/.tox/.tmp/package/1/exa-1.0.zip'\n\nLooking in indexes: https://d.rzon.de:8141/fschulze/dev/+simple/\nProcessing ./.tox/.tmp/package/1/exa-1.0.zip\nBuilding wheels for collected packages: exa\n  Building wheel for exa (setup.py): started\n  Building wheel for exa (setup.py): finished with status 'done'\n  Stored in directory: /Users/fschulze/Library/Caches/pip/wheels/f7/5d/5e/77cd954dfa61ac347261f199bd23022304543d148113d74582\nSuccessfully built exa\nInstalling collected packages: exa\n  Found existing installation: exa 1.0\n    Not uninstalling exa at /Users/fschulze/temp/toxtemp/exa-1.0, outside environment /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python\n    Can't uninstall 'exa'. No files were found to uninstall.\nSuccessfully installed exa-1.0\n",
          "retcode": "0"
        },
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python",
            "-m",
            "pip",
            "freeze"
          ],
          "output": "actionid: python\nmsg: envreport\ncmdargs: '/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python -m pip freeze'\n\nexa==1.0\n",
          "retcode": "0"
        }
      ],
      "python": {
        "executable": "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python",
        "version_info": [
          3,
          6,
          8,
          "final",
          0
        ],
        "version": "3.6.8 (default, Feb 19 2019, 13:39:03) \n[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.10.44.4)]"
      },
      "installed_packages": [
        "exa==1.0"
      ],
      "test": [
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python",
            "-c",
            "print('ok')"
          ],
          "output": "actionid: python\nmsg: runtests\ncmdargs: '/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python -c \\'print(\\'\"\\'\"\\'ok\\'\"\\'\"\\')\\''\n\nok\n",
          "retcode": "0"
        }
      ]
    }
  },
  "installpkg": {
    "md5": "2f6155628cc5485bceef152e181744fb",
    "sha256": "5afff8613783e33fd05be1c7f880a30fd873241821c9eb26fac1269687ae165a",
    "basename": "exa-1.0.zip"
  }
}

With 3.8.0 the following is generated:

{
  "reportversion": "1",
  "toxversion": "3.8.0",
  "platform": "darwin",
  "host": "trubel.lan",
  "commands": [
    {
      "command": [
        "/Users/fschulze/temp/toxtemp/bin/python3.6",
        "setup.py",
        "sdist",
        "--formats=zip",
        "--dist-dir",
        "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/dist"
      ],
      "output": "action: GLOB, msg: packaging\ncwd: /Users/fschulze/temp/toxtemp/exa-1.0\ncmd: /Users/fschulze/temp/toxtemp/bin/python3.6 setup.py sdist --formats=zip --dist-dir /Users/fschulze/temp/toxtemp/exa-1.0/.tox/dist\nrunning sdist\nrunning egg_info\ncreating exa.egg-info\nwriting exa.egg-info/PKG-INFO\nwriting dependency_links to exa.egg-info/dependency_links.txt\nwriting top-level names to exa.egg-info/top_level.txt\nwriting manifest file 'exa.egg-info/SOURCES.txt'\nreading manifest file 'exa.egg-info/SOURCES.txt'\nreading manifest template 'MANIFEST.in'\nwriting manifest file 'exa.egg-info/SOURCES.txt'\nwarning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md\n\nrunning check\nwarning: check: missing required meta-data: url\n\nwarning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied\n\ncreating exa-1.0\ncreating exa-1.0/exa\ncreating exa-1.0/exa.egg-info\ncopying files to exa-1.0...\ncopying MANIFEST.in -> exa-1.0\ncopying setup.cfg -> exa-1.0\ncopying setup.py -> exa-1.0\ncopying tox.ini -> exa-1.0\ncopying ./exa/__init__.py -> exa-1.0/./exa\ncopying exa.egg-info/PKG-INFO -> exa-1.0/exa.egg-info\ncopying exa.egg-info/SOURCES.txt -> exa-1.0/exa.egg-info\ncopying exa.egg-info/dependency_links.txt -> exa-1.0/exa.egg-info\ncopying exa.egg-info/top_level.txt -> exa-1.0/exa.egg-info\nWriting exa-1.0/setup.cfg\ncreating /Users/fschulze/temp/toxtemp/exa-1.0/.tox/dist\ncreating '/Users/fschulze/temp/toxtemp/exa-1.0/.tox/dist/exa-1.0.zip' and adding 'exa-1.0' to it\nadding 'exa-1.0'\nadding 'exa-1.0/exa.egg-info'\nadding 'exa-1.0/exa'\nadding 'exa-1.0/PKG-INFO'\nadding 'exa-1.0/MANIFEST.in'\nadding 'exa-1.0/setup.py'\nadding 'exa-1.0/tox.ini'\nadding 'exa-1.0/setup.cfg'\nadding 'exa-1.0/exa.egg-info/PKG-INFO'\nadding 'exa-1.0/exa.egg-info/SOURCES.txt'\nadding 'exa-1.0/exa.egg-info/top_level.txt'\nadding 'exa-1.0/exa.egg-info/dependency_links.txt'\nadding 'exa-1.0/exa/__init__.py'\nremoving 'exa-1.0' (and everything under it)\n",
      "retcode": "0"
    }
  ],
  "testenvs": {
    "python": {
      "setup": [
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/bin/python3.6",
            "-m",
            "virtualenv",
            "--python",
            "/Users/fschulze/temp/toxtemp/bin/python3.6",
            "python"
          ],
          "output": "action: python, msg: getenv\ncwd: /Users/fschulze/temp/toxtemp/exa-1.0/.tox\ncmd: /Users/fschulze/temp/toxtemp/bin/python3.6 -m virtualenv --python /Users/fschulze/temp/toxtemp/bin/python3.6 python\nAlready using interpreter /Users/fschulze/temp/toxtemp/bin/python3.6\nUsing real prefix '/Users/fschulze/.pyenv/versions/3.6.8'\nNew python executable in /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python3.6\nAlso creating executable in /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python\nPlease make sure you remove any previous custom paths from your /Users/fschulze/.pydistutils.cfg file.\nInstalling setuptools, pip, wheel...\ndone.\n",
          "retcode": "0"
        },
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python",
            "-m",
            "pip",
            "install",
            "--exists-action",
            "w",
            ".tox/.tmp/package/1/exa-1.0.zip"
          ],
          "output": "action: python, msg: installpkg\ncwd: /Users/fschulze/temp/toxtemp/exa-1.0\ncmd: /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python -m pip install --exists-action w .tox/.tmp/package/1/exa-1.0.zip\nLooking in indexes: https://d.rzon.de:8141/fschulze/dev/+simple/\nProcessing ./.tox/.tmp/package/1/exa-1.0.zip\nBuilding wheels for collected packages: exa\n  Building wheel for exa (setup.py): started\n  Building wheel for exa (setup.py): finished with status 'done'\n  Stored in directory: /Users/fschulze/Library/Caches/pip/wheels/f7/5d/5e/77cd954dfa61ac347261f199bd23022304543d148113d74582\nSuccessfully built exa\nInstalling collected packages: exa\n  Found existing installation: exa 1.0\n    Not uninstalling exa at /Users/fschulze/temp/toxtemp/exa-1.0, outside environment /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python\n    Can't uninstall 'exa'. No files were found to uninstall.\nSuccessfully installed exa-1.0\n",
          "retcode": "0"
        },
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python",
            "-m",
            "pip",
            "freeze"
          ],
          "output": "action: python, msg: envreport\ncwd: /Users/fschulze/temp/toxtemp/exa-1.0\ncmd: /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python -m pip freeze\nexa==1.0\n",
          "retcode": "0"
        },
        {
          "command": [
            "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python",
            "-c",
            "print('ok')"
          ],
          "output": "action: python, msg: runtests\ncwd: /Users/fschulze/temp/toxtemp/exa-1.0\ncmd: /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python -c 'print('\"'\"'ok'\"'\"')'\nok\n",
          "retcode": "0"
        }
      ],
      "python": {
        "executable": "/Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python",
        "version_info": [
          3,
          6,
          8,
          "final",
          0
        ],
        "version": "3.6.8 (default, Feb 19 2019, 13:39:03) \n[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.10.44.4)]",
        "sysplatform": "darwin"
      },
      "installpkg": {
        "md5": "5259a1405151921b052f9adec2255937",
        "sha256": "b1257e978ea13929c8299241d82b851a8d7b74e8f2013cc8f9d3d6038e65e232",
        "basename": "exa-1.0.zip"
      },
      "installed_packages": [
        "action: python, msg: envreport",
        "cwd: /Users/fschulze/temp/toxtemp/exa-1.0",
        "cmd: /Users/fschulze/temp/toxtemp/exa-1.0/.tox/python/bin/python -m pip freeze",
        "exa==1.0"
      ]
    }
  }
}

Notice that with 3.8.0 there is no "test" section at all and the print('ok') part is in the "setup" section.

[gaborbernat]

Fixed via #1216