Debugging unit tests in a sub directory

[martingbrown]

I'm working on Windows 10 with VSCode and the Python plugin. These are all up to date as far as I can tell.

I've some unittest tests that sit in a subdirectory of my main working directory called .\src. These in return load files in a second sub directory .\src\test\test.dat. The tests simply refer to this as test\test.dat as they assume the current directory is .\src.

I'm having real trouble getting testing setup so that this works. I've been able to run these in the past but debugging a test wasn't working. I changed my settings to:

    "python.linting.pydocstyleEnabled": true,
    "python.testing.unittestArgs": [
        "-p",
        "*test.py",
        "-s",
        "./src",
    ],
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,    
    "python.testing.unittestEnabled": true

And debugging started working, but the tests don't find the data file. Because the current working directory is just .\. If I change the settings to this debugging stops working. Surprisingly though vscode can still find and run the tests, even though my reading of this is that it should be looking in .\src\src which does not exist.

     "python.testing.unittestArgs": [
        "-p",
        "*test.py",
        "-s",
        "./src",
    ],
    "python.testing.cwd": ".\\src",
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,    
    "python.testing.unittestEnabled": true

If I change the settings to this it can't even discover the tests:

    "python.testing.unittestArgs": [
        "-p",
        "*test.py"
    ],
    "python.testing.cwd": ".\\src",
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,    
    "python.testing.unittestEnabled": true

I'm now totally confused as to how to set this up so that running tests works and debugging tests works and they both find the test files.

[karrtikr]

Unittest args require a -s flag to work. However the following didn't work because

     "python.testing.unittestArgs": [
        "-p",
        "*test.py",
        "-s",
        "./src",
    ],
    "python.testing.cwd": ".\\src",
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,    
    "python.testing.unittestEnabled": true

Because ./src in python.testing.unittestArgs is resolved to ./src/src as cwd is set to ./src, and as that directory doesn't exist, discovering tests fail.

You can simply use the full path to specify the test location, or better use the variable ${workspaceFolder}

     "python.testing.unittestArgs": [
        "-p",
        "*test.py",
        "-s",
        "${workspaceFolder}/src",
    ],
    "python.testing.cwd": ".\\src",
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,    
    "python.testing.unittestEnabled": true

Let me know if that works.

[martingbrown]

When you say "the following didn't work", it did work to a degree. With the settings like the following, it discovers the tests and runs them. But the tests are not run with the .\src directory as the working directory. And also debugging tests doesn't seem to work.

     "python.testing.unittestArgs": [
        "-p",
        "*test.py",
        "-s",
        "./src",
    ],
    "python.testing.cwd": ".\\src",
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,    
    "python.testing.unittestEnabled": true

I changed my settings as you suggested to the following but it then fails to discover the tests:

    "python.testing.autoTestDiscoverOnSaveEnabled": true,
    "python.testing.unittestArgs": [
        "-v",
        "-s",
        "${workspaceFolder}/src",
        "-p",
        "*test.py"
    ],
    "python.testing.cwd": ".\\src",
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.unittestEnabled": true,

[karrtikr]

Hey @martingbrown , I am able to spot multiple issues here:

• It seems like debugging tests doesn't pick up the cwd from the setting python.testing.cwd. Instead you've to create the following entry in launch.json - I can confirm this works, so this should get your testing setup

   {
        "name": "Unit Tests",
        "type": "python",
        "request": "test",
        "cwd": "${workspaceFolder}/src"
    } 

Multiple places to set cwd is confusing, so we'll address these as part of #8339.

The above alone should get you working, but I found some more issues:

• Looks like ${workspaceFolder} & . in ${workspaceFolder}/src & ./src is not being resolved correctly (1), this is an issue (created ${workspaceFolder} & . not being resolved in settings in some cases #11671). For now, please try using the full path to the source directory instead,
  <full path to workspace folder>/src

For instance the following worked for me

  "python.testing.unittestArgs": [
    "-v",
    "-s",
    "C:/Users/karraj/Desktop/Desktop/test_demo/src",
    "-p",
    "*test*.py"
  ],

• But surprisingly using backslashes is path (C:\\Users\\karraj\\Desktop\\Desktop\\test_demo\\src) is not working. That's another issue (2). As it perfectly works with python.pythonPath and other settings. Created issue Backslashes doesn't work when specifying the paths with python.testing.unittestArgs #11670
  I think the cwd is not being set for you because of (2). So try using "python.testing.cwd": "./src" instead of "python.testing.cwd": ".\\src". Running the tests using the play button should work after that. - This didn't work, I'll open up a new issue

• Seems like python.testing.cwd is not working anymore. Created issue python.testing.cwd does not work properly #11669

Apologies for so many issues, let me know if this helps. Yours reminded me of #7176, where the guy used the above to solve his issues.

[martingbrown]

I have now the settings.json set like this:

    "python.testing.cwd": ".\\src",
    "python.testing.unittestArgs": [
        "-v",
        "-s",
        "d:/dir1/dir2/src",
        "-p",
        "*test.py"
    ],    
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.unittestEnabled": true,

And I also have this in launch.json

        {
            "name": "Unit Testing",
            "type": "python",
            "request": "test",
            "cwd": "${workspaceFolder}/src"
        },

And now it all seems to be working.

Thanks for that. I appreiate the effort. If I had time I'd try and crack open the source code and try to help you out.

[karrtikr]

No worries. But does running tests using the green play button work? From my understanding, as python.testing.cwd don't work, it shouldn't work either.