Can't run only one test case from combinatorial test

[stryczekk]

Hi, I have updated Visual Studio to the newest version, and i have some problems with executing test cases in vs.
I have combinatorial test and if i want to run only one test case, but nunit executing all cases for selected test

In the output window i can see that all 6 cases are executed
output.txt

How can I execute only few test cases? Is there any workaround?
It's very urgent for me

• NUnit v3.12.0

• NUnit3TestAdapter v3.16.0

• Visual Studio edition and full version number (see Help About): v16.4.2

• A short repro, preferably attached or pointing to a git repo or gist
  nunitadapterbug.zip

• What .net platform and version is being targeted: .NET Framework 4.7

[OsirisTerje]

This is part of the behavior change in 3.16, done to fix data driven tests, but has this unfortunate side effect. We need to look into this.
I suggest you downgrade the adapter to use version 3.15.1, which will do what you want.

[stryczekk]

Thanks for your reply.
With extension v3.15.1 and without nuget package in project is working as I expected.

[coleburg]

I have noticed the same behaviour with my automated tests, I use the browser in the TestFixture, with 3.16.0 when you build, its removes the (chrome) for example from the tests and separates all tests with parameters and non parametised, which means its breaks all automated tests linked to Azure Dev Ops.

If you need more info to help fix the bug, please let me know.

Thanks

[OsirisTerje]

@coleburg It would be nice to have some more info on how your tests are being used. Also, for builds in ADO I assume you then have some filters, can you describe them (as examples).
The change in 3.16 will change the way tests are named, since we have changed from Names to FQN. The behavior described by @stryczekk is unwanted, and since the tests he shows are simpler than the ones the change in 3.16 is meant to fix, we need to change the default behavior here.

[OsirisTerje]

In 3.16.1 the default will be to use the FQN of the test itself, not the parent. There will be a switch in the runsettings that can be used to opt in the parent FQN behaviour.

[srinirao59]

I have noticed the same behaviour with my automated tests, I use the browser in the TestFixture, with 3.16.0 when you build, its removes the (chrome) for example from the tests and separates all tests with parameters and non parametised, which means its breaks all automated tests linked to Azure Dev Ops.

If you need more info to help fix the bug, please let me know.

Thanks

Hi I am facing the same issue too.

Please see the below stack trace

DiscoveryMessage : NUnit Adapter 3.16.0.0: Test discovery starting

##[warning]DiscoveryMessage : Exception NUnit.Engine.NUnitEngineException, Exception thrown discovering tests in E:\vsagent.......

##[warning]DiscoveryMessage : An exception occurred in the driver while loading tests.

##[warning]DiscoveryMessage : at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)

at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()

at NUnit.Engine.Runners.TestDomainRunner.LoadPackage()

at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()

at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)

at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)

at NUnit.VisualStudio.TestAdapter.NUnit3TestDiscoverer.DiscoverTests(IEnumerable`1 sources, IDiscoveryContext discoveryContext, IMessageLogger messageLogger, ITestCaseDiscoverySink discoverySink) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestDiscoverer.cs:line 91

##[warning]DiscoveryMessage : Innerexception: System.IO.FileNotFoundException: Could not load file or assembly 'nunit.framework, Version=3.12.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' or one of its dependencies. The system cannot find the file specified.

File name: 'nunit.framework, Version=3.12.0.0, Culture=neutral, PublicKeyToken=2638cd05610744eb'

at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)

at System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)

at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)

at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)

at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)

at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)

at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)

at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)

at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)

[OsirisTerje]

@srinirao59 @stryczekk Please verify if this beta version works in your case https://www.myget.org/feed/nunit/package/nuget/NUnit3TestAdapter/3.16.1-dev-01246

[stryczekk]

Yes, works as expected. Thanks!

[srinirao59]

@OsirisTerje : It working now thank you, when we can expect this release in nuget?

[OsirisTerje]

@srinirao59 Awesome!. Eta is saturday.

[OsirisTerje]

This issue is resolved by Version 3.16.1 which is released now on nuget.org and VSIX on Visual Studio marketplace. See all issues resolved and Release notes for details.