dotnet test eats quotes in TestRunParameters

[nohwnd]

Note

Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo

For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

If you have an issue or request for the Azure Pipelines service, use developer community instead:

https://developercommunity.visualstudio.com/spaces/21/index.html )

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: DotNetCoreCLI@2

list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

• Server - Azure Pipelines or TFS on-premises?
  Azure pipelines

  • If using TFS on-premises, provide the version:

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number:

• Agent - Hosted or Private:

  • If using Hosted agent, provide agent queue name:

  • If using private agent, provide the OS of the machine running the agent and the agent version:

Issue Description

- task: DotNetCoreCLI@2
  inputs:
    command: 'test'
    arguments: '-- TestRunParameters.Parameter(name=%22EdiStagingFunctionsKey%22,value=%22key2%22) TestRunParameters.Parameter(name=%22EdiAADKey%22,value=%22key1%22)'

This is the only way I am able to pass TestRunParameters to the run. This was reported here: microsoft/vstest#2387 and the escaping was fixed in vstest and dotnet sdk, but some additional rules are applied on the AzDO task side. Because for example \ is replaced by / and quotes are eaten by the task. So it is not true that what works on the command line works in the task (mentioned here #10712 (comment) ) and none of the approaches described there worked for me, except for providing the quotes as URL escapes %22.

Ideally the user would be able to copy paste from the command line, where \" is the way to escape the quotes properly.

Task logs

https://dev.azure.com/jajares/params/_build?definitionId=6&_a=summary

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

[Insert error from the logs here for a quick overview]

[saquibkhan]

Hi, I have started looking at it.
So far my analysis is as below:
Issue Bug is with azure-task-lib _argStringToArray function - https://github.com/microsoft/azure-pipelines-task-lib/blob/master/node/toolrunner.ts#L88
Changing this function to provide the fix may require a lot of regression testing hence we are exploring other options to provide a mitigation fix in azure-pipeline-tasks

Stack-->
https://github.com/microsoft/azure-pipelines-task-lib/blob/master/node/toolrunner.ts#L88
https://github.com/microsoft/azure-pipelines-task-lib/blob/master/node/toolrunner.ts#L838
https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/DotNetCoreCLIV2/dotnetcore.ts#L170

Other Recommendations:
Fixing at the azure-pipeline-tasks level. Currently exploring this option.

[saquibkhan]

Below are our findings and next steps:

1. If we try passing quotes as a value even that is being eaten up by task-lib
   Input: -- TestRunParameters.Parameter(name=paramName,value=argwith"aquote)
   Output -- TestRunParameters.Parameter(name=paramName,value=argwithaquote)
2. If we use backslash then also quote is eaten up.
   Input: -- TestRunParameters.Parameter(name=paramName,value=argwith\"aquote)
   Output -- TestRunParameters.Parameter(name=paramName,value=argwith\aquote)

Other Inputs:
Input: -- TestRunParameters.Parameter(name=\"EdiStagingFunctionsKey\",value=\"key2\")
Output: TestRunParameters.Parameter(name=\EdiStagingFunctionsKey\",value=\"key2\")
Note: Above the first escaped quotes are eaten up.

Input: TestRunParameters.Parameter(name="EdiStagingFunctionsKey",value="key2"
Output: TestRunParameters.Parameter(name=EdiStagingFunctionsKey,value=key2)

Next Steps:

1. We will update the faq document to help our customers with the issue and a workaround.
2. azure-task-lib issue raised toolrunner _argStringToArray eats up escaped quotes azure-pipelines-task-lib#648

[nohwnd]

microsoft/vstest#862 (comment) I was exploring and fixing the vstest part here. It would be nice if the final syntax was compatible with these ways of escaping, as we don't know from whichOS the user will copy the command. ( Notice that on bash we escape parentheses and spaces.)

# cmd
dotnet test  -- TestRunParameters.Parameter(name=\"myParam\", value=\"value\")

# powershell
dotnet test --%  -- TestRunParameters.Parameter(name=\"myParam\", value=\"value\") 

# bash
dotnet test -- TestRunParameters.Parameter\(name=\"myParam\",\ value=\"value\"\) 

[saquibkhan]

yes @nohwnd once this issue microsoft/azure-pipelines-task-lib#648 is fixed the escape sequence will work

[kcentralsquare]

I am experiencing this same issue with the --collect parameter when attempting to use coverlet as part of my build pipeline. URL escapes do not work.

[saquibkhan]

@kcentralsquare Can you try using %22 instead of quotes and see if that works. For time being this seems to be the only workaround here.

[kcentralsquare]

That did not work for me @saquibkhan.

[saquibkhan]

@kcentralsquare If it's possible can you share the command and argument?

[nehsharmMS]

@kcentralsquare are you still blocked on this issue or can we close this now?

[kcentralsquare]

@nehsharmMS Have you fixed the issue? I've switched to a script instead of using this task, but I would much prefer to use the task.

[jamierytlewski]

@saquibkhan I tried with the %22 and it's still not working. I was trying to use multiple formats for coverlet and was able to use %2c to change out the commas. I have tried this

'--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat="opencover,json,cobertura" /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/ /p:Exclude="[*]Optimist.MrClean.*Service.*"'

And it "eats" up the quotes so the OutputFormat won't show up properly. I'm using a Linux instance to build

[ShreyasRmsft]

@jamierytlewski @kcentralsquare we are dependent on this issue microsoft/azure-pipelines-task-lib#648 getting fixed. Could you please re-create that issue there since the old one got closed due to no activity.

[ALM-at-RZL]

WORKAROUND
Will work with ".NET Core"-Build-Task with Command "test" on on-premise server 2019.0.1. Later not tested. But maybe...

-- "TestRunParameters.Parameter(name=\\\"<name1>\\\", value=\\\"<value1>\\\")" "TestRunParameters.Parameter(name=\\\"<name2>\\\", value=\\\"<value2>\\\")"

[github-actions]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[nohwnd]

@phanikmmsft please remove the stale label. Can we get this fixed please?

[github-actions]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[kcentralsquare]

@ShreyasRmsft Looks like the upstream issue was fixed about 9 months ago. Any updates on this one getting fixed?

[ishansrivastava90]

Any updates on this issue? I am getting failures passing multiple TestRunParameters due to this issue.

The cmd below fails

dotnet test --% -- "TestRunParameters.Parameter(name=\"Name1\", value=\"Val1")" "TestRunParameters.Parameter(name=\"Name2\", value=\"Val2\")"

C:\Program Files\dotnet\sdk\6.0.302\NuGet.RestoreEx.targets(19,5): error : The name "TestRunParameters.Parameter(name" contains an invalid character "."

[wsucoug69]

Is there any update on this?

[davmhelm]

Per earlier comment, once upstream microsoft/azure-pipelines-task-lib#648 was fixed this should automatically resolve, but that does not appear to be the case.

@ShreyasRmsft @nehsharmMS @saquibkhan are there any updates to share on this?

[Plesoo]

Issue is still there. This worked for me:

- task: DotNetCoreCLI@2
    inputs:
    command: 'test'
    arguments: '-- "TestRunParameters.Parameter(name=\"param\", value=\"value\")"'

[ALEJOAREIZA]

Issue is still there. This worked for me:

- task: DotNetCoreCLI@2
    inputs:
    command: 'test'
    arguments: '-- "TestRunParameters.Parameter(name=\"param\", value=\"value\")"'

this work for me as well (y)

[github-actions]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[nohwnd]

@rasunkar this is still a problem, please consider re-opening and fixing the escaping. It would help us with test filtering when there are special characters that need escaping.