Can't download pipeline artifact files use matching patterns starting with artifact name

[beatcracker]

I'm raising this issue here since pipeline artifacts publish/download tasks are agent plugins.

Agent Version and Platform

Agent 2.186.1 on ubuntu-20.04

Azure DevOps Type and Version

dev.azure.com

What's not working?

Pipeline artifact download task with file matching patterns patterns doesn't work as expected. When downloading multiple artifacts, you can't filter files by artifact name.

According to docs:

File matching patterns should assume the first segment of the pattern is (or matches) an artifact name. For example, WebApp/** matches all files from the WebApp artifact. The pattern */*.dll matches all files with a .dll extension at the root of each artifact.

So one might expect that for existing artifact WebApp, the pattern WebApp/** will download all files. Unfortunately this doesn't happen and such patterns result in 0 files being downloaded.

Related Stack Overflow question: https://stackoverflow.com/questions/67567408

Example

This template will not download any files in the Download job.

Minimatch patterns: [test/*.xyz]
Filtered 0 files from the Minimatch filters supplied.

pool:
  vmImage: ubuntu-latest

jobs:
  - job: Publish
    steps:
      - bash:
          mkdir -p test
          touch test/test.xyz
        displayName: Create artifact

      - bash: find . -name '*.xyz'
        displayName: List created artifact

      - publish: test
        artifact: test
        displayName: Publish test

  - job: Download
    dependsOn: Publish
    steps:
      - download: current
        patterns: test/*.xyz
        displayName: Download using pattern

      - bash: find $(Pipeline.Workspace) -name '*.xyz' 
        displayName: List downloaded artifacts

[anatolybolshakov]

Hi @beatcracker 'patterns' input for Download Pipeline Artifacts represents patterns for paths relevant to the published artifact folder - so if you specified 'test' folder as a targetPath for Publish Pipeline Artifacts task - 'test' folder content will be published, and you need to specify path mask relatively to root of this folder ('.xyz' instead of 'test/.xyz').
This is expected behavior - please see also docs for Download Pipeline Artifacts and Publish Pipeline Artifacts.

[beatcracker]

@anatolybolshakov Yes, current behavior make sense from the practical standpoint, but it doesn't match what documentation says, i.e., For example, WebApp/** matches all files from the WebApp artifact. Should I raise a docs issue to change this bit about patterns in conjunction with multiple artifacts?

[anatolybolshakov]

@beatcracker oh yes, I see - let us prepare documentation changes for this.

[github-actions]

This issue has had no activity in 180 days. Please comment if it is not actually stale

[beatcracker]

Docs are not updated yet, so not stale.

[jamesthurley]

The incorrect docs just caught me out as well.

I have a parallel job producing artifacts called MyArtifact_$(System.JobPositionInPhase), then in another job I want to download all those artifacts and merge them. It seemed like the way to do this was to specify something like patterns: MyArtifact_*/* in the download step, however this doesn't work. Neither do other variations on that pattern like patterns: MyArtifact_*/**/*.

It would be helpful if either the docs were updated or, even better, if the functionality was implemented as documented.

[tknerr]

In fact, I see a totally different behaviour: the artifact name can be matched via patterns: in the download, but using multiple patterns (which are also supposed to work according to the docs) do not work at all!?

For me this is on Linux agents with agent version 2.185.1, and DownloadPipelineArtifacts version is 2.198.0

@beatcracker @jamesthurley mind re-checking if your examples are working now (maybe in the meantime something had changed on server side)?

[github-actions]

This issue has had no activity in 180 days. Please comment if it is not actually stale

[sloncho]

@anatolybolshakov This is still an issue, and must be reopened. The documentation is incorrect. And the question stays - besides using multiple download steps, is there a way to download multiple artifacts (but not all)?