Is COPY/ADD supposed to work with files in subdirectories?

[srcshelton]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

I have a Dockerfile containing:

COPY subdirectory/file /destination/
RUN  ls -l /destination/file

... and I've found that the COPY directive succeeds, then the RUN command fails. Actually, it seems that the COPY (and ADD) directives can't handle subdirectories, but regardless succeed even though no file has been copied.

I've solved this by changing directory in my build script to effectively move the context folder one level deeper so that I can just do COPY file /destination (which works, for otherwise exactly the same setup).

But surely, either the COPY/ADD directives should copy the specified file/directory, or they should fail. They should not succeed without creating the intended files within the container (and if locating a source file in a subdirectory is disallowed, then it really should say so!)
Output of podman version:

Version:      2.0.0-rc6
API Version:  1
Go Version:   go1.14.2
Git Commit:   e74e84aa0637033c821955d817fca85f56a90d85
Built:        Mon Jun 29 01:04:58 2020
OS/Arch:      linux/amd64

(... still on 2.0.0_rc6 pending the resolution of #6785)

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.15.0-dev
  cgroupVersion: v1
  conmon:
    package: Unknown
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.17, commit: 41877362fc4685d55e0473d2e4a1cbe5e1debee0'
  cpus: 8
  distribution:
    distribution: gentoo
    version: unknown
  eventLogger: file
  hostname: dellr330
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.4.38-gentoo
  linkmode: dynamic
  memFree: 1262424064
  memTotal: 8132182016
  ociRuntime:
    name: runc
    package: Unknown
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc10
      commit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
      spec: 1.0.1-dev
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  rootless: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 25122934784
  swapTotal: 25769787392
  uptime: 656h 34m 23.71s (Approximately 27.33 days)
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.ignore_chown_errors: "false"
  graphRoot: /space/podman/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 931
  runRoot: /space/podman/run
  volumePath: /space/podman/storage/volumes
version:
  APIVersion: 1
  Built: 1593392698
  BuiltTime: Mon Jun 29 01:04:58 2020
  GitCommit: e74e84aa0637033c821955d817fca85f56a90d85
  GoVersion: go1.14.2
  OsArch: linux/amd64
  Version: 2.0.0-rc6

[mheon]

@TomSweeneyRedHat PTAL

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[zhangguanzhang]

this is the reulst of the docker:

$ mkdir temp
$ mkdir temp/temp
$ touch temp/temp/test1
$ vi Dockerfile
$ cat Dockerfile
FROM alpine
COPY temp/temp /test
RUN ls -l /test

build it

$ docker build -t test .
Sending build context to Docker daemon  13.82kB
Step 1/3 : FROM alpine
latest: Pulling from library/alpine
df20fa9351a1: Pull complete
Digest: sha256:185518070891758909c9f839cf4ca393ee977ac378609f700f60a771a2dfe321
Status: Downloaded newer image for alpine:latest
 ---> a24bb4013296
Step 2/3 : COPY temp/temp /test
 ---> 7b715c462071
Step 3/3 : RUN ls -l /test
 ---> Running in e206f5ff9289
total 0
-rw-r--r--    1 root     root             0 Aug  2 05:00 test1
Removing intermediate container e206f5ff9289
 ---> f9a32d896597
Successfully built f9a32d896597
Successfully tagged test:latest

[rhatdan]

 $ podman build .
STEP 1: FROM alpine
STEP 2: COPY temp/temp /test
--> 56804ae1c44
STEP 3: RUN ls -l /test
total 0
-rw-rw-r--    1 root     root             0 Aug  4 18:44 test1
STEP 4: COMMIT
--> 9b122533af1
9b122533af105fcb6c1e9bbeaaa9752e0756281623501686d8c94410d8ed0712

What am I missing?

[srcshelton]

What am I missing?

Confirmed - since raising this issue, I've upgraded from podman version 2.0.0-rc6 to 2.0.3 and I now get the same result as you, that the file is successfully copied.

Is there any means to verify that there are no other edge-cases where COPY/ADD can succeed without having created the intended file, since I assume that the false-positive in 2.0.0-rc6 wasn't picked-up by any automated testing?

[rhatdan]

@nalind is adding a ton of testing for building container images to Buildah, which will hopefully catch these issues.