test composefs on rawhide #22425

edsantiago · 2024-04-18T15:15:10Z

Run root e2e & system tests using composefs on rawhide.

Write magic settings to storage.conf. That part is easy.

e2e tests, however, ignore storage.conf. They require everything
to be specified on the command line. And "everything", in the
case of composefs, includes a long complicated --pull-options
string which in turn requires containers-storage PR 1966
which, as of this writing, is finally vendored into podman.

Signed-off-by: Ed Santiago santiago@redhat.com

None

rhatdan · 2024-04-18T15:20:33Z

Thanks @edsantiago LGTM

packit-as-a-service · 2024-04-18T15:23:35Z

Ephemeral COPR build failed. @containers/packit-build please check.

edsantiago · 2024-04-18T16:41:06Z

Failures so far:

Swagger:

mkcomposefs: Invalid integer 628

int (all of them):

# podman [options] save -o /tmp/quay.io-libpod-cirros-latest.tar quay.io/libpod/cirros:latest
Copying blob sha256:b993cfcfd8fdc2731b308222fce8b5ce7a681780ab276b6a23e1b0fe3ce06bbb
Copying blob sha256:aa107a40759291c596cd432ca14cf15e781b91cf82121dc5a335eb734cafe1c4
Copying blob sha256:858d98ac489379d0c34c20811558091180c724d7d185693c3a601ba8c0a1cedf
Error: writing blob: archive/tar: write too long

sys remote:

# podman-remote --url unix:///tmp/podman_tmp_ijkO run -d --name myweb -p 5054:80 -v /tmp/podman_bats.CpsvZW/myimage-D4TDFZjD.tar:/var/www/image.tar:Z -w /var/www quay.io/libpod/testimage:20240123 /bin/busybox-extras httpd -f -p 80
Trying to pull quay.io/libpod/testimage:20240123...
...
Writing manifest to image destination
Error: committing the finished image: \
     failed to put layer using a partial pull: \
          rename /var/lib/containers/storage/overlay/staging/2508350620 /var/lib/containers/storage/overlay/SHA/diff: no such file or directory

Luap99 · 2024-04-18T16:54:19Z

Failures so far:

* [Swagger](https://cirrus-ci.com/task/5151327842467840):

mkcomposefs: Invalid integer 628

Tacked in #22422

Maybe file new issues for the other ones

edsantiago · 2024-04-18T17:08:42Z

mkcomposefs: Invalid integer 628

I'm like 90% sure that 628 is a valid integer, but I don't have enough fingers and toes to confirm.

edsantiago · 2024-06-11T12:39:13Z

Failing rootless:

Error: configure storage: composefs is not supported in user namespaces

This is with c-storage @ main as of this morning.

giuseppe · 2024-06-11T13:11:38Z

Failing rootless:
Error: configure storage: composefs is not supported in user namespaces
This is with c-storage @ main as of this morning.

that is expected. composefs doesn't work with rootless as it needs to mount EROFS

edsantiago · 2024-07-03T15:19:51Z

Vendoring in c-image, storage, common @ main yields:

CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build \
         \
        -ldflags '-X github.com/containers/podman/v5/libpod/define.gitCommit=cb98e769e6be7dc22662889580e49b41ae64817b -X github.com/containers/podman/v5/libpod/define.buildInfo=1720019889 -X github.com/containers/podman/v5/libpod/config._installPrefix=/usr/local -X github.com/containers/podman/v5/libpod/config._etcDir=/etc -X github.com/containers/podman/v5/pkg/systemd/quadlet._binDir=/home/esm/bin/x86_64 -X github.com/containers/common/pkg/config.additionalHelperBinariesDir= ' \
        -tags "   systemd libsubid exclude_graphdriver_devicemapper seccomp " \
        -o bin/podman ./cmd/podman
# github.com/containers/podman/v5/pkg/api/handlers/compat
pkg/api/handlers/compat/containers.go:394:16: cannot use struct{NetworkMode string `json:",omitempty"`}{…} (value of type struct{NetworkMode string "json:\",omitempty\""}) as struct{NetworkMode string "json:\",omitempty\""; Annotations map[string]string "json:\",omitempty\""} value in struct literal
pkg/api/handlers/compat/networks.go:201:24: cannot use networkCreate.EnableIPv6 (variable of type *bool) as bool value in assignment
pkg/api/handlers/compat/system.go:60:16: cannot use struct{NetworkMode string `json:",omitempty"`}{} (value of type struct{NetworkMode string "json:\",omitempty\""}) as struct{NetworkMode string "json:\",omitempty\""; Annotations map[string]string "json:\",omitempty\""} value in struct literal
make: *** [Makefile:380: bin/podman] Error 1

cc @mheon

Luap99 · 2024-07-03T15:27:48Z

That is the docker update: #23093

Run root e2e & system tests using composefs on rawhide. Write magic settings to storage.conf. That part is easy. e2e tests, however, ignore storage.conf. They require everything to be specified on the command line. And "everything", in the case of composefs, includes a long complicated --pull-options string which in turn requires containers-storage PR 1966 which, as of this writing, is finally vendored into podman. Signed-off-by: Ed Santiago <santiago@redhat.com>

edsantiago

This is ready for review, although I foresee (and welcome!) challenges to some of my decisions.

edsantiago · 2024-07-10T21:59:52Z

docs/source/markdown/podman.1.md

@@ -263,7 +263,7 @@ Set default `--storage-driver` value.

 #### **STORAGE_OPTS**

-Set default `--storage-opts` value.
+Set default `--storage-opt` value.


Completely unrelated to my PR, I'm just sneaking this in as a drive-by fix

edsantiago · 2024-07-10T22:02:05Z

test/e2e/common_test.go

+		// Look for STORAGE_OPTIONS_OVERLAY / STORAGE_OPTIONS_VFS
+		extraOptions := os.Getenv("STORAGE_OPTIONS_" + strings.ToUpper(storageFs))
+		if extraOptions != "" {
+			storageOptions += " --storage-opt " + extraOptions
+		}


New behavior. I think it makes sense to require separate envariables for each storage fs, but please shoot down if you disagree.

edsantiago · 2024-07-10T22:04:58Z

test/e2e/config_amd64.go

-	STORAGE_FS               = "overlay"                                                                                                                   //nolint:revive,stylecheck
-	STORAGE_OPTIONS          = "--storage-driver overlay"                                                                                                  //nolint:revive,stylecheck
-	ROOTLESS_STORAGE_FS      = "overlay"                                                                                                                   //nolint:revive,stylecheck
-	ROOTLESS_STORAGE_OPTIONS = "--storage-driver overlay"                                                                                                  //nolint:revive,stylecheck


ROOTLESS_STORAGE_OPTIONS was nuked in #18544. (The rest of this block is whitespace-only. Same with the next two config_xxx.go files).

Luap99

LGTM

giuseppe

/lgtm

openshift-ci · 2024-07-11T10:04:22Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: edsantiago, giuseppe, Luap99

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Luap99,edsantiago,giuseppe]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot added release-note-none approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Apr 18, 2024

edsantiago force-pushed the test-composefs branch from c59cb11 to 0abdfd7 Compare April 18, 2024 15:37

edsantiago mentioned this pull request Apr 18, 2024

composefs: podman save: tar: write too long containers/storage#1892

Closed

edsantiago force-pushed the test-composefs branch 4 times, most recently from 569e322 to d694396 Compare April 26, 2024 01:17

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 27, 2024

edsantiago force-pushed the test-composefs branch 2 times, most recently from 9b2045e to 1dfde92 Compare April 29, 2024 14:06

openshift-merge-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 29, 2024

edsantiago force-pushed the test-composefs branch from 1dfde92 to 7ad8591 Compare May 1, 2024 11:12

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 1, 2024

edsantiago force-pushed the test-composefs branch from 7ad8591 to 0de4b1d Compare May 2, 2024 10:56

edsantiago mentioned this pull request May 2, 2024

overlay: lock staging directories containers/storage#1916

Merged

edsantiago force-pushed the test-composefs branch from 0de4b1d to 0ae6ca6 Compare May 2, 2024 14:58

edsantiago added the bloat_approved Approve a PR in which binary file size grows by over 50k label May 2, 2024

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 10, 2024

edsantiago force-pushed the test-composefs branch from 0ae6ca6 to 870b3db Compare May 10, 2024 17:01

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 10, 2024

edsantiago force-pushed the test-composefs branch from 870b3db to 1f9d502 Compare May 12, 2024 20:52

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 13, 2024

edsantiago force-pushed the test-composefs branch from 1f9d502 to e2b064c Compare May 13, 2024 12:02

edsantiago force-pushed the test-composefs branch 2 times, most recently from 69f7e01 to b316833 Compare June 18, 2024 13:54

edsantiago force-pushed the test-composefs branch 2 times, most recently from 4c382de to 0360870 Compare June 18, 2024 14:55

edsantiago mentioned this pull request Jun 20, 2024

systests: kube: bump up a timeout #23059

Merged

edsantiago force-pushed the test-composefs branch from 0360870 to c24f2b3 Compare June 27, 2024 12:10

edsantiago force-pushed the test-composefs branch 4 times, most recently from 48f5e78 to 803f791 Compare July 10, 2024 19:44

edsantiago mentioned this pull request Jul 10, 2024

podman start --filter restart-policy=always : container state improper #23246

Closed

edsantiago changed the title ~~[DO NOT MERGE] test composefs~~ test composefs on rawhide Jul 10, 2024

edsantiago removed the bloat_approved Approve a PR in which binary file size grows by over 50k label Jul 10, 2024

edsantiago force-pushed the test-composefs branch from 803f791 to d4c0e7e Compare July 10, 2024 21:01

edsantiago commented Jul 10, 2024

View reviewed changes

Luap99 approved these changes Jul 11, 2024

View reviewed changes

giuseppe approved these changes Jul 11, 2024

View reviewed changes

openshift-ci bot assigned giuseppe Jul 11, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 11, 2024

openshift-merge-bot bot merged commit 7cc9b41 into containers:main Jul 11, 2024
89 checks passed

edsantiago deleted the test-composefs branch July 11, 2024 10:48

edsantiago mentioned this pull request Jul 22, 2024

podman-machine uncategorized flakes #22551

Open

stale-locking-app bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Oct 10, 2024

stale-locking-app bot locked as resolved and limited conversation to collaborators Oct 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test composefs on rawhide #22425

test composefs on rawhide #22425

edsantiago commented Apr 18, 2024 •

edited

Loading

rhatdan commented Apr 18, 2024

packit-as-a-service bot commented Apr 18, 2024

edsantiago commented Apr 18, 2024

Luap99 commented Apr 18, 2024

edsantiago commented Apr 18, 2024

edsantiago commented Jun 11, 2024

giuseppe commented Jun 11, 2024

edsantiago commented Jul 3, 2024

Luap99 commented Jul 3, 2024

edsantiago left a comment

edsantiago Jul 10, 2024

edsantiago Jul 10, 2024

edsantiago Jul 10, 2024

Luap99 left a comment

giuseppe left a comment

openshift-ci bot commented Jul 11, 2024

test composefs on rawhide #22425

test composefs on rawhide #22425

Conversation

edsantiago commented Apr 18, 2024 • edited Loading

rhatdan commented Apr 18, 2024

packit-as-a-service bot commented Apr 18, 2024

edsantiago commented Apr 18, 2024

Luap99 commented Apr 18, 2024

edsantiago commented Apr 18, 2024

edsantiago commented Jun 11, 2024

giuseppe commented Jun 11, 2024

edsantiago commented Jul 3, 2024

Luap99 commented Jul 3, 2024

edsantiago left a comment

Choose a reason for hiding this comment

edsantiago Jul 10, 2024

Choose a reason for hiding this comment

edsantiago Jul 10, 2024

Choose a reason for hiding this comment

edsantiago Jul 10, 2024

Choose a reason for hiding this comment

Luap99 left a comment

Choose a reason for hiding this comment

giuseppe left a comment

Choose a reason for hiding this comment

openshift-ci bot commented Jul 11, 2024

edsantiago commented Apr 18, 2024 •

edited

Loading