Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overlay: extend overlayFileGetter to understand composefs #1950

Merged

Conversation

giuseppe
Copy link
Member

@giuseppe giuseppe commented Jun 5, 2024

extend the overlayFileGetter interface to correctly read files from a composefs layer.

The overlayFileGetter creates a temporary mount of the composefs blob, grab a directory to its root and immediately unmount it so the mount is visible (and could be leaked) only for a short period.

Alternative to #1944

Marked as a Draft until Podman tests pass

Copy link
Contributor

openshift-ci bot commented Jun 5, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: giuseppe

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved label Jun 5, 2024
@mtrmac
Copy link
Collaborator

mtrmac commented Jun 5, 2024

I’m afraid this is not helpful, but just to avoid later surprises: Per #1899 (comment), I will not be approving PRs adding any more alternatives or indirections to the overlay driver.

That doesn’t at all mean that I think no-one should be working on those things, just that I can’t be helping.

@cgwalters
Copy link
Contributor

Seems fair; at a procedural level here: we're looking at using c/storage underneath bootc and I'm trying to spin up on the codebase more as I'm sure you've noticed. Let me know what would be the most helpful/useful way to do that. It'd be good to try to get to a place where have rough consensus and working code.

@giuseppe
Copy link
Member Author

giuseppe commented Jun 5, 2024

I’m afraid this is not helpful, but just to avoid later surprises: Per #1899 (comment), I will not be approving PRs adding any more alternatives or indirections to the overlay driver.

so where should it be done if the composefs implementation is in the overlay driver?

@mtrmac
Copy link
Collaborator

mtrmac commented Jun 5, 2024

I’m not saying this is the wrong place; just that I can’t keep up with the complexity of this subpackage as it is, and as it continues to grow.

@mtrmac
Copy link
Collaborator

mtrmac commented Jun 5, 2024

… thats “a me problem”, as the young kids say.

@giuseppe giuseppe marked this pull request as ready for review June 5, 2024 14:05
@giuseppe giuseppe force-pushed the overlay-getter-composefs branch from df2f61f to 28254bb Compare June 5, 2024 14:35
@rhatdan
Copy link
Member

rhatdan commented Jun 5, 2024

LGTM
@cgwalters @saschagrunert @nalind @alexlarsson PTAL

@giuseppe giuseppe force-pushed the overlay-getter-composefs branch from 28254bb to d48b0e8 Compare June 5, 2024 19:32
@giuseppe
Copy link
Member Author

giuseppe commented Jun 6, 2024

we need this for the composefs tests in the Podman CI

@edsantiago PTAL

Copy link
Contributor

@cgwalters cgwalters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know this code well at all, but this seems overall sane to me, FWIW.

Comment on lines +2032 to +2034
// there is no *at equivalent for getxattr, but it can be emulated by opening the file under /proc/self/fd/$FD/$PATH
len, err := unix.Getxattr(fmt.Sprintf("/proc/self/fd/%d/%s", int(f.Fd()), path), "trusted.overlay.redirect", buf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Getxattr probably wants at some point to be factored out to a helper function, we have a lot of copies of the /proc/self/fd/ trick in this codebase, and there is actually finally some movement on a proper syscall for lgetxattrat that we could change in one place instead of many.


On a different topic it feels a bit like we're poking into composefs internals here? One thing I've thought about in the past that would be nice perhaps is something like an ioctl() usable on an overlayfs that would reopen the file descriptor with the real lower file. Specifically, what I hit is I wanted to be able to reflink across an overlay view and a target in the real underlying filesystem. That's a case that it seems to me we could make safe and viable, and in that model the overlayfs kernel implementation would handle "chasing" this trusted.overlay.redirect.

@giuseppe giuseppe force-pushed the overlay-getter-composefs branch 2 times, most recently from 7a0f217 to c8f3004 Compare June 7, 2024 15:33
@cgwalters cgwalters requested a review from nalind June 7, 2024 17:08
extend the overlayFileGetter interface to correctly read files from a
composefs layer.

The overlayFileGetter creates a temporary mount of the composefs blob,
grab a directory to its root and immediately unmount it so the mount
is visible (and could be leaked) only for a short period.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@giuseppe giuseppe force-pushed the overlay-getter-composefs branch from c8f3004 to f1a8923 Compare June 7, 2024 19:30
giuseppe added a commit to giuseppe/libpod that referenced this pull request Jun 7, 2024
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
@rhatdan
Copy link
Member

rhatdan commented Jun 7, 2024

/lgtm
/hold
Wait for Podman tests to pass.

@giuseppe
Copy link
Member Author

giuseppe commented Jun 7, 2024

@openshift-merge-bot openshift-merge-bot bot merged commit 9ad69c8 into containers:main Jun 7, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants