containers: support derivation

[cgwalters]

We may in the future support the equivalent of a Dockerfile that does

FROM quay.io/exampleos/exampleos:latest
RUN curl ...
ADD ...
RUN ostree container-finalize-bootable

What this will do is basically an ostree-tar version of ostree commit --tree=dir=/ --selinux-policy-from-base that will do e.g. selinux labeling, hardlink deduplication and computing the sha256 digest.

In theory, we don't require this "blessing" of ostree container-finalize-bootable - we could do it client side too. But I think it's better for the client side to be "dumb" by default - it makes it more like an image system.

Use/integration with rpm-ostree and higher level projects

This project is independent of rpm-ostree and other tools. However we need to support them; specifically e.g.:

FROM quay.io/coreos/fedora:stable
RUN rpm-ostree install cowsay
RUN ostree container-finalize-bootable

Following the above. And so our library/model should also support what rpm-ostree natively wants to do in unpacking rpms into ostree commits for example.

Ultimately, this scope creeps basically into a new ostree repo mode container-tar or so?

[cgwalters]

If we want to support Dockerfile then it basically means we need to understand container layer deltas.

[cgwalters]

Thinking through this more, it seems to me that derivation support does scope creep into mapping container image layers into ostree commits, and handling GC. Basically to gain the real benefit, we need to avoid re-pulling layers we already have.

This isn't hard at all, and similar code exists in containers/storage's ostree backend. But it is definitely a scope increase.

[cgwalters]

First, you need the code from coreos/rpm-ostree#3139 and most notably #99 plus https://github.com/cgwalters/container-image-proxy stuck in the image too.

These instructions have been tested on current FCOS stable 34.20210904.3.0. But note that the syntax for rebasing to containers has changed in the newer rpm-ostree v2021.11 (with newer ostree-rs-ext) which is included in this test container I've uploaded. Anyways you can copy and paste these commands:

# systemctl mask --now zincati
# rpm-ostree rebase --experimental docker://quay.io/cgwalters/fcos-dev:latest
...
# systemctl reboot

Now from there, try creating a derived image! You could do it on the same machine, but I think it's much clearer to use a separate build system, which could be the quay.io buildsystem or e.g. an OpenShift cluster or whatever.

There's an example here https://github.com/cgwalters/fcos-derivation-example

Right now you can easily e.g. add files - notably things like systemd units. What doesn't work yet is yum install...there's some nontrivial work to do on the rpm-ostree side for that, but you can do rpm -Uvh.

Anyways, once you have your image uploaded somewhere, try rebasing to it:

# rpm-ostree rebase --experimental ostree-unverified-image:docker://quay.io/cgwalters/fcos-derivation-example:latest

From there you can apply via e.g. systemctl reboot or alternatively try rpm-ostree ex apply-live!

[cgwalters]

A big looming problem with this whole thing is handling /etc/passwd. This touches on coreos/rpm-ostree#49

Basically do we support people doing e.g. useradd foo; chown foo /etc/someconfig type stuff in Dockerfile?

[mkenigs]

Would a Dockerfile be used or some other format? Just thinking about making MCO/MCBS more declarative and wondering how this would fit in.

[cgwalters]

See https://github.com/coreos/enhancements/pull/7/files#diff-bc7b9e656253160761d8ef791b3e206a1264197112e45916edfa9344e97359e7R37

[cgwalters]

But to clarify, this proposal does not require Dockerfile- any tool which can build container images should work.

[cgwalters]

The initial version of this is implemented in the 0.4.0 release. Closing this issue since we have a working PoC - a lot more remains to be done obviously, but will make a followup tracker issue.