Pushing an image index

[sjackman]

Does ORAS support pushing an image index application/vnd.oci.image.index.v1+json that contains multiple application/vnd.oci.image.manifest.v1+json?
See https://github.com/opencontainers/image-spec/blob/master/media-types.md#oci-image-media-types
and https://github.com/opencontainers/image-spec/blob/master/image-index.md

[sjackman]

• skopeo copy -a source dest works for OCI images and creates application/vnd.oci.image.index.v1+json https://github.com/containers/skopeo
• docker buildx imagetools create --tag dest src1 src2 works for OCI images and creates application/vnd.docker.distribution.manifest.list.v2+json

A solution for ORAS would be nice. Since my artifacts are tarballs, I'm just pretending they're Docker layers, which works.

[SteveLasker]

We were initially planning on adding support for oci.index, if it added a oci.index.config object to understand which type of artifact the index represented.
As index doesn't actually support blobs, as it's really an index to other manifests that have content, we're looking at other means to store content, with a collection of references.
Please see:

• OCI artifact manifest #29
• WIP generic object spec #37

These are the current artifact reference types we're hoping to evolve.

Can you help with a bit more details for what you're looking to generate to understand where it would fit in?

[sjackman]

Homebrew https://brew.sh is a package manager that supports both macOS and Linux. We have binary packages (called bottles), which are tarballs, for multiple versions of macOS and one universal Linux bottle that works on all distributions. We store each bottle in an ORAS artifact in an image manifest. We bundle these image manifests up into a single image index. We use the .manifests[].platform object, which includes architecture, variant, os, and os.version to select which bottle to download.
See https://github.com/opencontainers/image-spec/blob/master/image-index.md#image-index-property-descriptions

You can see examples of these ORAS image indexes at https://github.com/orgs/brewsci/packages and https://github.com/orgs/homebrew/packages.

I now use skopeo to upload the artifacts and either curl or oras or skopeo to download the artifacts.

You can close this issue now, but I'm happy to answer follow up questions if you think this would be an interesting use case for ORAS.

[deitch]

Homebrew stores its bottles as artifacts in an OCI registry? I didn't know that; been using it since earliest days, must be close to a decade now, switched over from, was it "macports"?. Which registry do you use, @sjackman?

@SteveLasker I think @sjackman is looking for something very similar to how container images work. One root index, with manifests property, which is a list of manifests, tagged by target platform (including OS version). Except that the actual manifests are not container images (application/vnd.oci.image.manifest.v1+json), nor is the root a container index (application/vnd.oci.image.index.v1+json), but rather they are appropriate types.

This seems like the right use case for oras.

[sjackman]

Homebrew stores its bottles as artifacts in an OCI registry?

As of this week!

Which registry do you use, @sjackman?

GitHub Packages, also known as GitHub Container Registry https://ghcr.io
See https://github.com/orgs/homebrew/packages

Except that the actual manifests are not container images (application/vnd.oci.image.manifest.v1+json), nor is the root a container index (application/vnd.oci.image.index.v1+json), but rather they are appropriate types.

We actually do use the media types application/vnd.oci.image.index.v1+json and application/vnd.oci.image.manifest.v1+json and even application/vnd.oci.image.layer.v1.tar+gzip for compatibility with oras, skopeo, even docker.

[sjackman]

For example https://github.com/orgs/Homebrew/packages/container/package/core/hello

$ oras pull ghcr.io/homebrew/core/hello:2.10
Downloaded 449de5ea35d0 hello-2.10.catalina.bottle.tar.gz
Downloaded b3b083db0807 hello-2.10.arm64_big_sur.bottle.tar.gz
Downloaded 54ac46b692fc hello-2.10.el_capitan.bottle.tar.gz
Downloaded 69489ae397e4 hello-2.10.big_sur.bottle.tar.gz
Downloaded f9d6285eafa4 hello-2.10.mojave.bottle.tar.gz
Downloaded 1b66790d4266 hello-2.10.high_sierra.bottle.tar.gz
Pulled ghcr.io/homebrew/core/hello:2.10
Digest: sha256:4f1d476de4e32f4a70a46df406fb917a0f1d990dcefd29be0a16c282a0a285d5
$ skopeo copy docker://ghcr.io/homebrew/core/hello:2.10 oci:hello:2.10
Getting image source signatures
Copying blob 449de5ea35d0 [--------------------------------------] 0.0b / 0.0b
Copying config bd012697f6 done  
Writing manifest to image destination
Storing signatures
$ docker pull --platform=darwin/amd64 ghcr.io/homebrew/core/hello:2.10
2.10: Pulling from homebrew/core/hello
operating system is not supported
$ docker pull ghcr.io/linuxbrew/core/hello:2.10
2.10: Pulling from linuxbrew/core/hello
f81d7c0a3eee: Pull complete 
Digest: sha256:b36a2cb7220504776fc540b67e9fe93947b97635d40fd98c62f27970be16d9ff
Status: Downloaded newer image for ghcr.io/linuxbrew/core/hello:2.10
ghcr.io/linuxbrew/core/hello:2.10

[deitch]

We actually do use the media types application/vnd.oci.image.index.v1+json and application/vnd.oci.image.manifest.v1+json and even application/vnd.oci.image.layer.v1.tar+gzip for compatibility with oras, skopeo, even docker

Then what is missing from oras for your support? It looks pretty solid.

[deitch]

For example https://github.com/orgs/Homebrew/packages/container/package/core/hello

I like that. I am a big advocate for using registries as a universal distribution mechanism. The combination of layers, manifests with annotations and stacking, and hashes (especially hashes), plus natively-attached config to each manifest, makes for a very powerful system.

I have my own tool for doing the same, mostly to inspect and pull down images.

[sjackman]

Then what is missing from oras for your support? It looks pretty solid.

Nothing. Using oras for download works out of the box, and that's great. I don't believe oras supports pushing a multi-architecture image index application/vnd.oci.image.index.v1+json. We're using skopeo for upload, and that solution is working just fine for us.

You can close this issue now, but I'm happy to answer follow up questions if you think this would be an interesting use case for ORAS.

[sagikazarmark]

This might be an interesting use case for pushing (and pulling) multi-arch binaries (multiple binaries for different architectures), but I'm not sure it's within the scope of oras.

[SteveLasker]

multi-arch binaries.
Now, that's an interesting angle I hadn't thought of. Since these are new artifacts, we'll need to think through whether the oci.index is the best solution, or possibly one of the oci artifact manifest proposals

• OCI artifact manifest, Phase 1-Reference Types opencontainers/artifacts#29
• WIP generic object spec opencontainers/artifacts#37

@justincormack for thoughts around multi-arch for other artifactTypes.

[sagikazarmark]

The upside of using image index is that it works with skopeo and other tools. It doesn't have to be a requirement though, if there are alternatives.

I'm not 100% sure that oras needs to be the tool that supports multi-arch artifacts as it tries to a be a generic tool as far as I can tell. But it could be, I don't know.

I think this is a topic that's worth standardizing though.

[SteveLasker]

I wouldn't say ORAS is trying to be the container image tool, as there are great tools for that. Rather, it's the non-container image tool for utilizing registries, without the constraints of the container image specific artifact type.
That said, the multi-arch pivot was the eye-opener (duhh moment).

[deitch]

Rather, it's the non-container image tool for utilizing registries, without the constraints of the container image specific artifact type

Yes, that is exactly how I look at it.

[volo-droid]

@sjackman do you have an idea why docker pull complains with "operating system is not supported"?

$ docker pull --platform=darwin/amd64 ghcr.io/homebrew/core/hello:2.10
2.10: Pulling from homebrew/core/hello
operating system is not supported

[SteveLasker]

Hi @volo-droid , You're pulling with the docker client. Since this isn't a container runtime, rather a homebrew package, you'll need to use the specific client. https://github.com/homebrew/homebrew-core/pkgs/container/core%2Fhello#how-do-i-install-these-formulae

[sjackman]

$ docker pull --platform=darwin/amd64 ghcr.io/homebrew/core/hello:2.10
2.10: Pulling from homebrew/core/hello
operating system is not supported
$ docker pull --platform=linux/amd64 ghcr.io/homebrew/core/hello:2.10
2.10: Pulling from homebrew/core/hello
e6980196298e: Pull complete 

The Docker client does not support pulling images for darwin, even on macOS. It can pull images for linux though.

[AaronFriel]

I'm curious what the status of this & the status of this item of the artifact authors guidance:

Future Scope

Future versions will support new artifact types representing collections of artifacts using OCI > Index. A means to identify an index as a type of artifact will be required.

This section is absent from the artifact specification.

There seem to be two distinct philosophies:

• Use an index (or nested indices) to define a DAG to multiple artifacts. This appears to be unsupported by the oras CLI and Go SDK.
• Use referrers to relate artifacts to one another, discoverable through the /referrers API.

It's unclear to me what the tradeoffs are, or what the "future-looking" perspective here would be.

[qweeah]

new artifact types representing collections of artifacts using OCI > Index. A means to identify an index as a type of artifact will be required.

@AaronFriel The above use case is already supported in oras-go v2.0.0-rc.4 and oras cli 0.16.0 .

When pushing an OCI artifact with non-nil subject field, Referrer API will be tried first, if it's not supported, fallback to pushing an OCI index with tag schema.

This section is absent from the artifact specification.

I think it's covered in distribution spec but not reflected in image specification.

cc @Wwwsylvia @FeynmanZhou @shizhMSFT

[AaronFriel]

That's very interesting and helpful. I must have missed that in the docs that two strategies would be utilized.

It does leave me with a few more questions about how we'd go about filtering artifacts by platform/os/etc, but I suspect we can use annotations in our client to distinguish these.

[qweeah]

@AaronFriel Feel free to elaborate your question in oras discussion. Maybe the feature you need is already supported in the current version. Sorry for the lacking of documentation but we are planning to add user guide after 1.0.0 release.

[FeynmanZhou]

@AaronFriel ORAS has fully supported the OCI artifact manifest in the latest release of CLI and ORAS Go SDK. We will create some documentation to explain it on the ORAS website.

It does leave me with a few more questions about how we'd go about filtering artifacts by platform/os/etc, but I suspect we can use annotations in our client to distinguish these.

I think adding annotations is also a workaround to distinguish them.

May I know your requirements for ORAS?

[AaronFriel]

Yes, I think tentatively I'm looking at storing a tree like so:

userName/packageFoo:latest (index/manifest or artifact?)
├── artifact: schema.json
├── plugin (index/manifest?)
│   ├── artifact: plugin-darwin-amd64.tar.gz
│   │   └── artifact: signature
│   ├── artifact: plugin-linux-amd64.tar.gz
│   └── ...
└── artifact: sbom
    └── artifact: signature

To implement plugin discovery for the Pulumi engine. This is all in the prototyping/design stage at the moment, and it seems like ORAS is an ideal mechanism for us.

The root tag userName/packageFoo:latest would be an entrypoint for discovering the related artifacts - it sounds like depending on the OCI registry's capabilities this will occur through either a manifest or via references.

[justincormack]

@sjackman ran into this - your indexes you are pushing are missing the MediaType field, so they are being processed as manifests by the Docker Hub pipeline, not indexes.

[shizhMSFT]

Closing since it is now supported by oras manifest push.

Note oras manifest push pushes the manifest file as it is.

[LinuxSuRen]

hi @sjackman I'm wondering if you have any document or notes to describe how to use skopeo uploading a multi-arch artifacts to image registry. I have gone through all the reference, but didn't find it. Thanks for your time.