oras pull with cache

[shizhMSFT]

Resolves #193. The cache is enabled only when the environment variable ORAS_CACHE is set in the CLI.

Example usage:

# Set cache root
export ORAS_CACHE=~/.oras/cache

# Pull artifacts as usual
oras pull localhost:5000/hello:latest

[sajayantony]

Could you please show the layout of the cache?

[shizhMSFT]

@sajayantony The cache uses github.com/containerd/containerd/content/local as its content store.

The layout of the cache is

$ORAS_CACHE
├── blobs
│   └── <algorithm>
│       └── <digest>
└── ingest
    └── <ref_digest>
        ├── data
        ├── ref
        ├── startedat
        ├── total
        └── updatedat

Note that the ingest folder is used only when downloading. After downloading, the ingest folder should be empty.

[jdolitsky]

Hey, this is great. Does it make sense to include index.json (as in OCI Layout?)

We have OCIStore to manage this file: https://github.com/deislabs/oras/blob/main/pkg/content/oci.go

Can we combine that with the containerd/local cache?

[shizhMSFT]

@jdolitsky The cache store here must be a pure content-addressable storage (i.e. no refs) so that the cache can be used by multiple oras clients concurrently with no race conditions.

Besides, we are actually caching at the blob level not images. Therefore, it does not make sense to add an index.

[SteveLasker]

Josh, are you asking if we can store the manifests? Both image-index and image-manifest, so they are cached and available to compute digests?

[jdolitsky]

I'm thinking of Helm here. Had this type of cache (combined with index.json support) had existed, we would not have needed to implemented.. I understand an "oras" pull of reference when working with files does not require saving of the reference (e.g. localhost:5000/hello:latest).

This has value as general purpose local CAS, will approve. Later if needed we can make a hybrid store from this and OCIStore. @shizhMSFT merge when ready