Registry White Lists in registries.conf

[adambkaplan]

Enhance registries.conf to support explicit registry white-lists. If white-listing is utilized, only images from the listed repositories are allowed to be pulled, pushed, and run.

Ideally implement similar behavior that was supported in OpenShift 3.x - see OpenShift day two guide. Please note that registries.search should not be re-purposed to support white-list behavior.

[adambkaplan]

@runcom added here per Slack conversation.

[runcom]

@umohnani8 @QiWang19 PTAL

[vrothberg]

@runcom, I think we can apply a trick to make this work. The new prefix-based format of the sysregistriesv2 package can be extended to support wildcards. So we could have something as follows (format is wrong but the idea should be clear):

registry-A:
prefix=*
blocked=true

registry-B:
prefix=my-trusted-registry.com:5000

The prefix matching will make sure that either registry-B is used and that anything else will be blocked.

But this would just be one of many nice features when adding wildcard-support.

@rhatdan WDYT?

[vrothberg]

A complementary statement to the idea above. It will work as always the registry with the longest prefix match will be selected.

[rhatdan]

We already have this type of functionality in policy.json. Where we can reject all images and then only allow images from specific registries. @mtrmac correct?

[runcom]

@rhatdan yes, but that's playing with trust of a given registry, it's not a dummy block registry functionality, I'd rather not mix the two

[mtrmac]

Yes, policy.json can already be configured with default: reject, requiring every registry to be explicitly configured.

One thing to consider — I’m not at all sure about this, but just to bring it up:

A real whitelist works better when it is independent from any other configuration, and a single centralized configuration item (i.e. a new registry-whitelist entry in registries.conf) — so that the configuration management system has exactly one place where the whitelist is configured (though the system can of course assemble the value from several internal variables). That way, only people who explicitly want to (or, are allowed to), extend the whitelist, can extend the whitelist.

With a “default reject” + “anything explicitly configured is allowed” model, the whitelist is “distributed” over all the individual configurations; e.g., in the policy.json case, a person who only wants to record the known public key for an ISV more or less automatically enables the ISV’s registry to be accepted, because the configuration no longer falls through to the “default: reject” case. The same thing would happen with the prefix=* proposal, is someone just wanted to configure a registry as insecure.

Of course, the two models are semantically strictly equivalent (essentially, either there is a whitelist in the container configuration, and the CMS restricts who can modify the whitelist variable, or there is a whitelist in the CMS “meta-configuration”, and the CMS rejects any variable changes that don’t fit into the whitelist), and a separate centralized whitelist is somewhat likely to be redundant with other configuration (e.g. very likely to be redundant with policy.json on a locked-down system).

Which one is better, more or less, depends on what the configuration management system can do.

[Compare with the idea of (not) automatically opening firewall ports when a service is enabled — same centralize control vs. don’t repeat yourself dichotomy..]

[rhatdan]

My fear is having two ways of doing this is going to lead to confusion. Which one wins. If I reject everything in policy.json and I whitelist in registries.conf?

[runcom]

Trust and registry whitelisting are two separate and orthogonal thighs. Even if I whitelist a registry, it doesn't mean I trust images from that registry. Having people deal with trust as a mere mean to block certain registries is misleading and can lead to confusion as well.
The confusion can be made clear by well defining the purpose of both configuration, but using the trust one to configure blocking is misleading to me.

[rhatdan]

I guess you would have to define what "whitelisting" a registry means, I would think it means trusting the images at the registry.

[adambkaplan]

I think an analogy with firewalls and SSL certs is helpful here. You can use a firewall to block IPs and domains en masse, as well as use CA's to trust/distrust a particular website.

Ditto here - use the whitelist to only allow pull/push to specific registries, use policy.json to trust images based on cert/key.

[rhatdan]

Perhaps I am clueless, I don't see the difference.
Could someone give use cases based on podman commands where they would expect a difference.

[vrothberg]

Perhaps I am clueless, I don't see the difference.
Could someone give use cases based on podman commands where they would expect a difference.

A university environment is usually a good scenario. An admin may forbid accessing porn-container registries and trust all others but some internal staging registry.

[mtrmac]

The way I generally understand “whitelist”, it is a an alternative to a ”blacklist”, i.e. “somebody must explicitly put you on the whitelist for you to ‘pass’” vs. “anyone who is not explicitly added to the blacklist can ‘pass’ by default”.

That is orthogonal to what “‘pass’ the list check” means. A hypothetical {black,white}list could in principle be set up either an independent check to signature enforcement (i.e. an image must pass both), or as an alternative (an image only needs to pass one of them).

Only the independent check (must pass both) approach matches the way I think – it allows considering each of these mechanisms independently, rather than having to always consider (in principle arbitrary) interactions between many mechanisms. But that does not mean that a computer can’t implement anything else — even a JavaScript policy language able to call these various implementations at will (/me *shudder*s at the thought of polkit).

---

It would, ideally, be best to go back to the underlying requirements behind the Docker black/whitelist mechanisms, and the contexts in which they are used. We could well find out that what the users actually need is an Ansible “assertion” feature, or something like that, and nothing at a node level.

I guess we are stuck with providing a compatibility mechanism for the current implementation / layering design, though, and trying to accommodate all workflows that have accumulated since. In that case I prefer keeping the signature configuration and black/whitelist configurations orthogonal, to make reasoning about them much simpler.

[rhatdan]

Ok lets go with the example of university wants to block access to
porn.com

What does the admin do?
With policy.json he just executes

podman image trust -t reject porn.com

Now users can not pull from porn.com ( I guess this is a black list)

What do you see an admin doing with this new proposed "whitelist" where the admin wants to allow everything except porn.com?

[adambkaplan]

There is certainly a use case for OpenShift cluster admins - ex. only allow image pulls from the internal registry domain (which itself can act as a proxy to other registries).

[rhatdan]

@adambkaplan Wouldn't
podman image trust -t reject default
podman image trust -t accept openshift.registry.company.com
Work then?

[runcom]

I'm sure it would, I'm not against something easy, UI wise, like that. I'm just worried about using trust to block registries. The command above means "don't trust images from that registry, hence, avoid pulling them" whether we need something like "avoid pulling from that registry because I say so" even if the commands may overlap in practice

[rhatdan]

I don't care about the word trust. If the use case is I only want my users to pull from
openshift.registry.company.com and if they pull from Docker.io, I want it blocked then the policy.json file satisfies this need.

I don't see how adding something like whitelist/blacklist to registies.conf changes this?

If the goal is to allow user dwlash pull from anywhere while runcom can only pull from openshift.registry, then we have a different issue.

[bparees]

Here are the scenarios we need to solve. If they can be solved w/ some combination of policy.json and registries.conf today, great (but I agree with @rhatdan that having a whitelist in registries.conf and similar config in policy.json is confusing):

1. reject: all, allow: good.registry.com # disallow pulling/pushing for all registries except some set
2. reject: bad.registry.com, allow: all # disallow pulling/pushing for some specific registries, allow others
3. reject bad.registry.com, allow: good.registry.com # we need to define what happens for unlisted registries in this case?

if you guys can provide examples of what you'd expect registries.conf + policy.json to look like for those 3 scenarios (Assuming all 3 are possibly today), i think we will have what we need to define an openshift admin api for configuring this, and wiring it through builds down to podman.

@sjenning i think you guys are going to need to worry about this too. (whitelist/blacklisting registries so nodes can/can't pull from them)

[vrothberg]

Thanks, @bparees, for adding those use cases! Scenario 2 is already working today by adding bad.registry.com to the list of blocked registries in the registries.conf.

In my opinion, those things should be configured in the registries.conf as I find it more user-friendly to have one config file for all/most things related to registries. Such behaviour could be achieved by adding a new config switch (e.g., default_policy="reject_all"). @mtrmac WDYT?

[bparees]

In my opinion, those things should be configured in the registries.conf as I find it more user-friendly to have one config file for all/most things related to registries.

I agree.

Such behaviour could be achieved by adding a new config switch

how do i implement scenario (1) in registries.conf? where do i list the "allowed" registries in registries.conf?

[vrothberg]

how do i implement scenario (1) in registries.conf? where do i list the "allowed" registries in registries.conf?

That doesn't work with registries.conf at the moment. If we want to support that we need an additional mechanism to configure the default_policy.

[bparees]

That doesn't work with registries.conf at the moment. If we want to support that we need an additional mechanism to configure the default_policy.

even assuming we could configure the default_policy, what does the rest of the configuration look like?

[rhatdan]

I think we need to look at long term and short term. In the short term we can handle use case 1 and 2 in policy.conf

Reject everything except registries.access.redhat.com

reject: all, allow: good.registry.com # disallow pulling/pushing for all registries except some set

more /etc/containers/policy.json 
{
    "default": [
        {
            "type": "reject"
        }
    ],
    "transports": {
        "docker": {
            "registries.access.redhat.com": [
                {
                    "type": "insecureAcceptAnything"
                }
            ]
        }
    }
}

[rhatdan]

reject: bad.registry.com, allow: all # disallow pulling/pushing for some specific registries, allow others

# cat /etc/containers/policy.json
{
    "default": [
        {
            "type": "insecureAcceptAnything"
        }
    ],
    "transports": {
        "docker": {
            "bad.registry.com": [
                {
                    "type": "reject"
                }
            ]
        }
    }
}

[bparees]

thanks @rhatdan. So that syntax is supported today?

[rhatdan]

Yes

[rhatdan]

podman image trust
makes it easy to play with.

[rhatdan]

The third test is just depends on what you set for default.

[bparees]

{
    "default": [
        {
            "type": "insecureAcceptAnything"
        }
    ],
    "transports": {
        "docker": {
            "bad.registry.com": [
                {
                    "type": "reject"
                }
            ]
        }
    }
}

Doesn't this also mean we'd treat all registries as insecure (ie allow invalid TLS certs + non-TLS transport?)

[rhatdan]

I think that it is handled in registries.conf.

[bparees]

I guess i am confused by "type": "insecureAcceptAnything", that implies to me that it's going to accept insecure registries.

[rhatdan]

Basically it means, it does not require signatures. We have a third option which would require signed images.

[bparees]

ok, so not related to TLS/transport.

[bparees]

@adambkaplan so it sounds like we should proceed in the following manner:

1. wire search registries -> registries.conf
2. wire insecure registries -> registries.conf
3. wire allowed/blocked registries -> policy.json in the way that @rhatdan outlined above.

sound good?

[adambkaplan]

@bparees sounds good to me.

@rhatdan can policy.json be encoded via a go type? I assume yes.

[rhatdan]

It is just simple json, so I see not reason why not. Containers/image might do something with that now. @vrothberg Do you know?

[nalind]

@adambkaplan it should be a github.com/containers/image/signature.Policy. The package's tests include an example.

[adambkaplan]

Closing this issue, as policy.json is the better path forward for this capability.

[mtrmac]

Such behaviour could be achieved by adding a new config switch (e.g., default_policy="reject_all"). @mtrmac WDYT?

What happens when policy.json contains a different default? It’s much simpler when such an ambiguity can’t happen and we don’t have to answer that question.

[mtrmac]

3. wire allowed/blocked registries -> policy.json in the way that @rhatdan outlined above.

If you want to do it this way, be very careful about pre-existing data. policy.json is an OS-wide config file, not in general something that is free for OpenShift to just overwrite (unless OpenShift “owns” the whole OS, of course).

It would be extremely bad to replace a type: signedBy policy entry (or, really, any pre-existing entry) with a type: insecureAcceptAnything one. And make sure to handle the case of a >1-entry array defined for a scope.

Also, I think you’ll want to express the “block all non-whitelisted registries” in "transports"."docker"."" (docker://-specific default), not in the global default (which would also impact references to images in local storage, dir: and everything else).

@adambkaplan it should be a github.com/containers/image/signature.Policy. The package's tests include an example.

The API of that package is, right now, focused on evaluating the policy, and in programmatically creating the data structure, but quite a few of the underlying config fields are private. I.e. you can create a policy and/or parse it from JSON, and then either marshal it to JSON or evaluate it. OTOH if you want to load the policy and then make decisions based on its contents, you might find that the necessary data is private. (This was originally motivated a) by hoping to avoid making the less stable parts of the data a public API, and b) by … “encouraging” consumers to use the provided policy implementation instead of writing a parallel one.)

We might need to relax that, or *shrug* maybe use the other formulation recently added to libpod/pkg/trust (careful, not sure that it’s complete) which defined its own types instead of relaxing the c/image/signature ones (… at which point keeping the c/image/signature types private does not have that much of a point any more.)

[bparees]

If you want to do it this way, be very careful about pre-existing data. policy.json is an OS-wide config file, not in general something that is free for OpenShift to just overwrite (unless OpenShift “owns” the whole OS, of course).

we're doing this inside a pod that has no prior settings, we own it end to end.