UPSTREAM: <carry>: openshift-kube-apiserver: add kube-apiserver patches

Origin-commit: 170dd7d25cca990fd7683eaf424d00bcd776c39c Origin-commit: 35ef039cb099dc609c576cf594aadd849212a00b UPSTREAM: <carry>: openshift-kube-apiserver: enabled conversion gen for admission configs UPSTREAM: <carry>: openshift-kube-apiserver/admission: fix featuregates resource name UPSTREAM: <carry>: openshift-kube-apiserver/admission: add missing FeatureSets UPSTREAM: <carry>: openshift-kube-apiserver: use github.com/openshift/apiserver-library-go/pkg/labelselector UPSTREAM: <carry>: openshift authenticator: don't allow old-style tokens UPSTREAM: <carry>: oauth-authn: support sha256 prefixed tokens UPSTREAM: <carry>: oauth-token-authn: switch to sha256~ prefix UPSTREAM: <carry>: oauth-token-authn: add sha256~ support to bootstrap authenticator UPSTREAM: <drop>: remove the openshift authenticator from the apiserver In 4.8, we moved the authenticator to be configured via webhookTokenAuthenticators to an endpoint in the oauth-apiserver, this should now be safe to remove. UPSTREAM: <carry>: set ResourceQuotaValidationOptions to true When PodAffinityNamespaceSelector goes to beta or GA this might affect how our ClusterResourceQuota might work UPSTREAM: <carry>: simplify the authorizer patch to allow the flags to function Origin-commit: 0d7fb2d769d631054ec9ac0721aee623c96c1001 UPSTREAM: <carry>: eliminate unnecessary closure in openshift configuration wiring Origin-commit: 3b0c72dd7b9f9367dda8f8645909d9277a6c29e9 openshift-rebase(v1.24):source=78e37fdfb28 UPSTREAM: <carry>: add crdvalidation for apiserver.spec.tlsSecurityProfile Origin-commit: 84ba7fc304870a30df7136da14bccb4d5232f075 openshift-rebase(v1.24):source=eecae1591a1 UPSTREAM: <carry>: openshift-kube-apiserver: Add custom resource validation for network spec openshift-rebase(v1.24):source=2af991c43b1 UPSTREAM: <carry>: stop overriding flags that are explicitly set openshift-rebase(v1.24):source=8355d726bbf UPSTREAM: <carry>: add readyz check for openshift apiserver availability openshift-rebase(v1.24):source=3784942f6fc UPSTREAM: <carry>: wait for oauth-apiserver accessibility openshift-rebase(v1.24):source=0c175222685 UPSTREAM: <carry>: provide a new admission plugin to mutate management pods CPUs requests The ManagementCPUOverride admission plugin replaces pod container CPU requests with a new management resource. It applies to all pods that: 1. are in an allowed namespace 2. and have the workload annotation. It also sets the new management resource request and limit and set resource annotation that CRI-O can recognize and apply the relevant changes. For more information, see - openshift/enhancements#703 Conditions for CPUs requests deletion: 1. The namespace should have allowed annotation "workload.openshift.io/allowed": "management" 2. The pod should have management annotation: "workload.openshift.io/management": "{"effect": "PreferredDuringScheduling"}" 3. All nodes under the cluster should have new management resource - "management.workload.openshift.io/cores" 4. The CPU request deletion will not change the pod QoS class Signed-off-by: Artyom Lukianov <alukiano@redhat.com> UPSTREAM: <carry>: Does not prevent pod creation because of no nodes reason when it runs under the regular cluster Check the `cluster` infrastructure resource status to be sure that we run on top of a SNO cluster and in case if the pod runs on top of regular cluster, exit before node existence check. Signed-off-by: Artyom Lukianov <alukiano@redhat.com> UPSTREAM: <carry>: do not mutate pods when it has a container with both CPU request and limit Removing the CPU request from the container that has a CPU limit will result in the defaulter to set the CPU request back equals to the CPU limit. Signed-off-by: Artyom Lukianov <alukiano@redhat.com> UPSTREAM: <carry>: Reject the pod creation when we can not decide the cluster type It is possible a race condition between pod creation and the update of the infrastructure resource status with correct values under Status.ControlPlaneTopology and Status.InfrastructureTopology. Signed-off-by: Artyom Lukianov <alukiano@redhat.com> openshift-rebase(v1.24):source=ea874aa684f UPSTREAM: <carry>: add CRD validation for dnses Add an admission plugin that validates the dnses.operator.openshift.io custom resource. For now, the plugin only validates the DNS pod node-placement parameters. This commit fixes bug 1967745. https://bugzilla.redhat.com/show_bug.cgi?id=1967745 * openshift-kube-apiserver/admission/customresourcevalidation/attributes.go (init): Install operatorv1 into supportedObjectsScheme. * openshift-kube-apiserver/admission/customresourcevalidation/customresourcevalidationregistration/cr_validation_registration.go (AllCustomResourceValidators, RegisterCustomResourceValidation): Register the new plugin. * openshift-kube-apiserver/admission/customresourcevalidation/dns/validate_dns.go: New file. (PluginName): New const. (Register): New function. Register the plugin. (toDNSV1): New function. Convert a runtime object to a versioned DNS. (dnsV1): New type to represent a runtime object that is validated as a versioned DNS. (ValidateCreate, ValidateUpdate, ValidateStatusUpdate): New methods. Implement the ObjectValidator interface, using the validateDNSSpecCreate and validateDNSSpecUpdate helpers. (validateDNSSpecCreate, validateDNSSpecUpdate): New functions. Validate a DNS, using the validateDNSSpec helper. (validateDNSSpec): New function. Validate the spec field of a DNS, using the validateDNSNodePlacement helper. (validateDNSNodePlacement): New function. Validate the node selector and tolerations in a DNS's node-placement parameters, using validateTolerations. (validateTolerations): New function. Validate a slice of corev1.Toleration. * openshift-kube-apiserver/admission/customresourcevalidation/dns/validate_dns_test.go: New file. (TestFailValidateDNSSpec): Verify that validateDNSSpec rejects invalid DNS specs. (TestSucceedValidateDNSSpec): Verify that validateDNSSpec accepts valid DNS specs. * vendor/*: Regenerate. openshift-rebase(v1.24):source=bd9a55803db UPSTREAM: <carry>: prevent the kubecontrollermanager service-ca from getting less secure openshift-rebase(v1.24):source=cc96bfa11af UPSTREAM: <carry>: allow SCC to be disabled on a per-namespace basis openshift-rebase(v1.24):source=6f2d9a525bc UPSTREAM: <carry>: verify required http2 cipher suites In the Apiserver admission, we need to return an error if the required http2 cipher suites are missing from a custom tlsSecurityProfile. Currently, custom cipher suites missing ECDHE_RSA_WITH_AES_128_GCM_SHA256 or ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 result in invalid http2 Server configuration causing the apiservers to crash. See: go/x/net/http2.ConfigureServer for futher information. Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com> openshift-rebase(v1.24):source=e2fb8191644 UPSTREAM: <carry>: drop the warning to use --keep-annotations When a user runs the `oc debug` command for the pod with the management resource, we will inform him that he should pass `--keep-annotations` parameter to the debug command. Signed-off-by: Artyom Lukianov <alukiano@redhat.com> openshift-rebase(v1.24):source=9726268c979 UPSTREAM: <carry>: admission/managementcpusoverride: cover the roll-back case During the upgrade and roll-back flow 4.7->4.8->4.7, the topology related fields under the infrastructure can be empty because the old API does not support them. The code will equal the empty infrastructure section with the current one. When the status has some other non-empty field, and topology fields are empty, we assume that the cluster currently passes via roll-back and not via the clean install. Signed-off-by: Artyom Lukianov <alukiano@redhat.com> openshift-rebase(v1.24):source=35a93248f51 UPSTREAM: <carry>: Remove pod warning annotation when workload partitioning is disabled openshift-rebase(v1.24):source=67e1c1dac22 UPSTREAM: <carry>: use new access token inactivity timeout field. openshift-rebase(v1.24):source=79be14211df UPSTREAM: <carry>: apirequestcount validation openshift-rebase(v1.24):source=5fbf4195cbb UPSTREAM: <carry>: Added config node object validation for extreme latency profiles Signed-off-by: Swarup Ghosh <swghosh@redhat.com> UPSTREAM: <carry>: Add Upstream validation in the DNS admission check patches UPSTREAM: <carry>: Make RestrictedEndpointsAdmission check NotReadyAddresses UPSTREAM: <carry>: Make RestrictedEndpointsAdmission restrict EndpointSlices as well openshift-rebase(v1.24):source=819a64c501a
openshift · Sep 20, 2022 · 932411e · 932411e
1 parent bcf574c
commit 932411e
Show file tree

Hide file tree

Showing 162 changed files with 16,879 additions and 52 deletions.
diff --git a/cmd/kube-apiserver/app/patch_openshift.go b/cmd/kube-apiserver/app/patch_openshift.go
diff --git a/cmd/kube-apiserver/app/server.go b/cmd/kube-apiserver/app/server.go
@@ -29,7 +29,7 @@ import (
 	"strings"
 	"time"
 
-	"k8s.io/kubernetes/openshift-kube-apiserver/configdefault"
+	"k8s.io/kubernetes/openshift-kube-apiserver/admission/admissionenablement"
 	"k8s.io/kubernetes/openshift-kube-apiserver/enablement"
 	"k8s.io/kubernetes/openshift-kube-apiserver/openshiftkubeapiserver"
 
@@ -123,28 +123,32 @@ cluster's shared state through which all other components interact.`,
 			cliflag.PrintFlags(fs)
 
 			if len(s.OpenShiftConfig) > 0 {
-				enablement.ForceOpenShift()
+				// if we are running openshift, we modify the admission chain defaults accordingly
+				admissionenablement.InstallOpenShiftAdmissionPlugins(s)
+
 				openshiftConfig, err := enablement.GetOpenshiftConfig(s.OpenShiftConfig)
 				if err != nil {
 					klog.Fatal(err)
 				}
-
-				// this forces a patch to be called
-				// TODO we're going to try to remove bits of the patching.
-				configPatchFn, serverPatchContext := openshiftkubeapiserver.NewOpenShiftKubeAPIServerConfigPatch(genericapiserver.NewEmptyDelegate(), openshiftConfig)
-				OpenShiftKubeAPIServerConfigPatch = configPatchFn
-				OpenShiftKubeAPIServerServerPatch = serverPatchContext.PatchServer
+				enablement.ForceOpenShift(openshiftConfig)
 
 				args, err := openshiftkubeapiserver.ConfigToFlags(openshiftConfig)
 				if err != nil {
 					return err
 				}
+
 				// hopefully this resets the flags?
 				if err := cmd.ParseFlags(args); err != nil {
 					return err
 				}
 
-				enablement.ForceGlobalInitializationForOpenShift(s)
+				// print merged flags (merged from OpenshiftConfig)
+				cliflag.PrintFlags(cmd.Flags())
+
+				enablement.ForceGlobalInitializationForOpenShift()
+			} else {
+				// print default flags
+				cliflag.PrintFlags(cmd.Flags())
 			}
 
 			// set default options
@@ -230,10 +234,6 @@ func CreateServerChain(completedOptions completedServerRunOptions) (*aggregatora
 		return nil, err
 	}
 
-	if err := PatchKubeAPIServerServer(kubeAPIServer); err != nil {
-		return nil, err
-	}
-
 	// aggregator comes last in the chain
 	aggregatorConfig, err := createAggregatorConfig(*kubeAPIServerConfig.GenericConfig, completedOptions.ServerRunOptions, kubeAPIServerConfig.ExtraConfig.VersionedInformers, serviceResolver, kubeAPIServerConfig.ExtraConfig.ProxyTransport, pluginInitializer)
 	if err != nil {
@@ -472,6 +472,8 @@ func buildGenericConfig(
 	// on a fast local network
 	genericConfig.LoopbackClientConfig.DisableCompression = true
 
+	enablement.SetLoopbackClientConfig(genericConfig.LoopbackClientConfig)
+
 	kubeClientConfig := genericConfig.LoopbackClientConfig
 	clientgoExternalClient, err := clientgoclientset.NewForConfig(kubeClientConfig)
 	if err != nil {
@@ -511,14 +513,13 @@ func buildGenericConfig(
 		return
 	}
 
-	StartingDelegate, err = PatchKubeAPIServerConfig(genericConfig, versionedInformers, &pluginInitializers)
-	if err != nil {
+	if err := openshiftkubeapiserver.OpenShiftKubeAPIServerConfigPatch(genericConfig, versionedInformers, &pluginInitializers); err != nil {
 		lastErr = fmt.Errorf("failed to patch: %v", err)
 		return
 	}
 
 	if enablement.IsOpenShift() {
-		configdefault.SetAdmissionDefaults(s, versionedInformers, clientgoExternalClient)
+		admissionenablement.SetAdmissionDefaults(s, versionedInformers, clientgoExternalClient)
 	}
 	err = s.Admission.ApplyTo(
 		genericConfig,

diff --git a/openshift-kube-apiserver/admission/admissionenablement/admission.go b/openshift-kube-apiserver/admission/admissionenablement/admission.go
@@ -0,0 +1,15 @@
+package admissionenablement
+
+import (
+	"k8s.io/kubernetes/cmd/kube-apiserver/app/options"
+	"k8s.io/kubernetes/openshift-kube-apiserver/admission/customresourcevalidation/customresourcevalidationregistration"
+)
+
+func InstallOpenShiftAdmissionPlugins(o *options.ServerRunOptions) {
+	existingAdmissionOrder := o.Admission.GenericAdmission.RecommendedPluginOrder
+	o.Admission.GenericAdmission.RecommendedPluginOrder = NewOrderedKubeAdmissionPlugins(existingAdmissionOrder)
+	RegisterOpenshiftKubeAdmissionPlugins(o.Admission.GenericAdmission.Plugins)
+	customresourcevalidationregistration.RegisterCustomResourceValidation(o.Admission.GenericAdmission.Plugins)
+	existingDefaultOff := o.Admission.GenericAdmission.DefaultOffPlugins
+	o.Admission.GenericAdmission.DefaultOffPlugins = NewDefaultOffPluginsFunc(existingDefaultOff)()
+}
diff --git a/openshift-kube-apiserver/admission/admissionenablement/admission_config.go b/openshift-kube-apiserver/admission/admissionenablement/admission_config.go
@@ -0,0 +1,51 @@
+package admissionenablement
+
+import (
+	"time"
+
+	"github.com/openshift/library-go/pkg/apiserver/admission/admissiontimeout"
+	"k8s.io/apimachinery/pkg/labels"
+	"k8s.io/apiserver/pkg/admission"
+	"k8s.io/client-go/informers"
+	"k8s.io/client-go/kubernetes"
+	"k8s.io/kubernetes/cmd/kube-apiserver/app/options"
+	"k8s.io/kubernetes/openshift-kube-apiserver/admission/namespaceconditions"
+)
+
+const disableSCCLevelLabel = "security.openshift.io/disable-securitycontextconstraints"
+
+var enforceSCCSelector labels.Selector
+
+func init() {
+	var err error
+	enforceSCCSelector, err = labels.Parse(disableSCCLevelLabel + " != true")
+	if err != nil {
+		panic(err)
+	}
+}
+
+func SetAdmissionDefaults(o *options.ServerRunOptions, informers informers.SharedInformerFactory, kubeClient kubernetes.Interface) {
+	// set up the decorators we need.  This is done late and out of order because our decorators currently require informers which are not
+	// present until we start running
+	namespaceLabelDecorator := namespaceconditions.NamespaceLabelConditions{
+		NamespaceClient: kubeClient.CoreV1(),
+		NamespaceLister: informers.Core().V1().Namespaces().Lister(),
+
+		SkipLevelZeroNames: SkipRunLevelZeroPlugins,
+		SkipLevelOneNames:  SkipRunLevelOnePlugins,
+	}
+	sccLabelDecorator := namespaceconditions.NewConditionalAdmissionPlugins(
+		kubeClient.CoreV1(), informers.Core().V1().Namespaces().Lister(), enforceSCCSelector,
+		"security.openshift.io/SecurityContextConstraint", "security.openshift.io/SCCExecRestrictions")
+
+	o.Admission.GenericAdmission.Decorators = append(o.Admission.GenericAdmission.Decorators,
+		admission.Decorators{
+			// SCC can be skipped by setting a namespace label `security.openshift.io/disable-securitycontextconstraints = true`
+			// This is useful for disabling SCC and using PodSecurity admission instead.
+			admission.DecoratorFunc(sccLabelDecorator.WithNamespaceLabelSelector),
+
+			admission.DecoratorFunc(namespaceLabelDecorator.WithNamespaceLabelConditions),
+			admission.DecoratorFunc(admissiontimeout.AdmissionTimeout{Timeout: 13 * time.Second}.WithTimeout),
+		},
+	)
+}
diff --git a/openshift-kube-apiserver/admission/admissionenablement/register.go b/openshift-kube-apiserver/admission/admissionenablement/register.go
@@ -0,0 +1,114 @@
+package admissionenablement
+
+import (
+	"k8s.io/apimachinery/pkg/util/sets"
+	"k8s.io/apiserver/pkg/admission"
+	"k8s.io/apiserver/pkg/admission/plugin/resourcequota"
+	mutatingwebhook "k8s.io/apiserver/pkg/admission/plugin/webhook/mutating"
+
+	"github.com/openshift/apiserver-library-go/pkg/admission/imagepolicy"
+	imagepolicyapiv1 "github.com/openshift/apiserver-library-go/pkg/admission/imagepolicy/apis/imagepolicy/v1"
+	quotaclusterresourcequota "github.com/openshift/apiserver-library-go/pkg/admission/quota/clusterresourcequota"
+	"github.com/openshift/apiserver-library-go/pkg/securitycontextconstraints/sccadmission"
+	authorizationrestrictusers "k8s.io/kubernetes/openshift-kube-apiserver/admission/authorization/restrictusers"
+	quotaclusterresourceoverride "k8s.io/kubernetes/openshift-kube-apiserver/admission/autoscaling/clusterresourceoverride"
+	"k8s.io/kubernetes/openshift-kube-apiserver/admission/autoscaling/managementcpusoverride"
+	quotarunonceduration "k8s.io/kubernetes/openshift-kube-apiserver/admission/autoscaling/runonceduration"
+	"k8s.io/kubernetes/openshift-kube-apiserver/admission/customresourcevalidation/customresourcevalidationregistration"
+	"k8s.io/kubernetes/openshift-kube-apiserver/admission/network/externalipranger"
+	"k8s.io/kubernetes/openshift-kube-apiserver/admission/network/restrictedendpoints"
+	ingressadmission "k8s.io/kubernetes/openshift-kube-apiserver/admission/route"
+	projectnodeenv "k8s.io/kubernetes/openshift-kube-apiserver/admission/scheduler/nodeenv"
+	schedulerpodnodeconstraints "k8s.io/kubernetes/openshift-kube-apiserver/admission/scheduler/podnodeconstraints"
+)
+
+func RegisterOpenshiftKubeAdmissionPlugins(plugins *admission.Plugins) {
+	authorizationrestrictusers.Register(plugins)
+	imagepolicy.Register(plugins)
+	ingressadmission.Register(plugins)
+	managementcpusoverride.Register(plugins)
+	projectnodeenv.Register(plugins)
+	quotaclusterresourceoverride.Register(plugins)
+	quotaclusterresourcequota.Register(plugins)
+	quotarunonceduration.Register(plugins)
+	schedulerpodnodeconstraints.Register(plugins)
+	sccadmission.Register(plugins)
+	sccadmission.RegisterSCCExecRestrictions(plugins)
+	externalipranger.RegisterExternalIP(plugins)
+	restrictedendpoints.RegisterRestrictedEndpoints(plugins)
+}
+
+var (
+
+	// these are admission plugins that cannot be applied until after the kubeapiserver starts.
+	// TODO if nothing comes to mind in 3.10, kill this
+	SkipRunLevelZeroPlugins = sets.NewString()
+	// these are admission plugins that cannot be applied until after the openshiftapiserver apiserver starts.
+	SkipRunLevelOnePlugins = sets.NewString(
+		imagepolicyapiv1.PluginName, // "image.openshift.io/ImagePolicy"
+		"quota.openshift.io/ClusterResourceQuota",
+		"security.openshift.io/SecurityContextConstraint",
+		"security.openshift.io/SCCExecRestrictions",
+	)
+
+	// openshiftAdmissionPluginsForKubeBeforeMutating are the admission plugins to add after kube admission, before mutating webhooks
+	openshiftAdmissionPluginsForKubeBeforeMutating = []string{
+		"autoscaling.openshift.io/ClusterResourceOverride",
+		managementcpusoverride.PluginName, // "autoscaling.openshift.io/ManagementCPUsOverride"
+		"authorization.openshift.io/RestrictSubjectBindings",
+		"autoscaling.openshift.io/RunOnceDuration",
+		"scheduling.openshift.io/PodNodeConstraints",
+		"scheduling.openshift.io/OriginPodNodeEnvironment",
+		"network.openshift.io/ExternalIPRanger",
+		"network.openshift.io/RestrictedEndpointsAdmission",
+		imagepolicyapiv1.PluginName, // "image.openshift.io/ImagePolicy"
+		"security.openshift.io/SecurityContextConstraint",
+		"security.openshift.io/SCCExecRestrictions",
+		"route.openshift.io/IngressAdmission",
+	}
+
+	// openshiftAdmissionPluginsForKubeAfterResourceQuota are the plugins to add after ResourceQuota plugin
+	openshiftAdmissionPluginsForKubeAfterResourceQuota = []string{
+		"quota.openshift.io/ClusterResourceQuota",
+	}
+
+	// additionalDefaultOnPlugins is a list of plugins we turn on by default that core kube does not.
+	additionalDefaultOnPlugins = sets.NewString(
+		"NodeRestriction",
+		"OwnerReferencesPermissionEnforcement",
+		"PersistentVolumeLabel",
+		"PodNodeSelector",
+		"PodTolerationRestriction",
+		"Priority",
+		imagepolicyapiv1.PluginName, // "image.openshift.io/ImagePolicy"
+		"StorageObjectInUseProtection",
+	)
+)
+
+func NewOrderedKubeAdmissionPlugins(kubeAdmissionOrder []string) []string {
+	ret := []string{}
+	for _, curr := range kubeAdmissionOrder {
+		if curr == mutatingwebhook.PluginName {
+			ret = append(ret, openshiftAdmissionPluginsForKubeBeforeMutating...)
+			ret = append(ret, customresourcevalidationregistration.AllCustomResourceValidators...)
+		}
+
+		ret = append(ret, curr)
+
+		if curr == resourcequota.PluginName {
+			ret = append(ret, openshiftAdmissionPluginsForKubeAfterResourceQuota...)
+		}
+	}
+	return ret
+}
+
+func NewDefaultOffPluginsFunc(kubeDefaultOffAdmission sets.String) func() sets.String {
+	return func() sets.String {
+		kubeOff := sets.NewString(kubeDefaultOffAdmission.UnsortedList()...)
+		kubeOff.Delete(additionalDefaultOnPlugins.List()...)
+		kubeOff.Delete(openshiftAdmissionPluginsForKubeBeforeMutating...)
+		kubeOff.Delete(openshiftAdmissionPluginsForKubeAfterResourceQuota...)
+		kubeOff.Delete(customresourcevalidationregistration.AllCustomResourceValidators...)
+		return kubeOff
+	}
+}
diff --git a/openshift-kube-apiserver/admission/admissionenablement/register_test.go b/openshift-kube-apiserver/admission/admissionenablement/register_test.go
@@ -0,0 +1,54 @@
+package admissionenablement
+
+import (
+	"reflect"
+	"testing"
+
+	"k8s.io/apiserver/pkg/admission"
+	genericapiserver "k8s.io/apiserver/pkg/server"
+	"k8s.io/kubernetes/pkg/kubeapiserver/options"
+
+	"github.com/openshift/library-go/pkg/apiserver/admission/admissionregistrationtesting"
+	"k8s.io/kubernetes/openshift-kube-apiserver/admission/customresourcevalidation/customresourcevalidationregistration"
+)
+
+func TestAdmissionRegistration(t *testing.T) {
+	orderedAdmissionChain := NewOrderedKubeAdmissionPlugins(options.AllOrderedPlugins)
+	defaultOffPlugins := NewDefaultOffPluginsFunc(options.DefaultOffAdmissionPlugins())()
+	registerAllAdmissionPlugins := func(plugins *admission.Plugins) {
+		genericapiserver.RegisterAllAdmissionPlugins(plugins)
+		options.RegisterAllAdmissionPlugins(plugins)
+		RegisterOpenshiftKubeAdmissionPlugins(plugins)
+		customresourcevalidationregistration.RegisterCustomResourceValidation(plugins)
+	}
+	plugins := admission.NewPlugins()
+	registerAllAdmissionPlugins(plugins)
+
+	err := admissionregistrationtesting.AdmissionRegistrationTest(plugins, orderedAdmissionChain, defaultOffPlugins)
+	if err != nil {
+		t.Fatal(err)
+	}
+}
+
+// TestResourceQuotaBeforeClusterResourceQuota simply test wheather ResourceQuota plugin is before ClusterResourceQuota plugin
+func TestResourceQuotaBeforeClusterResourceQuota(t *testing.T) {
+	orderedAdmissionChain := NewOrderedKubeAdmissionPlugins(options.AllOrderedPlugins)
+
+	expectedOrderedAdmissionSubChain := []string{"ResourceQuota", "quota.openshift.io/ClusterResourceQuota", "AlwaysDeny"}
+	actualOrderedAdmissionChain := extractSubChain(orderedAdmissionChain, expectedOrderedAdmissionSubChain[0])
+
+	if !reflect.DeepEqual(actualOrderedAdmissionChain, expectedOrderedAdmissionSubChain) {
+		t.Fatalf("expected %v, got %v ", expectedOrderedAdmissionSubChain, actualOrderedAdmissionChain)
+	}
+}
+
+func extractSubChain(admissionChain []string, takeFrom string) []string {
+	indexOfTake := 0
+	for index, admission := range admissionChain {
+		if admission == takeFrom {
+			indexOfTake = index
+			break
+		}
+	}
+	return admissionChain[indexOfTake:]
+}
diff --git a/openshift-kube-apiserver/admission/authorization/restrictusers/groupcache_test.go b/openshift-kube-apiserver/admission/authorization/restrictusers/groupcache_test.go
@@ -0,0 +1,28 @@
+package restrictusers
+
+import (
+	userv1 "github.com/openshift/api/user/v1"
+)
+
+type fakeGroupCache struct {
+	groups []userv1.Group
+}
+
+func (g fakeGroupCache) GroupsFor(user string) ([]*userv1.Group, error) {
+	ret := []*userv1.Group{}
+	for i := range g.groups {
+		group := &g.groups[i]
+		for _, currUser := range group.Users {
+			if user == currUser {
+				ret = append(ret, group)
+				break
+			}
+		}
+
+	}
+	return ret, nil
+}
+
+func (g fakeGroupCache) HasSynced() bool {
+	return true
+}