[release-1.12] Add OIDC support (#994)

* Update contract with fields for OIDC information (knative-extensions#3632) * Update contract to include OIDC information * Run hack/update-codegen.sh * Move OIDC SA to egress * Expose OIDC audience of KafkaChannel in its status (knative-extensions#3622) * Provision .status.address.audience and .status.addresses[*].audience in KafkaChannel * Add kafka Channel e2e test to check if audience is provisioned * Run goimport * Update deps * Auto generate Triggers OIDC identity service account and expose in its status (knative-extensions#3604) * Support auto generation of Triggers identity service account and expose in AuthStatus Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * make ServiceAccountLister public Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * add oidc unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * integrate oidc unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * add some logic to reconcile triggers, if the features config map gets updated Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * revert vendor/knative.dev/pkg/webhook/resourcesemantics/defaulting/controller.go Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix Verify Deps and Codegen test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test and reconcile triggers, in case of the features configmap changes Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> --------- Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * Create KafkaSources OIDC service account and expose in its status (knative-extensions#3660) * Create KafkaSources OIDC service account and expose in its status * Run goimport * Provision contract with OIDC information (knative-extensions#3646) * Provision contract with OIDC information * Add DLS audience in KafkaChannel CRD * Update KafkaSource to expose its sinks audience in status * Update Trigger test to include OIDC SA in contract * Propagate KafkaSources OIDC serviceAccountName to consumer and consumergroup * Propagate triggerv2s serviceAccountName to consumergroup * Fix unit test * [data-plane] Add the caching for OIDC JWT token to tokenprovider (knative-extensions#3663) * Add the caching to tokenprovider * Add the boiler * Running the codegen * Running the codegen * Revert "Running the codegen" This reverts commit 0ccf69c. * Use constant, set buffer before token expire, remove unnecessary change * Codegen changes * Codegen changes * Update data-plane/core/src/main/java/dev/knative/eventing/kafka/broker/core/oidc/TokenProvider.java Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Fix the review comments * Run codegen * Codegen changes * Code gen again --------- Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Receiver reject requests for wrong audience (knative-extensions#3675) * Receiver: reject request for wrong audience * Switch to AuthenticationHandler * Fix "Request has already been read" issue * Change TokenVerifier to an interface * Initialize TokenVerifier in main * Add test for AuthenticationHandler * Only initialize OIDC discovery config in main and create a TokenVerifier per verticle instance. * Rerun hack/update-codegen.sh * Move TokenVerifier setup into setup() to prevent null pointer exception when vertx is null * Update KafkaChannel OIDC e2e tests, to run OIDC conformance tests so the receiver is tested too. * Run OIDC e2e tests as part of the reconciler suite * Fix KafkaChannelOIDC e2e test * Fix lint issue * Address review comments * Dispatcher authenticate requests (knative-extensions#3677) * Change TokenProvider to return future to get a token * Dispatcher add OIDC to token, when target has an audience set * Add e2e test * Support exposing the Audience of a Broker (knative-extensions#3600) * Support exposing the Audience of a Broker * fix formatting * fix formatting * test fixes * Populate broker.status.addresses[*].audience field too * Run goimports and gofmt * Fix unit test --------- Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Add broker OIDC e2e tests (knative-extensions#3685) * Add broker OIDC e2e tests * Fix broker template to allow TLS & OIDC configuration on dead letter sink * Remove unneeded check when setting broker audience (knative-extensions#3708) * Check status code of OIDC discovery response (knative-extensions#3707) * Check status code of OIDC discovery endpoint * Run update-codegen.sh * Only allow 200 status code on OIDC discovery endpoint * Run update-deps.sh * Add OIDC tests to encryption/auth test suite * run make generate-release * TokenVerifier: execute blocking calls in parallel (knative-extensions#3728) * TokenVerifier: execute blocking calls in parallel * Revert "TokenVerifier: execute blocking calls in parallel" This reverts commit f3dbde9. * Revert: removed changes in contract.pb.go * Cancel receiver pod start on invalid OIDC config only if authentication.oidc is enabled (knative-extensions#3761) * Cancel pod start on invalid OIDC config only if authentication.oidc is enabled * Update namespaced broker to copy features configmap too. * Add unit test for FeaturesConfig class (knative-extensions#3771) * Add unit test for FeaturesConfig class * Update data-plane/core/src/test/java/dev/knative/eventing/kafka/broker/core/features/FeaturesConfigTest.java Co-authored-by: Calum Murray <cmurray@redhat.com> --------- Co-authored-by: Calum Murray <cmurray@redhat.com> * Run make generate-release again --------- Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> Co-authored-by: cola <45722758+xiangpingjiang@users.noreply.github.com> Co-authored-by: Leo Li <leoli@redhat.com> Co-authored-by: Gunish Matta <33680363+gunishmatta@users.noreply.github.com> Co-authored-by: Partha Ghosh <112557191+parth721@users.noreply.github.com> Co-authored-by: Calum Murray <cmurray@redhat.com>
openshift-knative · Apr 3, 2024 · 07c386b · 07c386b
1 parent ef01093
commit 07c386b
Show file tree

Hide file tree

Showing 92 changed files with 3,852 additions and 491 deletions.
diff --git a/control-plane/config/eventing-kafka-broker/100-channel/100-kafka-channel.yaml b/control-plane/config/eventing-kafka-broker/100-channel/100-kafka-channel.yaml
@@ -86,6 +86,9 @@ spec:
                           type: string
                         CACerts:
                           type: string
+                        audience:
+                          description: Audience is the OIDC audience for the deadLetterSink.
+                          type: string
                     retry:
                       description: Retry is the minimum number of retries the sender should attempt when sending an event before moving it to the dead letter sink.
                       type: integer
@@ -132,6 +135,9 @@ spec:
                                 type: string
                               CACerts:
                                 type: string
+                              audience:
+                                description: Audience is the OIDC audience for the deadLetterSink.
+                                type: string
                           retry:
                             description: Retry is the minimum number of retries the sender should attempt when sending an event before moving it to the dead letter sink.
                             type: integer
@@ -147,15 +153,28 @@ spec:
                       replyCACerts:
                         description: replyCACerts is the CA certs to trust for the reply.
                         type: string
+                      replyAudience:
+                        description: ReplyAudience is the OIDC audience for the replyUri.
+                        type: string
                       subscriberUri:
                         description: SubscriberURI is the endpoint for the subscriber
                         type: string
                       subscriberCACerts:
                         description: SubscriberCACerts is the CA certs to trust for the subscriber.
                         type: string
+                      subscriberAudience:
+                        description: SubscriberAudience is the OIDC audience for the subscriberUri.
+                        type: string
                       uid:
                         description: UID is used to understand the origin of the subscriber.
                         type: string
+                      auth:
+                        description: Auth provides the relevant information for OIDC authentication.
+                        type: object
+                        properties:
+                          serviceAccountName:
+                            description: ServiceAccountName is the name of the generated service account used for this components OIDC authentication.
+                            type: string
             status:
               description: Status represents the current state of the KafkaChannel. This data may be out of date.
               type: object
@@ -239,6 +258,9 @@ spec:
                   type: string
                 deadLetterSinkCACerts:
                   type: string
+                deadLetterSinkAudience:
+                  description: OIDC audience of the dead letter sink.
+                  type: string
                 observedGeneration:
                   description: ObservedGeneration is the 'Generation' of the Service that was last processed by the controller.
                   type: integer
@@ -262,6 +284,13 @@ spec:
                       uid:
                         description: UID is used to understand the origin of the subscriber.
                         type: string
+                      auth:
+                        description: Auth provides the relevant information for OIDC authentication.
+                        type: object
+                        properties:
+                          serviceAccountName:
+                            description: ServiceAccountName is the name of the generated service account used for this components OIDC authentication.
+                            type: string
       additionalPrinterColumns:
         - name: Ready
           type: string

diff --git a/control-plane/config/eventing-kafka-broker/100-source/100-kafka-source.yaml b/control-plane/config/eventing-kafka-broker/100-source/100-kafka-source.yaml
@@ -87,9 +87,6 @@ spec:
                       description: DeadLetterSink is the sink receiving event that could not be sent to a destination.
                       type: object
                       properties:
-                        CACerts:
-                          description: CACerts are Certification Authority (CA) certificates in PEM format according to https://www.rfc-editor.org/rfc/rfc7468. If set, these CAs are appended to the set of CAs provided by the Addressable target, if any.
-                          type: string
                         ref:
                           description: Ref points to an Addressable.
                           type: object
@@ -118,6 +115,12 @@ spec:
                         uri:
                           description: URI can be an absolute URL(non-empty scheme and non-empty host) pointing to the target or a relative URI. Relative URIs will be resolved using the base URI retrieved from Ref.
                           type: string
+                        CACerts:
+                          description: CACerts are Certification Authority (CA) certificates in PEM format according to https://www.rfc-editor.org/rfc/rfc7468. If set, these CAs are appended to the set of CAs provided by the Addressable target, if any.
+                          type: string
+                        audience:
+                          description: Audience is the OIDC audience for the deadLetterSink.
+                          type: string
                     retry:
                       description: Retry is the minimum number of retries the sender should attempt when sending an event before moving it to the dead letter sink.
                       type: integer
@@ -271,9 +274,6 @@ spec:
                   description: Sink is a reference to an object that will resolve to a uri to use as the sink.
                   type: object
                   properties:
-                    CACerts:
-                      description: CACerts are Certification Authority (CA) certificates in PEM format according to https://www.rfc-editor.org/rfc/rfc7468. If set, these CAs are appended to the set of CAs provided by the Addressable target, if any.
-                      type: string
                     ref:
                       description: Ref points to an Addressable.
                       type: object
@@ -302,6 +302,12 @@ spec:
                     uri:
                       description: URI can be an absolute URL(non-empty scheme and non-empty host) pointing to the target or a relative URI. Relative URIs will be resolved using the base URI retrieved from Ref.
                       type: string
+                    CACerts:
+                      description: CACerts are Certification Authority (CA) certificates in PEM format according to https://www.rfc-editor.org/rfc/rfc7468. If set, these CAs are appended to the set of CAs provided by the Addressable target, if any.
+                      type: string
+                    audience:
+                      description: Audience is the OIDC audience for the sink.
+                      type: string
                 topics:
                   description: Topic topics to consume messages from
                   type: array
@@ -392,6 +398,16 @@ spec:
                 sinkUri:
                   description: SinkURI is the current active sink URI that has been configured for the Source.
                   type: string
+                sinkAudience:
+                  description: SinkAudience is the OIDC audience of the sink.
+                  type: string
+                auth:
+                  description: Auth provides the relevant information for OIDC authentication.
+                  type: object
+                  properties:
+                    serviceAccountName:
+                      description: ServiceAccountName is the name of the generated service account used for this components OIDC authentication.
+                      type: string
       subresources:
         status: {}
         scale:

diff --git a/control-plane/config/eventing-kafka-broker/200-controller/200-controller-cluster-role.yaml b/control-plane/config/eventing-kafka-broker/200-controller/200-controller-cluster-role.yaml
@@ -140,6 +140,13 @@ rules:
       - update
       - create
       - delete
+  # To grant NamespacedBroker permissions to create OIDC tokens
+  - apiGroups:
+      - ""
+    resources:
+      - serviceaccounts/token
+    verbs:
+      - create
 
   # Scheduler permissions
   - apiGroups:

diff --git a/control-plane/pkg/apis/internals/kafka/eventing/v1alpha1/consumer_group_types.go b/control-plane/pkg/apis/internals/kafka/eventing/v1alpha1/consumer_group_types.go
@@ -100,6 +100,10 @@ type ConsumerGroupSpec struct {
 	// More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors
 	// +optional
 	Selector map[string]string `json:"selector,omitempty" protobuf:"bytes,2,rep,name=selector"`
+
+	// OIDCServiceAccountName is the name of service account used for this components
+	// OIDC authentication.
+	OIDCServiceAccountName *string `json:"oidcServiceAccountName,omitempty"`
 }
 
 type ConsumerGroupStatus struct {
@@ -120,6 +124,10 @@ type ConsumerGroupStatus struct {
 	// +optional
 	SubscriberCACerts *string `json:"subscriberCACerts,omitempty"`
 
+	// SubscriberAudience is the OIDC audience for the resolved URI
+	// +optional
+	SubscriberAudience *string `json:"subscriberAudience,omitempty"`
+
 	// DeliveryStatus contains a resolved URL to the dead letter sink address, and any other
 	// resolved delivery options.
 	eventingduckv1.DeliveryStatus `json:",inline"`

diff --git a/control-plane/pkg/apis/internals/kafka/eventing/v1alpha1/consumer_types.go b/control-plane/pkg/apis/internals/kafka/eventing/v1alpha1/consumer_types.go
@@ -103,6 +103,10 @@ type ConsumerSpec struct {
 
 	// PodBind represents a reference to the pod in which the consumer should be placed.
 	PodBind *PodBind `json:"podBind"`
+
+	// OIDCServiceAccountName is the name of the generated service account
+	// used for this components OIDC authentication.
+	OIDCServiceAccountName *string `json:"oidcServiceAccountName,omitempty"`
 }
 
 type ReplyStrategy struct {
@@ -208,6 +212,10 @@ type ConsumerStatus struct {
 	// +optional
 	SubscriberCACerts *string `json:"subscriberCACerts,omitempty"`
 
+	// SubscriberAudience is the OIDC audience for the resolved URI
+	// +optional
+	SubscriberAudience *string `json:"subscriberAudience,omitempty"`
+
 	// DeliveryStatus contains a resolved URL to the dead letter sink address, and any other
 	// resolved delivery options.
 	eventingduck.DeliveryStatus `json:",inline"`

diff --git a/control-plane/pkg/apis/internals/kafka/eventing/v1alpha1/zz_generated.deepcopy.go b/control-plane/pkg/apis/internals/kafka/eventing/v1alpha1/zz_generated.deepcopy.go
diff --git a/control-plane/pkg/apis/sources/v1beta1/kafka_lifecycle.go b/control-plane/pkg/apis/sources/v1beta1/kafka_lifecycle.go
@@ -46,6 +46,9 @@ const (
 	// KafkaConditionInitialOffsetsCommitted is True when the KafkaSource has committed the
 	// initial offset of all claims
 	KafkaConditionInitialOffsetsCommitted apis.ConditionType = "InitialOffsetsCommitted"
+
+	// KafkaConditionOIDCIdentityCreated has status True when the KafkaSource has created an OIDC identity.
+	KafkaConditionOIDCIdentityCreated apis.ConditionType = "OIDCIdentityCreated"
 )
 
 var (
@@ -54,6 +57,7 @@ var (
 		KafkaConditionDeployed,
 		KafkaConditionConnectionEstablished,
 		KafkaConditionInitialOffsetsCommitted,
+		KafkaConditionOIDCIdentityCreated,
 	)
 
 	kafkaCondSetLock = sync.RWMutex{}
@@ -91,6 +95,7 @@ func (s *KafkaSourceStatus) MarkSink(addr *duckv1.Addressable) {
 	if addr.URL != nil && !addr.URL.IsEmpty() {
 		s.SinkURI = addr.URL
 		s.SinkCACerts = addr.CACerts
+		s.SinkAudience = addr.Audience
 		KafkaSourceCondSet.Manage(s).MarkTrue(KafkaConditionSinkProvided)
 	} else {
 		KafkaSourceCondSet.Manage(s).MarkUnknown(KafkaConditionSinkProvided, "SinkEmpty", "Sink has resolved to empty.%s", "")
@@ -160,6 +165,22 @@ func (s *KafkaSourceStatus) MarkInitialOffsetNotCommitted(reason, messageFormat
 	KafkaSourceCondSet.Manage(s).MarkFalse(KafkaConditionInitialOffsetsCommitted, reason, messageFormat, messageA...)
 }
 
+func (s *KafkaSourceStatus) MarkOIDCIdentityCreatedSucceeded() {
+	KafkaSourceCondSet.Manage(s).MarkTrue(KafkaConditionOIDCIdentityCreated)
+}
+
+func (s *KafkaSourceStatus) MarkOIDCIdentityCreatedSucceededWithReason(reason, messageFormat string, messageA ...interface{}) {
+	KafkaSourceCondSet.Manage(s).MarkTrueWithReason(KafkaConditionOIDCIdentityCreated, reason, messageFormat, messageA...)
+}
+
+func (s *KafkaSourceStatus) MarkOIDCIdentityCreatedFailed(reason, messageFormat string, messageA ...interface{}) {
+	KafkaSourceCondSet.Manage(s).MarkFalse(KafkaConditionOIDCIdentityCreated, reason, messageFormat, messageA...)
+}
+
+func (s *KafkaSourceStatus) MarkOIDCIdentityCreatedUnknown(reason, messageFormat string, messageA ...interface{}) {
+	KafkaSourceCondSet.Manage(s).MarkUnknown(KafkaConditionOIDCIdentityCreated, reason, messageFormat, messageA...)
+}
+
 func (s *KafkaSourceStatus) UpdateConsumerGroupStatus(status string) {
 	s.Claims = status
 }