-
Notifications
You must be signed in to change notification settings - Fork 905
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: KFP multi user mode PR1 - enable multi user mode without istio authorization #1342
Merged
k8s-ci-robot
merged 50 commits into
kubeflow:master
from
Bobgy:kfp_multi_user_without_istio
Jul 7, 2020
Merged
Changes from all commits
Commits
Show all changes
50 commits
Select commit
Hold shift + click to select a range
1c43da3
Add argo to stacks/generic
Bobgy 9e9c725
Pull pipelines manifest from upstream
Bobgy 3fa9d21
Updated kfp
Bobgy 50f2ff5
Minio v3 manifests
Bobgy b2cc166
Rename minio configmap
Bobgy f33ea93
Add generic minio install
Bobgy 6151fe9
Generate new test data
Bobgy d396143
Mysql kustomize v3 manifest - generic install
Bobgy 2544716
Add mysql gcp pd install
Bobgy 0a259a1
Generate test data
Bobgy b364e06
Pipelines kustomize v3 manifests
Bobgy 2f59319
Add kfp ui virtual service
Bobgy 569c640
Add metadata deployment to stacks/generic
Bobgy a3e5c3a
Use common cluster domain
Bobgy cfda2d9
Deploy metadata writer
Bobgy 0a4770b
Add kfp cache server
Bobgy 7907d06
Update test data
Bobgy 400aa0b
Merge remote-tracking branch 'upstream/master' into v3_kfp
Bobgy d4b33b6
Enable KFP multi user mode without istio security
Bobgy 5530b2a
Fix persistence agent watch namespace
Bobgy 97424f1
Fix namespace env for some deployments
Bobgy 805d6da
Fix cluster roles and bindings
Bobgy 7c32e6f
fix rename
Bobgy 7bddf40
Fix pipelines ui role
Bobgy 88cc01b
Updated kfp to rc2
Bobgy 5f92c94
simplify pipeline v3 manifest using updated kfp rc2 manifest
Bobgy 59b8ff9
Fix pipeline-install-config
Bobgy 72e598d
remove redundant configmap
Bobgy 66658b9
update tests
Bobgy 7358da0
Merge branch 'v3_kfp' into kfp_multi_user_without_istio
Bobgy 1eb4d56
updated to kfp 1.0.0-rc.3
Bobgy 9bc841b
Adapt to kfp 1.0rc3 refactoring
Bobgy e73382b
update test snapshots
Bobgy b7c6795
Merge branch 'kfp_rc3' into kfp_multi_user_without_istio
Bobgy 03a63d8
fix pull kfp script to detect empty dir
Bobgy 0df67aa
fix example ref
Bobgy 28452a3
update snapshot
Bobgy 4faf673
fix gcp pd manifest
Bobgy 2b35d9e
Update stacks ref
Bobgy 6df2abc
revert alice example to gcp stack
Bobgy 0230d0c
update snapshot
Bobgy 5b3d882
fix profile controller iam binding
Bobgy 2c5f8d0
Merge remote-tracking branch 'upstream/master' into kfp_multi_user_wi…
Bobgy 30552d1
Update kfp profile controller can be configured to different images and
Bobgy 604e8b2
Merge remote-tracking branch 'upstream/master' into kfp_multi_user_wi…
Bobgy c71bf14
add missing viewer controller cluster roles
Bobgy 4f2f96c
Use python3 for sync.py
Bobgy 78b3eea
Revert gcp stack back to use non multi user kfp
Bobgy 09cdd13
revert unintended changes
Bobgy e0baea1
revert upstream changes
Bobgy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
11 changes: 11 additions & 0 deletions
11
pipeline/installs/multi-user/api-service/cluster-role-binding.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1beta1 | ||
| kind: ClusterRoleBinding | ||
| metadata: | ||
| name: ml-pipeline | ||
| roleRef: | ||
| apiGroup: rbac.authorization.k8s.io | ||
| kind: ClusterRole | ||
| name: ml-pipeline | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: ml-pipeline |
34 changes: 34 additions & 0 deletions
34
pipeline/installs/multi-user/api-service/cluster-role.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1beta1 | ||
| kind: ClusterRole | ||
| metadata: | ||
| name: ml-pipeline | ||
| rules: | ||
| - apiGroups: | ||
| - argoproj.io | ||
| resources: | ||
| - workflows | ||
| verbs: | ||
| - create | ||
| - get | ||
| - list | ||
| - watch | ||
| - update | ||
| - patch | ||
| - delete | ||
| - apiGroups: | ||
| - kubeflow.org | ||
| resources: | ||
| - scheduledworkflows | ||
| verbs: | ||
| - create | ||
| - get | ||
| - list | ||
| - update | ||
| - patch | ||
| - delete | ||
| - apiGroups: | ||
| - "" | ||
| resources: | ||
| - pods | ||
| verbs: | ||
| - delete |
12 changes: 12 additions & 0 deletions
12
pipeline/installs/multi-user/api-service/deployment-patch.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: ml-pipeline | ||
| spec: | ||
| template: | ||
| spec: | ||
| containers: | ||
| - name: ml-pipeline-api-server | ||
| envFrom: | ||
| - configMapRef: | ||
| name: pipeline-api-server-config |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| resources: | ||
| - cluster-role-binding.yaml | ||
| - cluster-role.yaml | ||
| configMapGenerator: | ||
| - name: pipeline-api-server-config | ||
| env: params.env |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| MULTIUSER=true | ||
| DEFAULTPIPELINERUNNERSERVICEACCOUNT=default-editor | ||
| VISUALIZATIONSERVICE_NAME=ml-pipeline-visualizationserver | ||
| VISUALIZATIONSERVICE_PORT=8888 |
11 changes: 11 additions & 0 deletions
11
pipeline/installs/multi-user/cache/cluster-role-binding.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRoleBinding | ||
| metadata: | ||
| name: kubeflow-pipelines-cache-binding | ||
| roleRef: | ||
| apiGroup: rbac.authorization.k8s.io | ||
| kind: ClusterRole | ||
| name: kubeflow-pipelines-cache-role | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: kubeflow-pipelines-cache |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRole | ||
| metadata: | ||
| name: kubeflow-pipelines-cache-role | ||
| rules: | ||
| - apiGroups: | ||
| - "" | ||
| resources: | ||
| - pods | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch | ||
| - update | ||
| - patch | ||
| - apiGroups: | ||
| - "" | ||
| resources: | ||
| - configmaps | ||
| verbs: | ||
| - get | ||
| - apiGroups: | ||
| - argoproj.io | ||
| resources: | ||
| - workflows | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch | ||
| - update | ||
| - patch |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: cache-server | ||
| spec: | ||
| template: | ||
| spec: | ||
| containers: | ||
| - name: server | ||
| env: | ||
| - name: NAMESPACE_TO_WATCH | ||
| value: '' | ||
| valueFrom: null |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| resources: | ||
| - cluster-role.yaml | ||
| - cluster-role-binding.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| namespace: kubeflow | ||
| resources: | ||
| - ../generic | ||
| - api-service | ||
| - pipelines-ui | ||
| - pipelines-profile-controller | ||
| - scheduled-workflow | ||
| - viewer-controller | ||
| - persistence-agent | ||
| - cache | ||
| - metadata-writer | ||
| patchesStrategicMerge: | ||
| - api-service/deployment-patch.yaml | ||
| - pipelines-ui/deployment-patch.yaml | ||
| - scheduled-workflow/deployment-patch.yaml | ||
| - viewer-controller/deployment-patch.yaml | ||
| - persistence-agent/deployment-patch.yaml | ||
| - metadata-writer/deployment-patch.yaml | ||
| - cache/deployment-patch.yaml |
11 changes: 11 additions & 0 deletions
11
pipeline/installs/multi-user/metadata-writer/cluster-role-binding.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRoleBinding | ||
| metadata: | ||
| name: kubeflow-pipelines-metadata-writer-binding | ||
| roleRef: | ||
| apiGroup: rbac.authorization.k8s.io | ||
| kind: ClusterRole | ||
| name: kubeflow-pipelines-metadata-writer-role | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: kubeflow-pipelines-metadata-writer |
31 changes: 31 additions & 0 deletions
31
pipeline/installs/multi-user/metadata-writer/cluster-role.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRole | ||
| metadata: | ||
| name: kubeflow-pipelines-metadata-writer-role | ||
| rules: | ||
| - apiGroups: | ||
| - "" | ||
| resources: | ||
| - pods | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch | ||
| - update | ||
| - patch | ||
| - apiGroups: | ||
| - "" | ||
| resources: | ||
| - configmaps | ||
| verbs: | ||
| - get | ||
| - apiGroups: | ||
| - argoproj.io | ||
| resources: | ||
| - workflows | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch | ||
| - update | ||
| - patch |
13 changes: 13 additions & 0 deletions
13
pipeline/installs/multi-user/metadata-writer/deployment-patch.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: metadata-writer | ||
| spec: | ||
| template: | ||
| spec: | ||
| containers: | ||
| - name: main | ||
| env: | ||
| - name: NAMESPACE_TO_WATCH | ||
| value: '' | ||
| valueFrom: null |
5 changes: 5 additions & 0 deletions
5
pipeline/installs/multi-user/metadata-writer/kustomization.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| resources: | ||
| - cluster-role.yaml | ||
| - cluster-role-binding.yaml |
11 changes: 11 additions & 0 deletions
11
pipeline/installs/multi-user/persistence-agent/cluster-role-binding.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRoleBinding | ||
| metadata: | ||
| name: ml-pipeline-persistenceagent-binding | ||
| roleRef: | ||
| apiGroup: rbac.authorization.k8s.io | ||
| kind: ClusterRole | ||
| name: ml-pipeline-persistenceagent-role | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: ml-pipeline-persistenceagent |
21 changes: 21 additions & 0 deletions
21
pipeline/installs/multi-user/persistence-agent/cluster-role.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: ClusterRole | ||
| metadata: | ||
| name: ml-pipeline-persistenceagent-role | ||
| rules: | ||
| - apiGroups: | ||
| - argoproj.io | ||
| resources: | ||
| - workflows | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch | ||
| - apiGroups: | ||
| - kubeflow.org | ||
| resources: | ||
| - scheduledworkflows | ||
| verbs: | ||
| - get | ||
| - list | ||
| - watch |
13 changes: 13 additions & 0 deletions
13
pipeline/installs/multi-user/persistence-agent/deployment-patch.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: ml-pipeline-persistenceagent | ||
| spec: | ||
| template: | ||
| spec: | ||
| containers: | ||
| - name: ml-pipeline-persistenceagent | ||
| env: | ||
| - name: NAMESPACE | ||
| value: '' | ||
| valueFrom: null |
5 changes: 5 additions & 0 deletions
5
pipeline/installs/multi-user/persistence-agent/kustomization.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| resources: | ||
| - cluster-role.yaml | ||
| - cluster-role-binding.yaml |
43 changes: 43 additions & 0 deletions
43
pipeline/installs/multi-user/pipelines-profile-controller/composite-controller.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| apiVersion: metacontroller.k8s.io/v1alpha1 | ||
| kind: CompositeController | ||
| metadata: | ||
| name: profile-controller | ||
| spec: | ||
| generateSelector: true | ||
| resyncPeriodSeconds: 10 | ||
| parentResource: | ||
| apiVersion: v1 | ||
| resource: namespaces | ||
| childResources: | ||
| - apiVersion: v1 | ||
| resource: secrets | ||
| updateStrategy: | ||
| method: OnDelete | ||
| - apiVersion: v1 | ||
| resource: configmaps | ||
| updateStrategy: | ||
| method: OnDelete | ||
| - apiVersion: apps/v1 | ||
| resource: deployments | ||
| updateStrategy: | ||
| method: InPlace | ||
| - apiVersion: v1 | ||
| resource: services | ||
| updateStrategy: | ||
| method: InPlace | ||
| - apiVersion: networking.istio.io/v1alpha3 | ||
| resource: destinationrules | ||
| updateStrategy: | ||
| method: InPlace | ||
| - apiVersion: rbac.istio.io/v1alpha1 | ||
| resource: serviceroles | ||
| updateStrategy: | ||
| method: InPlace | ||
| - apiVersion: rbac.istio.io/v1alpha1 | ||
| resource: servicerolebindings | ||
| updateStrategy: | ||
| method: InPlace | ||
| hooks: | ||
| sync: | ||
| webhook: | ||
| url: http://kubeflow-pipelines-profile-controller/sync | ||
27 changes: 27 additions & 0 deletions
27
pipeline/installs/multi-user/pipelines-profile-controller/deployment.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| apiVersion: apps/v1beta1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: profile-controller | ||
| spec: | ||
| replicas: 1 | ||
| template: | ||
| metadata: | ||
| annotations: | ||
| sidecar.istio.io/inject: "false" | ||
| spec: | ||
| containers: | ||
| - name: profile-controller | ||
| image: python:3.7 | ||
| command: ["python", "/hooks/sync.py"] | ||
| envFrom: | ||
| - configMapRef: | ||
| name: profile-controller-env | ||
| volumeMounts: | ||
| - name: hooks | ||
| mountPath: /hooks | ||
| ports: | ||
| - containerPort: 80 | ||
| volumes: | ||
| - name: hooks | ||
| configMap: | ||
| name: profile-controller-code |
17 changes: 17 additions & 0 deletions
17
pipeline/installs/multi-user/pipelines-profile-controller/kustomization.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| namespace: kubeflow | ||
| nameprefix: kubeflow-pipelines- | ||
| commonLabels: | ||
| app: kubeflow-pipelines-profile-controller | ||
| resources: | ||
| - service.yaml | ||
| - deployment.yaml | ||
| - composite-controller.yaml | ||
| configMapGenerator: | ||
| - name: profile-controller-code | ||
| files: | ||
| - sync.py | ||
| - name: profile-controller-env | ||
| envs: | ||
| - params.env |
3 changes: 3 additions & 0 deletions
3
pipeline/installs/multi-user/pipelines-profile-controller/params.env
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| KFP_VERSION=1.0.0-rc.3 | ||
| # TODO: make visualization server work with sidecar | ||
| DISABLE_ISTIO_SIDECAR=true |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Bobgy This will likely fail on other platforms or on-prem installations, I had to point to the internal service name:
url: http://kubeflow-pipelines-profile-controller.kubeflow.svc.cluster.local:80/syncPerhaps this can be configurable via Kustomize vars. I think of the use of the name prefix just makes it a bit harder.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks.
is using prefix a GCP specific thing?
I thought it's kubernetes standard behavior.
let me investigate