Update docs for connecting to Kubeflow Pipelines from the same cluster in multi-user mode #2905

bartgras · 2021-08-30T16:07:04Z

Changes requested in issue, comment .

google-cla · 2021-08-30T16:07:08Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-oss-robot · 2021-08-30T16:07:16Z

Hi @bartgras. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Arhell

@bartgras please sign CLA

bartgras · 2021-08-31T12:48:26Z

@googlebot I signed it!

andreyvelich

Thank you for updating this @bartgras, it's very useful!
@zijianjoy @Bobgy @RFMVasconcelos Please take a look.

content/en/docs/components/pipelines/sdk/connect-api.md

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

Bobgy · 2021-09-03T09:59:45Z

Thank you for the contribution!
The content looks great, I will spare some time for more detailed comments.

thesuperzapper

@Bobgy @bartgras I have left some comments.

thesuperzapper · 2021-09-05T23:34:00Z

content/en/docs/components/pipelines/sdk/connect-api.md

+If token is found, `kfp.Client` puts it in the `Authorization: Bearer <token>` header where Kubeflow Pipelines API can read it. No additional setup is required to refresh tokens.
+
+Setup is similar for other cases like e.g. **cron tasks or in-cluster CI/CD**: Configure your Pod with token in either `KF_PIPELINES_SA_TOKEN_PATH` or `/var/run/secrets/kubeflow/pipelines/token` and `kfp.Client` will gain access to Kubeflow Pipelines.
+


@bartgras I think there is an important piece of information missing, namely how to allow cross-namespaces access to the pipelines API.

For example, giving Notebook Pods in namespace-1 access to namespace-2 Pipelines can be done by giving, namespace-1:ServiceAccount/default-editor the ClusterRole/kubeflow-edit in namespace-2.

apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: kubeflow-edit-namespace-1 namespace: namespace-2 roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubeflow-edit subjects: - kind: ServiceAccount name: default-editor namespace: namespace-1

I was under assumption that users would use "Manage Contributors" to achieve the same result without additional RBAC settings.

Specifically, both namespace-1 and namespace-2 would require PodDefaults and then namespace-2 would add namespace-2 through contributors. I admit I didn't test if that's enough to get the access.If not, I can extend docs with your comment.

Both options should work, it's useful to introduce rbac when users want to configure on their own, probably with different granularity.

thesuperzapper · 2021-09-05T23:35:01Z

content/en/docs/components/pipelines/sdk/connect-api.md

+          - serviceAccountToken:
+              path: token
+              expirationSeconds: 7200
+              audience: pipelines.kubeflow.org      


I think it's important to explain how this works, as most users will have never seen this feature of Kubernetes.

Also, we should clarify that this serviceAccountToken method respects the Pipelines RBAC, and does not allow access beyond what the ServiceAcount running the notebook Pod has.

I didn't want to overflow users with details some of them might not find important if their goal is just to have in-cluster access to pipelines working.
Would adding something like this be enough, or you want to provide more info:
"More details about PodDefault can be found here".

+1 for the detailed info, because they need the details to understand how to access KFP API from any pod.

I think your argument about focusing on the minimum they need to know is also valid. What about adding a section at the end of the page to introduce the details.

"More details about PodDefault can be found here".

It should be good enough for me

thesuperzapper · 2021-09-05T23:39:32Z

content/en/docs/components/pipelines/sdk/connect-api.md

+After the manifest is deployed, newly created Jupyter notebook contains additional option (`Allow access to Kubeflow Pipelines`) in section **configurations**.
+Read more about **configurations** on [Jupyter notebook server](/docs/components/notebooks/setup/#create-a-jupyter-notebook-server-and-add-a-notebook).
+
+Note, Kubeflow `kfp.Client` expects token either in `KF_PIPELINES_SA_TOKEN_PATH` environment variable or mounted to `/var/run/secrets/kubeflow/pipelines/token`. Do not change these values in the manifest. Similarly, `audience` should not be modified as well.


This could be put first in this section, explaining that KF_PIPELINES_SA_TOKEN_PATH is what the kfp.Client() is looking for, and then how serviceAccountToken actually works. (From an auth-flow perspective)

thesuperzapper · 2021-09-05T23:40:04Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+Setup is similar for other cases like e.g. **cron tasks or in-cluster CI/CD**: Configure your Pod with token in either `KF_PIPELINES_SA_TOKEN_PATH` or `/var/run/secrets/kubeflow/pipelines/token` and `kfp.Client` will gain access to Kubeflow Pipelines.
+
+#### Generic


I'm not sure we have a concept of "generic" @Bobgy what do you think?

I agree. I wasn't sure what would be the right wording in this case. Could not find any other name in docs.

Maybe we can start with non-multi-user mode?

thesuperzapper · 2021-09-05T23:52:37Z

content/en/docs/components/pipelines/multi-user.md

-
-There is work-in-progress to support this use-case, refer to [github issue #5138](https://github.com/kubeflow/pipelines/issues/5138).
+Alternatively, in-cluster workloads like Jupyter notebooks or cron tasks can be also accessed through public endpoint. This option is platform specific and explained in 
+[Connect to Kubeflow Pipelines from outside your cluster](/docs/components/pipelines/sdk/connect-api/#connect-to-kubeflow-pipelines-from-outside-your-cluster)


I know it's not 100% related to this PR, but we should expand how to connect to the Pipelines endpoint from outside the cluster. Right now, (when using Dex) we don't document how people can access their https://kubeflow.example.com/pipelines endpoint from outside the cluster.

NOTE: The default manifests use "dex + oidc-authservice" with a very web-UI-focused authentication-flow.

Therefore, we should document how to use a "kubeflow service account" + "kubeflow token" to access the Kubeflow Pipelines REST API, with a fully configured Dex (that might use LDAP/OAUTH for normal users).

@Bobgy what do you think?

@Bobgy @bartgras I have raised a separate issue for this discussion: #2916

thesuperzapper · 2021-09-05T23:53:32Z

content/en/docs/components/pipelines/multi-user.md

-notebooks or cron tasks, current suggested workaround is to connect through
-public endpoint and follow platform specific documentation to authenticate
-programmatically using user credentials. For Google Cloud, you can refer to
-[Connecting to Kubeflow Pipelines in a full Kubeflow deployment on Google Cloud](/docs/gke/pipelines/authentication-sdk/#connecting-to-kubeflow-pipelines-in-a-full-kubeflow-deployment).


While this is a GKE-specific doc, we might want to explain this generically rather than remove it.

The link I added explains it for GKE and other platforms.

thesuperzapper · 2021-09-05T23:55:55Z

content/en/docs/components/pipelines/multi-user.md


-Clients can only access the Kubeflow Pipelines API from the public endpoint


We probably want to keep some preamble like this (possibly reworded), about how the Istio auth is setup in Kubeflow.

For example, most users may not realize that the istio-ingressgateway is how Kubeflow APIs are exposed external to the clsuter.

I searched the term "ingress gateway" on kubeflow.org and found that it's often mentioned in distribution-specific installation steps.

Istio ingress gateway is not being used exclusively by Kubeflow Pipelines. If we want to give more details about access etc. then something like Traffic Flow section could serve as a starting point and would probably fit better on Overview page. If you agree, I could create separate issue+PR for that.

andreyvelich · 2021-09-06T10:27:06Z

/ok-to-test

Bobgy · 2021-09-10T10:46:34Z

content/en/docs/components/pipelines/multi-user.md

-[Connecting to Kubeflow Pipelines in a full Kubeflow deployment on Google Cloud](/docs/gke/pipelines/authentication-sdk/#connecting-to-kubeflow-pipelines-in-a-full-kubeflow-deployment).
-
-There is work-in-progress to support this use-case, refer to [github issue #5138](https://github.com/kubeflow/pipelines/issues/5138).
+Alternatively, in-cluster workloads like Jupyter notebooks or cron tasks can be also accessed through public endpoint. This option is platform specific and explained in 


nit: can also access Kubeflow Pipelines API through the public endpoint.

Bobgy · 2021-09-10T10:48:06Z

content/en/docs/components/pipelines/sdk/connect-api.md

+Authentication uses ServiceAccountToken 
+[projection](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-token-volume-projection) where token with limited lifetime, that can be verified by Kubeflow Pipelines API, is being injected into (e.g. Jupyter notebook's) Pod.
+
+In order to **access Kubeflow Pipeline from Jupyter notebook**, additional per namespace (profile) manifest is required:


nit: Kubeflow Pipelines

Bobgy · 2021-09-10T10:51:42Z

content/en/docs/components/pipelines/sdk/connect-api.md

+      value: /var/run/secrets/kubeflow/pipelines/token
+```
+
+After the manifest is deployed, newly created Jupyter notebook contains additional option (`Allow access to Kubeflow Pipelines`) in section **configurations**.


contains an additional option

Bobgy · 2021-09-10T10:53:13Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+Note, Kubeflow `kfp.Client` expects token either in `KF_PIPELINES_SA_TOKEN_PATH` environment variable or mounted to `/var/run/secrets/kubeflow/pipelines/token`. Do not change these values in the manifest. Similarly, `audience` should not be modified as well.
+
+If token is found, `kfp.Client` puts it in the `Authorization: Bearer <token>` header where Kubeflow Pipelines API can read it. No additional setup is required to refresh tokens.


Do users need to know these?

Bobgy · 2021-09-10T10:54:49Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+Setup is similar for other cases like e.g. **cron tasks or in-cluster CI/CD**: Configure your Pod with token in either `KF_PIPELINES_SA_TOKEN_PATH` or `/var/run/secrets/kubeflow/pipelines/token` and `kfp.Client` will gain access to Kubeflow Pipelines.
+
+#### Generic


Maybe we can start with non-multi-user mode?

bartgras · 2021-09-13T11:07:28Z

@thesuperzapper @Bobgy Thanks for suggestions. Would these changes be enough?

andreyvelich · 2021-09-20T14:55:06Z

/lgtm

Are we ready to merge this PR ?
/cc @Bobgy @thesuperzapper

andreyvelich · 2021-09-24T10:16:55Z

@zijianjoy what do you think about this PR ?

thesuperzapper · 2021-09-27T21:45:51Z

@bartgras @andreyvelich there are still some unresolved comments, if we resolve them, I think we are happy to merge.

bartgras · 2021-09-28T05:58:32Z

@thesuperzapper I thought I addressed them all in my last push.
I'd appreciate it you point out which require more changes.

zijianjoy · 2021-09-28T16:26:20Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+##### Managing cross-namespaces access to Kubeflow Pipelines API
+
+The setup can be extended to allow other namespaces to access Kubeflow Pipelines API through your namespace's Juputer notebook server.


NIT: Jupyter instead of Juputer

Maybe I misunderstood sth, I think the setting below allows default-editor service account in namespace-1 to call KFP API and create pipelines/runs in namespace-2. (it also allows access to notebooks, but we won't use a service account to access notebooks, the goal here is to grant namespaced pipeline creation permission to a notebook in a separate namespace)

@thesuperzapper is this what you wanted to convey?

zijianjoy

/lgtm

Thank you for the PR! It looks good to me, pending @Bobgy for approval.

capri-xiyue · 2021-09-28T18:01:48Z

/lgtm

bartgras · 2021-09-29T01:02:50Z

I've just noticed this comment from @juliusvonkohout .
If we merge 2 PRs he mentioned, then documentation should be updated as well.
I can make necessary changes but first would like to hear your opinion.

Bobgy · 2021-09-30T10:19:33Z

I've just noticed this comment from @juliusvonkohout . If we merge 2 PRs he mentioned, then documentation should be updated as well. I can make necessary changes but first would like to hear your opinion.

Sorry for the delay,
We can merge this documentation update first, the technical details section is necessary anyway.

Bobgy

Thank you again for the great documentation contribution!
I love the new content you are adding.

Some general comments:

regarding content, I'd argue that the main documentation should be written without dependency on PodDefault. I agree PodDefault is a nice to have, but I don't want people to treat it as a hard dependency of KFP.
also, can we also introduce how a general Pod can be configured to access KFP API, besides the documentation for accessing KFP API from Kubeflow notebooks?
I feel like the overall tech writing quality needs some improvement. Are there anyone who want to help polish the doc here?

What do you all think?
@andreyvelich @thesuperzapper @bartgras @zijianjoy @capri-xiyue

Bobgy · 2021-09-30T12:01:07Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+It is important to understand that `serviceAccountToken` method respects the Kubeflow Pipelines RBAC, and does not allow access beyond what the ServiceAcount running the notebook Pod has.
+
+#### Non-multi-user mode


nit: do you think moving this section before "multi-user" mode help users absorb the knowledge gradually?

Per my imagination, first reading some simple general introduction for how to connect,
and then dive into the complex configuration for how to authenticate in multi-user mode sounds like a better learning experience.

+1, it is helpful to start with simple approach. If user cannot resolve their need with simple approach, then they will keep reading for more guidance, which naturally leads to the change in new section.

Bobgy · 2021-09-30T12:02:00Z

content/en/docs/components/pipelines/sdk/connect-api.md

+  In this scenario, both `namespace-1` and `namespace-2` require `PodDefault` to be properly configured. Access is granted by `namespace-2` adding `namespace-1` as a  
+  [contributor](https://www.kubeflow.org/docs/components/multi-tenancy/getting-started/#managing-contributors-through-the-kubeflow-ui).
+
+##### How in-cluster authentication works


Suggested change

##### How in-cluster authentication works

##### How Multi-User mode in-cluster authentication works

Bobgy · 2021-09-30T12:09:11Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+* **By sharing access to the other profile.**
+
+  In this scenario, both `namespace-1` and `namespace-2` require `PodDefault` to be properly configured. Access is granted by `namespace-2` adding `namespace-1` as a  


nit: what text should we enter in the contributor UI textbox when adding namespace-1 default-editor service account?

Bobgy · 2021-09-30T15:28:46Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+More details about PodDefault can be found [here](https://github.com/kubeflow/kubeflow/blob/master/components/admission-webhook/README.md).
+
+It is important to understand that `serviceAccountToken` method respects the Kubeflow Pipelines RBAC, and does not allow access beyond what the ServiceAcount running the notebook Pod has.


Do you think it's helpful emphasizing on the two steps: authenticate and then authorize?

service account token projection allows you to authenticate as the service account (default-editor in your namespace)

KFP API will verify whether your authenticated identity has the right permissions to do certain actions (RBAC)

My understanding is that projected token is separate from the Pod's (e.g. default-editor) service account.
I've modified this paragraph to:
"Authentication uses ServiceAccountToken projection. Simply put, the token is first being injected into a Pod (e.g. Jupyter notebook's). Then Kubeflow Pipelines SDK uses this token to authorize against Kubeflow Pipelines API."

Please @Bobgy @elikatsis @andreyvelich @thesuperzapper correct me if I'm wrong.

Bobgy · 2021-09-30T15:32:55Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+##### Managing cross-namespaces access to Kubeflow Pipelines API
+
+The setup can be extended to allow other namespaces to access Kubeflow Pipelines API through your namespace's Juputer notebook server.


Maybe I misunderstood sth, I think the setting below allows default-editor service account in namespace-1 to call KFP API and create pipelines/runs in namespace-2. (it also allows access to notebooks, but we won't use a service account to access notebooks, the goal here is to grant namespaced pipeline creation permission to a notebook in a separate namespace)

@thesuperzapper is this what you wanted to convey?

andreyvelich · 2021-10-05T15:32:38Z

regarding content, I'd argue that the main documentation should be written without dependency on PodDefault. I agree PodDefault is a nice to have, but I don't want people to treat it as a hard dependency of KFP.

I think it's good to have instructions for PodDefault until we don't have any other options to inject environments and volumeMounts to the Notebook's Pod from the Kubeflow Notebook UI.

also, can we also introduce how a general Pod can be configured to access KFP API, besides the documentation for accessing KFP API from Kubeflow notebooks?

That is a good point. @bartgras What do you think about it ?

@Bobgy Since this doc PR is very important for using KFP from the Kubeflow Notebook, should we try to merge the initial doc changes before the Kubeflow 1.4 will be released ?
We can address your comments in the following PRs and improve this documentation.

What do you think ?

zijianjoy · 2021-10-08T15:59:12Z

I think it's good to have instructions for PodDefault until we don't have any other options to inject environments and volumeMounts to the Notebook's Pod from the Kubeflow Notebook UI.

I propose to organize the doc structure as followed:

Remove the following section heading but keep the text content within it, because the whole page is about using SDK client, so we don't need to have a duplicate heading.
~~## Connect to Kubeflow Pipelines using SDK~~


## Before you begin

## Connect from outside your cluster

## Connect from a Pod within cluster

(the content here is pulled from `Non-multi-user mode` section in current PR)

## Configure SDK client by environment variables

## Connect from Jupyter Notebook in multi-user mode

How do you think?

shuesc1 · 2021-10-08T15:21:46Z

content/en/docs/components/pipelines/multi-user.md

-
-There is work-in-progress to support this use-case, refer to [github issue #5138](https://github.com/kubeflow/pipelines/issues/5138).
+Alternatively, in-cluster workloads like Jupyter notebooks or cron tasks can also access Kubeflow Pipelines API through the public endpoint. This option is platform specific and explained in 
+[Connect to Kubeflow Pipelines from outside your cluster](/docs/components/pipelines/sdk/connect-api/#connect-to-kubeflow-pipelines-from-outside-your-cluster)


add period at end

shuesc1 · 2021-10-08T15:27:44Z

content/en/docs/components/pipelines/sdk/connect-api.md

-Note, this is not supported right now for multi-user Kubeflow Pipelines, refer to [Multi-User Isolation for Pipelines -- Current Limitations](/docs/components/pipelines/multi-user/#current-limitations).
+#### Multi-User mode
+
+In order to **access Kubeflow Pipelines from Jupyter notebook**, additional per namespace (profile) manifest is required:


"additional per namespace (profile) manifest is required" -> "an additional per-namespace (profile) manifest is required"

shuesc1 · 2021-10-08T15:30:02Z

content/en/docs/components/pipelines/sdk/connect-api.md

+Note, technical details were put in the [How in-cluster authentication works](#how-in-cluster-authentication-works) section below.
+
+After the manifest is applied, newly created Jupyter notebook contains an additional option in section **configurations**.
+Read more about **configurations** on [Jupyter notebook server](/docs/components/notebooks/setup/#create-a-jupyter-notebook-server-and-add-a-notebook).


Jupyter notebook server link not working for me (but maybe I'm missing something)

shuesc1 · 2021-10-08T15:32:38Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+Note, technical details were put in the [How in-cluster authentication works](#how-in-cluster-authentication-works) section below.
+
+After the manifest is applied, newly created Jupyter notebook contains an additional option in section **configurations**.


"newly created Jupyter notebook contains an additional option in section configurations" -> "the newly created Jupyter notebook contains an additional option in the configurations section"

shuesc1 · 2021-10-08T15:33:26Z

content/en/docs/components/pipelines/sdk/connect-api.md

+Note, technical details were put in the [How in-cluster authentication works](#how-in-cluster-authentication-works) section below.
+
+After the manifest is applied, newly created Jupyter notebook contains an additional option in section **configurations**.
+Read more about **configurations** on [Jupyter notebook server](/docs/components/notebooks/setup/#create-a-jupyter-notebook-server-and-add-a-notebook).


"configurations on" -> "configurations in the"

shuesc1 · 2021-10-08T16:13:38Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+##### Managing cross-namespaces access to Kubeflow Pipelines API
+
+The setup can be extended to allow other namespaces to access Kubeflow Pipelines API through your namespace's Juputer notebook server.


shuesc1 · 2021-10-08T16:15:46Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+The setup can be extended to allow other namespaces to access Kubeflow Pipelines API through your namespace's Juputer notebook server.
+
+Note, examples below assume that `namespace-1` is a namespace (profile) that will  be granted access to Kubeflow Pipelines API 


"Note, examples below assume that namespace-1 is a namespace (profile) that will be granted access to Kubeflow Pipelines API through namespace-2 Jupyter notebook server. namespace-2 is already configured to access Kubeflow Pipelines API."
->
"Note, the examples below assume that namespace-1 is a namespace (profile) that will be granted access to Kubeflow Pipelines API
through the namespace-2 Jupyter notebook server. The namespace-2 server should already be configured to access Kubeflow Pipelines API."

shuesc1 · 2021-10-08T16:18:14Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+Setup is similar for other cases like e.g. **cron tasks or in-cluster CI/CD**: Configure your Pod with token in either `KF_PIPELINES_SA_TOKEN_PATH` or `/var/run/secrets/kubeflow/pipelines/token` and `kfp.Client` will gain access to Kubeflow Pipelines.
+
+##### Managing cross-namespaces access to Kubeflow Pipelines API


Unless this wording is used elsewhere preference for something clearer like:

Managing access to Kubeflow Pipelines API across namespaces

shuesc1 · 2021-10-08T16:24:23Z

content/en/docs/components/pipelines/sdk/connect-api.md

+Note, examples below assume that `namespace-1` is a namespace (profile) that will  be granted access to Kubeflow Pipelines API 
+through `namespace-2` Jupyter notebook server. `namespace-2` is already configured to access Kubeflow Pipelines API.
+
+The access can be achieved in two ways:


Cross-namespace access can be achieved in two ways:

shuesc1 · 2021-10-08T16:32:08Z

content/en/docs/components/pipelines/sdk/connect-api.md

+
+##### How in-cluster authentication works
+
+Authentication uses ServiceAccountToken 


confusing-- consider breaking this up for clarity. Something like:

"Authentication uses ServiceAccountToken projection. This is where a verifiable (by Kubeflow Pipelines API) token with a limited lifetime is being injected into a Pod (e.g. Jupyter notebook's)."

bartgras · 2021-10-11T03:17:06Z

Thank you @Bobgy @zijianjoy @shuesc1 @andreyvelich for detailed feedback. Summary of the changes:

I've incorporated all nitpicks
Moved "Non-multi-user mode" above "Multi-User mode"
Added @shuesc1 suggestions

@Bobgy @andreyvelich

also, can we also introduce how a general Pod can be configured to access KFP API, besides the documentation for accessing KFP API from Kubeflow notebooks?

That is a good point. @bartgras What do you think about it ?

Previous version already mentioned general Pod case but it was only briefly mentioned at the end of "Access Kubeflow Pipelines from Jupyter notebook" section. Now it's split into 2 sections, one that addresses notebooks and the other that addresses Pods in general.

zijianjoy · 2021-10-11T17:07:50Z

/lgtm
/approve

@bartgras Thank you so much for your contributions to this doc PR! Since most comments have been addressed, let's merge this PR first and follow up for any remaining items in a separate PR.

google-oss-robot · 2021-10-11T17:07:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bartgras, zijianjoy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/docs/components/pipelines/OWNERS~~ [zijianjoy]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Update docs for connecting to Kubeflow Pipelines from the same cluster

2a55423

google-cla bot added the cla: no label Aug 30, 2021

google-oss-robot requested review from alfsuse and joeliedtke August 30, 2021 16:07

google-oss-robot added the size/M label Aug 30, 2021

google-oss-robot added the needs-ok-to-test label Aug 30, 2021

Arhell reviewed Aug 31, 2021

View reviewed changes

google-cla bot added cla: yes and removed cla: no labels Aug 31, 2021

andreyvelich reviewed Sep 2, 2021

View reviewed changes

content/en/docs/components/pipelines/sdk/connect-api.md Outdated Show resolved Hide resolved

Update content/en/docs/components/pipelines/sdk/connect-api.md

e4f2a54

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

thesuperzapper requested changes Sep 5, 2021

View reviewed changes

google-oss-robot added ok-to-test and removed needs-ok-to-test labels Sep 6, 2021

This was referenced Sep 7, 2021

create "Authentication Type" selector in Kubeflow Pipelines Runtime elyra-ai/elyra#2107

Closed

support Kubeflow Pipelines KF_PIPELINES_SA_TOKEN_PATH authentication method elyra-ai/elyra#2108

Closed

Bobgy reviewed Sep 10, 2021

View reviewed changes

Bart added 2 commits September 13, 2021 18:49

Improve in-cluster access to KFP API documentation

876d7d3

merge

3905680

google-oss-robot added size/L and removed size/M labels Sep 13, 2021

Merge branch 'kubeflow:master' into multi-user-update-docs

7c7b080

google-oss-robot requested a review from Bobgy September 20, 2021 14:55

google-oss-robot requested a review from thesuperzapper September 20, 2021 14:55

google-oss-robot assigned andreyvelich Sep 20, 2021

google-oss-robot added the lgtm label Sep 20, 2021

zijianjoy reviewed Sep 28, 2021

View reviewed changes

google-oss-robot assigned zijianjoy Sep 28, 2021

google-oss-robot assigned capri-xiyue Sep 28, 2021

Bobgy reviewed Sep 30, 2021

View reviewed changes

shuesc1 reviewed Oct 8, 2021

View reviewed changes

Update documentation

3d930b7

google-oss-robot removed the lgtm label Oct 11, 2021

google-oss-robot added the lgtm label Oct 11, 2021

google-oss-robot added the approved label Oct 11, 2021

google-oss-robot merged commit 4c74471 into kubeflow:master Oct 11, 2021

thesuperzapper mentioned this pull request Oct 12, 2021

improve docs for connecting Pipelines SDK to Kubeflow #3021

Merged

zijianjoy mentioned this pull request Oct 12, 2021

pipelines: Follow up on connecting to KFP in multi-user mode. #3022

Merged

		If token is found, `kfp.Client` puts it in the `Authorization: Bearer <token>` header where Kubeflow Pipelines API can read it. No additional setup is required to refresh tokens.

		Setup is similar for other cases like e.g. cron tasks or in-cluster CI/CD: Configure your Pod with token in either `KF_PIPELINES_SA_TOKEN_PATH` or `/var/run/secrets/kubeflow/pipelines/token` and `kfp.Client` will gain access to Kubeflow Pipelines.


		Setup is similar for other cases like e.g. cron tasks or in-cluster CI/CD: Configure your Pod with token in either `KF_PIPELINES_SA_TOKEN_PATH` or `/var/run/secrets/kubeflow/pipelines/token` and `kfp.Client` will gain access to Kubeflow Pipelines.

		#### Generic


		Clients can only access the Kubeflow Pipelines API from the public endpoint


		Note, Kubeflow `kfp.Client` expects token either in `KF_PIPELINES_SA_TOKEN_PATH` environment variable or mounted to `/var/run/secrets/kubeflow/pipelines/token`. Do not change these values in the manifest. Similarly, `audience` should not be modified as well.

		If token is found, `kfp.Client` puts it in the `Authorization: Bearer <token>` header where Kubeflow Pipelines API can read it. No additional setup is required to refresh tokens.


		##### Managing cross-namespaces access to Kubeflow Pipelines API

		The setup can be extended to allow other namespaces to access Kubeflow Pipelines API through your namespace's Juputer notebook server.


		It is important to understand that `serviceAccountToken` method respects the Kubeflow Pipelines RBAC, and does not allow access beyond what the ServiceAcount running the notebook Pod has.

		#### Non-multi-user mode

	##### How in-cluster authentication works
	##### How Multi-User mode in-cluster authentication works


		* By sharing access to the other profile.

		In this scenario, both `namespace-1` and `namespace-2` require `PodDefault` to be properly configured. Access is granted by `namespace-2` adding `namespace-1` as a


		More details about PodDefault can be found [here](https://github.com/kubeflow/kubeflow/blob/master/components/admission-webhook/README.md).

		It is important to understand that `serviceAccountToken` method respects the Kubeflow Pipelines RBAC, and does not allow access beyond what the ServiceAcount running the notebook Pod has.


		Note, technical details were put in the [How in-cluster authentication works](#how-in-cluster-authentication-works) section below.

		After the manifest is applied, newly created Jupyter notebook contains an additional option in section configurations.


		The setup can be extended to allow other namespaces to access Kubeflow Pipelines API through your namespace's Juputer notebook server.

		Note, examples below assume that `namespace-1` is a namespace (profile) that will be granted access to Kubeflow Pipelines API


		##### How in-cluster authentication works

		Authentication uses ServiceAccountToken

Update docs for connecting to Kubeflow Pipelines from the same cluster in multi-user mode #2905

Update docs for connecting to Kubeflow Pipelines from the same cluster in multi-user mode #2905

Conversation

bartgras commented Aug 30, 2021

google-cla bot commented Aug 30, 2021

What to do if you already signed the CLA

Individual signers

Corporate signers

google-oss-robot commented Aug 30, 2021

Arhell left a comment

Choose a reason for hiding this comment

bartgras commented Aug 31, 2021

andreyvelich left a comment

Choose a reason for hiding this comment

Bobgy commented Sep 3, 2021

thesuperzapper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yhwang Sep 14, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andreyvelich commented Sep 6, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bartgras commented Sep 13, 2021 • edited Loading

andreyvelich commented Sep 20, 2021

andreyvelich commented Sep 24, 2021

thesuperzapper commented Sep 27, 2021

bartgras commented Sep 28, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zijianjoy left a comment

Choose a reason for hiding this comment

capri-xiyue commented Sep 28, 2021

bartgras commented Sep 29, 2021

Bobgy commented Sep 30, 2021

Bobgy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andreyvelich commented Oct 5, 2021

zijianjoy commented Oct 8, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Managing access to Kubeflow Pipelines API across namespaces

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bartgras commented Oct 11, 2021

zijianjoy commented Oct 11, 2021

google-oss-robot commented Oct 11, 2021

yhwang Sep 14, 2021 •

edited

Loading

bartgras commented Sep 13, 2021 •

edited

Loading