diff --git a/manifests/gcp_marketplace/guide.md b/manifests/gcp_marketplace/guide.md index 10d40960ac3..bd8fb372570 100644 --- a/manifests/gcp_marketplace/guide.md +++ b/manifests/gcp_marketplace/guide.md @@ -43,6 +43,7 @@ gcloud projects add-iam-policy-binding $PROJECT_ID \ --member=serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \ --role=roles/storage.admin +# Note that you can not bind multiple roles in one line. gcloud projects add-iam-policy-binding $PROJECT_ID \ --member=serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \ --role=roles/ml.admin @@ -98,3 +99,25 @@ as `Service Account User`. The Google Service Account is [Compute Engine default - Please also add your account as `Project Viewer` via [IAM](https://console.cloud.google.com/iam-admin/iam). For simplicity but not good for security, adding as `Project Editor` also can work. + +### Pipeline steps got insufficient permission +If you see an error message stating that the pipeline got insufficient +permissions, for example: + +``` + Error executing an HTTP request: HTTP response code 403 with body '{ + "error": { + "errors": [ + { + "domain": "global", + "reason": "insufficientPermissions", + "message": "Insufficient Permission" + } + ], + "code": 403, + "message": "Insufficient Permission" + } +} +``` +please make sure following the procedure in [credential setup](#gcp-service-account-credentials). IAM configuration and/or + API enabling might take up to 5 mins to propagate. diff --git a/samples/contrib/parameterized_tfx_oss/README.md b/samples/contrib/parameterized_tfx_oss/README.md index f53718382d6..a29d3e563ee 100644 --- a/samples/contrib/parameterized_tfx_oss/README.md +++ b/samples/contrib/parameterized_tfx_oss/README.md @@ -1,4 +1,4 @@ -# Parameterized TFX pipeline sample +# Overview [Tensorflow Extended (TFX)](https://github.com/tensorflow/tfx) is a Google-production-scale machine learning platform based on TensorFlow. It provides a configuration framework to express ML pipelines @@ -9,15 +9,19 @@ This sample demonstrates how to author a ML pipeline in TFX and run it on a KFP Please refer to inline comments for the purpose of each step. In order to successfully compile this sample, you'll need to have a TFX installation at HEAD. -First, you can clone their repo and run `python setup.py install` from `tfx/tfx`. +First, you can clone their repo and run `python setup.py install` from `tfx/`. The image used in the pipeline is specified as `tfx_image` in the `KubeflowDagRunnerConfig`. Currently we're using our own patched version of TFX image containing visualization support. List of officially released nightly build image available can be found [here](https://hub.docker.com/r/tensorflow/tfx/tags)). After that, running `python3 chicago_taxi_pipeline_simple.py` compiles the TFX pipeline into KFP pipeline package. -This pipeline requires google storage permission to run. +# Permission + +This pipeline requires Google Cloud Storage permission to run. +If KFP was deployed through K8S marketplace, please follow instructions in [the guideline](https://github.com/kubeflow/pipelines/blob/master/manifests/gcp_marketplace/guide.md#gcp-service-account-credentials) +to make sure the service account has `storage.admin` role. ## Caveats @@ -28,8 +32,7 @@ objects `dsl.PipelineParam` and appending them to the `KubeflowDagRunner._params KubeflowDagRunner can correctly identify those pipeline parameters and interpret them as Argo placeholder correctly when compilation. However, this parameterization approach is a hack and we do not have plan for long-term support. Instead we're working with TFX team to support -pipeline parameterization using their [RuntimeParameter](https://github.com/tensorflow/tfx/blob/46bb4f975c36ea1defde4b3c33553e088b3dc5b8/tfx/orchestration/data_types.py#L108). - +pipeline parameterization using their [RuntimeParameter](https://github.com/tensorflow/tfx/blob/46bb4f975c36ea1defde4b3c33553e088b3dc5b8/tfx/orchestration/data_types.py#L108). ### Known issues * This approach only works for string-typed quantities. For example, you cannot parameterize `num_steps` of `Trainer` in this way. diff --git a/samples/core/xgboost_training_cm/README.md b/samples/core/xgboost_training_cm/README.md index 3469fac2afc..7cb6e3dbf34 100644 --- a/samples/core/xgboost_training_cm/README.md +++ b/samples/core/xgboost_training_cm/README.md @@ -12,8 +12,11 @@ or not. ## Requirements -Preprocessing uses Google Cloud DataProc. Therefore, you must enable the [DataProc API](https://cloud.google.com/endpoints/docs/openapi/enable-api) for the given GCP project. - +Preprocessing uses Google Cloud DataProc. Therefore, you must enable the +[Cloud Dataproc API](https://pantheon.corp.google.com/apis/library/dataproc.googleapis.com?q=dataproc) for the given GCP project. This is the +general [guideline](https://cloud.google.com/endpoints/docs/openapi/enable-api) for enabling GCP APIs. +If KFP was deployed through K8S marketplace, please follow instructions in [the guideline](https://github.com/kubeflow/pipelines/blob/master/manifests/gcp_marketplace/guide.md#gcp-service-account-credentials) +to make sure the service account used has the role `storage.admin` and `dataproc.admin`. ## Compile