Improve doc for gcp components. #1049
Conversation
|
/test kubeflow-pipeline-e2e-test |
| # Required Parameters | ||
| PROJECT_ID = '<Please put your project ID here>' | ||
| GCS_WORKING_DIR = 'gs://<Please put your GCS path here>' # No ending slash | ||
| %%capture |
There was a problem hiding this comment.
What is %%capture command?
There was a problem hiding this comment.
This is used to hide the outputs from the cell. It's usually not quite useful to show pip install logs here.
There was a problem hiding this comment.
Maybe we still need errors to be output.
https://ipython.readthedocs.io/en/stable/interactive/magics.html#cellmagic-capture shows a way to not capture stderr.
however, when I tried it in my notebook with !pip install, it does not work.
There was a problem hiding this comment.
BTW, pip has the --quiet option.
| @@ -33,20 +34,23 @@ inputs: | |||
| default: '' | |||
| type: String | |||
| - name: output_gcs_path | |||
| description: 'The GCS blob path to dump the query results to.' | |||
| description: 'The path to the Cloud Storage bucket to store the query output.' | |||
There was a problem hiding this comment.
To be honest, the word "bucket" is confusing here. You've probably meant "The path of GCS directory" or "The path of GCS file".
There was a problem hiding this comment.
This is a suggested change from tech writer. I'd like to follow their suggestion to make it consistent with other AIHub docs.
There was a problem hiding this comment.
OK.
We should probably still tell them that technically, a bucket is my-bucket while gs://my-bucket/some/dir/ is a GCS directory path, not a bucket.
| !pip3 install $KFP_PACKAGE --upgrade | ||
| ``` | ||
|
|
||
| 2. Load the component by DSL |
There was a problem hiding this comment.
"Load the component using KFP SDK"?
| "EXPERIMENT_NAME = 'Dataflow - Launch Template'\n", | ||
| "COMPONENT_SPEC_URI = 'https://raw.githubusercontent.com/kubeflow/pipelines/d2f5cc92a46012b9927209e2aaccab70961582dc/components/gcp/dataflow/launch_template/component.yaml'" | ||
| "COMPONENT_SPEC_URI = 'https://raw.githubusercontent.com/kubeflow/pipelines/d2f5cc92a46012b9927209e2aaccab70961582dc/components/gcp/dataflow/launch_template/component.yaml'\n", | ||
| "dataflow_template_op = comp.load_component_from_url(COMPONENT_SPEC_URI)\n", |
There was a problem hiding this comment.
Do we really need two lines to load component? Having a single line will make it much easier to copy/paste to a multi-component pipeline (otherwise, the COMPONENT_SPEC_URI variable name will be duplicated).
|
Such a big PR =) |
|
/lgtm |
|
/lgtm |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hongye-sun The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
1 similar comment
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hongye-sun The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
* update tekton to 0.38.4 * update tekton to 0.38.4
The change includes improvements on all GCP OOB component docs.
This change is