From 963aa40829045a349c8f78d4fa3dd10ffca46282 Mon Sep 17 00:00:00 2001 From: Xinru Zhang Date: Wed, 7 Dec 2022 17:51:27 -0500 Subject: [PATCH] TEP-0114: Migration Doc for Custom Task Beta Promotion Prior to this commit, we don't have docs for how to migrate custom task from alpha version to beta. This commits bridge the gap and updates runs.md, customruns.md and pipelines.md to keep the related instruction up-to-date. --- docs/customruns.md | 2 +- ...ating-v1alpha1.Run-to-v1beta1.CustomRun.md | 194 ++++++++++++++++++ docs/pipelines.md | 109 +++++++++- docs/runs.md | 2 +- 4 files changed, 294 insertions(+), 13 deletions(-) create mode 100644 docs/migrating-v1alpha1.Run-to-v1beta1.CustomRun.md diff --git a/docs/customruns.md b/docs/customruns.md index 9b20d69761d..dbdfdc0d50e 100644 --- a/docs/customruns.md +++ b/docs/customruns.md @@ -26,7 +26,7 @@ weight: 800 # Overview -*`CustomRun` is the `Beta` version of [`Run`](runs.md).* +*We are promoting Custom Task from [`v1alpha1.Run`](runs.md) to `v1beta1.CustomRun`, please refer to the [migration doc](migrating-v1alpha1.Run-to-v1beta1.CustomRun.md) for details.* A `CustomRun` allows you to instantiate and execute a [Custom Task](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md), diff --git a/docs/migrating-v1alpha1.Run-to-v1beta1.CustomRun.md b/docs/migrating-v1alpha1.Run-to-v1beta1.CustomRun.md new file mode 100644 index 00000000000..9c9ec9b9336 --- /dev/null +++ b/docs/migrating-v1alpha1.Run-to-v1beta1.CustomRun.md @@ -0,0 +1,194 @@ + + +# Migrating from v1alpha1.Run to v1beta1.CustomRun + +This document describes how to migrate from `v1alpha1.Run` and `v1beta1.CustomRun` +- [Changes to fields](#changes-to-fields) + - [Changes to the specification](#changes-to-the-specification) + - [Changes to the reference](#changes-to-the-reference) + - [Support `PodTemplate` in Custom Task Spec](#support-podtemplate-in-custom-task-spec) +- [Changes to implementation requirements](#changes-to-implementation-instructions) + - [Status Reporting](#status-reporting) + - [Cancellation](#cancellation) + - [Timeout](#timeout) + - [Retries & RetriesStatus](#4-retries--retriesstatus) +- [New feature flag `custom-task-version` for migration](#new-feature-flag-custom-task-version) + +## Changes to fields + +Comparing `v1alpha1.Run` with `v1beta1.CustomRun`, the following fields have been changed: + +| Old field | New field | +| ---------------------- | ------------------| +| `spec.spec` | [`spec.customSpec`](#changes-to-the-specification) | +| `spec.ref` | [`spec.customRef`](#changes-to-the-reference) | +| `spec.podTemplate` | removed, see [this section](#support-podtemplate-in-custom-task-spec) if you still want to support it| + + +### Changes to the specification + +In `v1beta1.CustomRun`, the specification is renamed from `spec.spec` to `spec.customSpec`. + +```yaml +# before (v1alpha1.Run) +apiVersion: tekton.dev/v1alpha1 +kind: Run +metadata: + name: run-with-spec +spec: + spec: + apiVersion: example.dev/v1alpha1 + kind: Example + spec: + field1: value1 + field2: value2 +# after (v1beta1.CustomRun) +apiVersion: tekton.dev/v1beta1 +kind: CustomRun +metadata: + name: customrun-with-spec +spec: + customSpec: + apiVersion: example.dev/v1beta1 + kind: Example + spec: + field1: value1 + field2: value2 +``` + +### Changes to the reference + +In `v1beta1.CustomRun`, the specification is renamed from `spec.ref` to `spec.customRef`. + +```yaml +# before (v1alpha1.Run) +apiVersion: tekton.dev/v1alpha1 +kind: Run +metadata: + name: run-with-reference +spec: + ref: + apiVersion: example.dev/v1alpha1 + kind: Example + name: my-example +--- +# after (v1beta1.CustomRun) +apiVersion: tekton.dev/v1beta1 +kind: CustomRun +metadata: + name: customrun-with-reference +spec: + customRef: + apiVersion: example.dev/v1beta1 + kind: Example + name: my-customtask +``` + +### Support `PodTemplate` in Custom Task Spec + +`spec.podTemplate` is removed in `v1beta1.CustomRun`. You can support that field in your own custom task spec if you want to. For example: + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: CustomRun +metadata: + name: customrun-with-podtemplate +spec: + customSpec: + apiVersion: example.dev/v1beta1 + kind: Example + spec: + podTemplate: + securityContext: + runAsUser: 1001 +``` + +## Changes to implementation instructions +We've changed four implementation instructions. Note that `status reporting` is the only required instruction to follow, others are recommendations. + +### Status Reporting + +You **MUST** report `v1beta1.CustomRun` as `Done` (set its `Conditions.Succeeded` status as `True` or `False`) **ONLY** when you want Pipeline Controller consider it as finished. + +For example, if the `CustomRun` failed, while it still has remaining `retries`. If you want pipeline controller to continue watching its status changing, you **MUST NOT** mark it as `Done`. Otherwise, the `PipelineRun` controller may consider it as finished. + +Here are statuses that indicating the `CustomRun` is done. +```yaml +Type: Succeeded +Status: False +Reason: TimedOut +``` +```yaml +Type: Succeeded +Status: True +Reason: Succeeded +``` + +### Cancellation + +Custom Task implementors are responsible for implementing `cancellation` to **support pipelineRun level timeouts and cancellation**. If a Custom Task implementor does not support cancellation via `customRun.spec.status`, `PipelineRun` can not timeout within the specified interval/duration and can not be cancelled as expected upon request. + +It is recommended to update the `CustomRun` status as following, once noticing `customRun.Spec.Status` is updated to `RunCancelled` + +```yaml +Type: Succeeded +Status: False +Reason: CustomRunCancelled +``` + +### Timeout + +We recommend `customRun.Timeout` to be set for each retry attempt instead of all retries. That's said, if one `CustomRun` execution fails on `Timeout` and it has remaining retries, the `CustomRun` controller SHOULD NOT set the status of that `CustomRun` as `False`. Instead, it SHOULD initiate another round of execution. + +### 4. Retries & RetriesStatus + +We recommend you use `customRun.Spec.Retries` if you want to implement the `retry` logic for your `Custom Task`, and archive history `customRun.Status` in `customRun.Status.RetriesStatus`. + +Say you started a `CustomRun` by setting the following condition: +```yaml +Status: + Conditions: + - Type: Succeeded + Status: Unknown + Reason: Running +``` +Now it failed for some reasons but has remaining retries, instead of setting Conditions as failed on TimedOut: +```yaml +Status: + Conditions: + - Type: Succeeded + Status: False + Reason: Failed +``` +We **recommend** you to do archive the failure status to `customRun.Status.retriesStatus`, and keep setting `customRun.Status` as `Unknown`: +```yaml +Status: + Conditions: + - Type: Succeeded + Status: Unknown + Reason: "xxx" + RetriesStatus: + - Conditions: + - Type: Succeeded + Status: False + Reason: Failed +``` + + +## New feature flag `custom-task-version` for migration + +You can use `custom-task-version` to control whether `v1alpha1.Run` or `v1beta1.CustomRun` should be created when a `Custom Task` is specified in a `Pipeline`. The feature flag currently supports two values: `v1alpha1` and `v1beta1`. + +We'll change its default value per the following timeline: +- v0.43.*: default to `v1alpha1`. +- v0.44.*: switch the default to `v1beta1` +- v0.47.*: remove the feature flag, stop supporting creating `v1alpha1.Run` + + +[cancel-pr]: https://github.com/tektoncd/pipeline/blob/main/docs/pipelineruns.md#cancelling-a-pipelinerun +[gracefully-cancel-pr]: (https://github.com/tektoncd/pipeline/blob/main/docs/pipelineruns.md#gracefully-cancelling-a-pipelinerun) \ No newline at end of file diff --git a/docs/pipelines.md b/docs/pipelines.md index f4ca7ea2c4f..600da4373a2 100644 --- a/docs/pipelines.md +++ b/docs/pipelines.md @@ -58,7 +58,10 @@ weight: 400 - [Specifying matrix](#specifying-matrix) - [Specifying workspaces](#specifying-workspaces-1) - [Using `Results`](#using-results-1) + - [Specifying `Timeout`](#specifying-timeout) + - [Specifying `Retries`](#specifying-retries) - [Limitations](#limitations) + - [Known Custom Tasks](#known-custom-tasks) - [Code examples](#code-examples) ## Overview @@ -1564,17 +1567,18 @@ no `runAfter` can be specified in `finally` tasks. ## Using Custom Tasks +We are promoting Custom Task from `v1alpha1` to `v1beta1`. Starting from `v0.43.0`, Pipeline Controller is able to create either `v1alpha1` or `v1beta1` Custom Task gated by a feature flag `custom-task-version`, defaulting to `v1alpha1`. You can set `custom-task-version` to `v1alpha1` or `v1beta1` to control which version to create. + +We'll switch the default value of `custom-task-version` to `v1beta1` in release v0.44.0, and remove the flag as well as the entire alpha version in release v0.47.0. See the [migration doc](migrating-v1alpha1.Run-to-v1beta1.CustomRun.md) for details. + [Custom Tasks](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md) can implement behavior that doesn't correspond directly to running a workload in a `Pod` on the cluster. For example, a custom task might execute some operation outside of the cluster and wait for its execution to complete. -A PipelineRun starts a custom task by creating a [`Run`](https://github.com/tektoncd/pipeline/blob/main/docs/runs.md) instead of a `TaskRun`. +A `PipelineRun` starts a custom task by creating a [`Run`](https://github.com/tektoncd/pipeline/blob/main/docs/runs.md)/[`CustomRun`](https://github.com/tektoncd/pipeline/blob/main/docs/customruns.md) instead of a `TaskRun`. In order for a custom task to execute, there must be a custom task controller running on the cluster -that is responsible for watching and updating `Run`s which reference their type. -If no such controller is running, those `Run`s will never complete and Pipelines using them will time out. - -Custom tasks are an **_experimental alpha feature_** and should be expected to change -in breaking ways or even be removed. +that is responsible for watching and updating `Run/CustomRun`s which reference their type. +If no such controller is running, those `Run/CustomRun`s will never complete and Pipelines using them will time out. ### Specifying the target Custom Task @@ -1590,7 +1594,7 @@ spec: kind: Example ``` -This creates a `Run` of a custom task of type `Example` in the `example.dev` API group with the version `v1alpha1`. +This creates a `Run/CustomRun` of a custom task of type `Example` in the `example.dev` API group with the version `v1alpha1`. You can also specify the `name` of a custom task resource object previously defined in the cluster. @@ -1612,6 +1616,7 @@ some default behavior for executing unnamed tasks. ### Specifying a Custom Task Spec in-line (or embedded) +**For `v1alpha1.Run`** ```yaml spec: tasks: @@ -1624,13 +1629,28 @@ spec: field2: value2 ``` -If the custom task controller supports the in-line or embedded task spec, this will create a `Run` of a custom task of +**For `v1beta1.CustomRun`** +```yaml +spec: + tasks: + - name: run-custom-task + taskSpec: + apiVersion: example.dev/v1alpha1 + kind: Example + customSpec: + field1: value1 + field2: value2 +``` + +If the custom task controller supports the in-line or embedded task spec, this will create a `Run/CustomRun` of a custom task of type `Example` in the `example.dev` API group with the version `v1alpha1`. If the `taskSpec` is not supported, the custom task controller should produce proper validation errors. Please take a look at the -[developer guide for custom controllers supporting `taskSpec`.](runs.md#developer-guide-for-custom-controllers-supporting-spec) +developer guide for custom controllers supporting `taskSpec`: +- [guidance for `Run`](runs.md#developer-guide-for-custom-controllers-supporting-spec) +- [guidance for `CustomRun`](customruns.md#developer-guide-for-custom-controllers-supporting-customspec) `taskSpec` support for `pipelineRun` was designed and discussed in [TEP-0061](https://github.com/tektoncd/community/blob/main/teps/0061-allow-custom-task-to-be-embedded-in-pipeline.md) @@ -1707,12 +1727,79 @@ Consult the documentation of the custom task that you are using to determine whe If the custom task produces results, you can reference them in a Pipeline using the normal syntax, `$(tasks..results.)`. +### Specifying `Timeout` + +#### `v1alpha1.Run` +If the custom task supports it as [we recommended](runs.md#developer-guide-for-custom-controllers-supporting-timeout), you can provide `timeout` to specify the maximum running time of a `CustomRun` (including all retry attempts or other operations). + +#### `v1beta1.CustomRun` +If the custom task supports it as [we recommended](customruns.md#developer-guide-for-custom-controllers-supporting-timeout), you can provide `timeout` to specify the maximum running time of one `CustomRun` execution. + +```yaml +spec: + tasks: + - name: run-custom-task + timeout: 2s + taskRef: + apiVersion: example.dev/v1alpha1 + kind: Example + name: myexample +``` + +Consult the documentation of the custom task that you are using to determine whether it supports `Timeout`. + +### Specifying `Retries` +If the custom task supports it, you can provide `retries` to specify how many times you want to retry the custom task. + +```yaml +spec: + tasks: + - name: run-custom-task + retries: 2 + taskRef: + apiVersion: example.dev/v1alpha1 + kind: Example + name: myexample +``` + +Consult the documentation of the custom task that you are using to determine whether it supports `Retries`. + ### Limitations Pipelines do not support the following items with custom tasks: * Pipeline Resources -* [`retries`](#using-the-retries-field) -* [`timeout`](#configuring-the-failure-timeout) + +### Known Custom Tasks + +We try to list as many known Custom Tasks as possible here so that users can easily find what they want. Please feel free to share the Custom Task you implemented in this table. + +#### v1beta1.CustomRun + +| Custom Task | Description | +|:---|:--- | +| [Wait Task Beta][wait-task-beta] | Waits a given amount of time before succeeding, specified by an input parameter named duration. Support `timeout` and `retries`. | + +#### v1alpha1.Run + +| Custom Task | Description | +|:---|:--- | +| [Pipeline Loops][pipeline-loops]| Runs a `Pipeline` in a loop with varying `Parameter` values. +| [Common Expression Language][cel]| Provides Common Expression Language support in Tekton Pipelines. +| [Wait][wait]| Waits a given amount of time, specified by a `Parameter` named "duration", before succeeding. +| [Approvals][approvals]| Pauses the execution of `PipelineRuns` and waits for manual approvals. +| [Pipelines in Pipelines][pipelines-in-pipelines]| Defines and executes a `Pipeline` in a `Pipeline`. +| [Task Group][task-group]| Groups `Tasks` together as a `Task`. +| [Pipeline in a Pod][pipeline-in-pod]| Runs `Pipeline` in a `Pod`. + +[pipeline-loops]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/pipeline-loops +[task-loops]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/task-loops +[cel]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/cel +[wait]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/wait-task +[approvals]: https://github.com/automatiko-io/automatiko-approval-task/tree/71da90361dff9444146d52d0a6e2b542d4bf4edc +[task-group]: https://github.com/openshift-pipelines/tekton-task-group/tree/39823f26be8f59504f242a45b9f2e791d4b36e1c +[pipelines-in-pipelines]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/pipelines-in-pipelines +[pipeline-in-pod]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/pipeline-in-pod +[wait-task-beta]: https://github.com/tektoncd/pipeline/tree/a127323da31bcb933a04a6a1b5dbb6e0411e3dc1/test/custom-task-ctrls/wait-task-beta ## Code examples diff --git a/docs/runs.md b/docs/runs.md index dfdcb1c89aa..6173a9bd38f 100644 --- a/docs/runs.md +++ b/docs/runs.md @@ -21,7 +21,7 @@ weight: 800 # Overview -*We've released the Beta version of `Run`: `CustomRun`, check [this page](customruns.md) for details.* +*We are promoting Custom Task from `v1alpha1.Run` to [`v1beta1.CustomRun`](customruns.md), please refer to the [migration doc](migrating-v1alpha1.Run-to-v1beta1.CustomRun.md) for details.* A `Run` allows you to instantiate and execute a [Custom Task](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md),