Skip to content

Commit

Permalink
TEP-0114: Migration Doc for Custom Task Beta Promotion
Browse files Browse the repository at this point in the history
Prior to this commit, we don't have docs for how to migrate custom
task from alpha version to beta. This commits bridge the gap and updates
runs.md, customruns.md and pipelines.md to keep the related instruction
up-to-date.
  • Loading branch information
XinruZhang authored and tekton-robot committed Dec 12, 2022
1 parent f9021b1 commit 963aa40
Show file tree
Hide file tree
Showing 4 changed files with 294 additions and 13 deletions.
2 changes: 1 addition & 1 deletion docs/customruns.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ weight: 800

# Overview

*`CustomRun` is the `Beta` version of [`Run`](runs.md).*
*We are promoting Custom Task from [`v1alpha1.Run`](runs.md) to `v1beta1.CustomRun`, please refer to the [migration doc](migrating-v1alpha1.Run-to-v1beta1.CustomRun.md) for details.*

A `CustomRun` allows you to instantiate and execute a [Custom
Task](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md),
Expand Down
194 changes: 194 additions & 0 deletions docs/migrating-v1alpha1.Run-to-v1beta1.CustomRun.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
<!--
---
linkTitle: "Migrating from v1alpha1.Run to v1beta1.CustomRun"
weight: 4000
---
-->

# Migrating from v1alpha1.Run to v1beta1.CustomRun

This document describes how to migrate from `v1alpha1.Run` and `v1beta1.CustomRun`
- [Changes to fields](#changes-to-fields)
- [Changes to the specification](#changes-to-the-specification)
- [Changes to the reference](#changes-to-the-reference)
- [Support `PodTemplate` in Custom Task Spec](#support-podtemplate-in-custom-task-spec)
- [Changes to implementation requirements](#changes-to-implementation-instructions)
- [Status Reporting](#status-reporting)
- [Cancellation](#cancellation)
- [Timeout](#timeout)
- [Retries & RetriesStatus](#4-retries--retriesstatus)
- [New feature flag `custom-task-version` for migration](#new-feature-flag-custom-task-version)

## Changes to fields

Comparing `v1alpha1.Run` with `v1beta1.CustomRun`, the following fields have been changed:

| Old field | New field |
| ---------------------- | ------------------|
| `spec.spec` | [`spec.customSpec`](#changes-to-the-specification) |
| `spec.ref` | [`spec.customRef`](#changes-to-the-reference) |
| `spec.podTemplate` | removed, see [this section](#support-podtemplate-in-custom-task-spec) if you still want to support it|


### Changes to the specification

In `v1beta1.CustomRun`, the specification is renamed from `spec.spec` to `spec.customSpec`.

```yaml
# before (v1alpha1.Run)
apiVersion: tekton.dev/v1alpha1
kind: Run
metadata:
name: run-with-spec
spec:
spec:
apiVersion: example.dev/v1alpha1
kind: Example
spec:
field1: value1
field2: value2
# after (v1beta1.CustomRun)
apiVersion: tekton.dev/v1beta1
kind: CustomRun
metadata:
name: customrun-with-spec
spec:
customSpec:
apiVersion: example.dev/v1beta1
kind: Example
spec:
field1: value1
field2: value2
```
### Changes to the reference
In `v1beta1.CustomRun`, the specification is renamed from `spec.ref` to `spec.customRef`.

```yaml
# before (v1alpha1.Run)
apiVersion: tekton.dev/v1alpha1
kind: Run
metadata:
name: run-with-reference
spec:
ref:
apiVersion: example.dev/v1alpha1
kind: Example
name: my-example
---
# after (v1beta1.CustomRun)
apiVersion: tekton.dev/v1beta1
kind: CustomRun
metadata:
name: customrun-with-reference
spec:
customRef:
apiVersion: example.dev/v1beta1
kind: Example
name: my-customtask
```

### Support `PodTemplate` in Custom Task Spec

`spec.podTemplate` is removed in `v1beta1.CustomRun`. You can support that field in your own custom task spec if you want to. For example:

```yaml
apiVersion: tekton.dev/v1beta1
kind: CustomRun
metadata:
name: customrun-with-podtemplate
spec:
customSpec:
apiVersion: example.dev/v1beta1
kind: Example
spec:
podTemplate:
securityContext:
runAsUser: 1001
```

## Changes to implementation instructions
We've changed four implementation instructions. Note that `status reporting` is the only required instruction to follow, others are recommendations.

### Status Reporting

You **MUST** report `v1beta1.CustomRun` as `Done` (set its `Conditions.Succeeded` status as `True` or `False`) **ONLY** when you want Pipeline Controller consider it as finished.

For example, if the `CustomRun` failed, while it still has remaining `retries`. If you want pipeline controller to continue watching its status changing, you **MUST NOT** mark it as `Done`. Otherwise, the `PipelineRun` controller may consider it as finished.

Here are statuses that indicating the `CustomRun` is done.
```yaml
Type: Succeeded
Status: False
Reason: TimedOut
```
```yaml
Type: Succeeded
Status: True
Reason: Succeeded
```

### Cancellation

Custom Task implementors are responsible for implementing `cancellation` to **support pipelineRun level timeouts and cancellation**. If a Custom Task implementor does not support cancellation via `customRun.spec.status`, `PipelineRun` can not timeout within the specified interval/duration and can not be cancelled as expected upon request.

It is recommended to update the `CustomRun` status as following, once noticing `customRun.Spec.Status` is updated to `RunCancelled`

```yaml
Type: Succeeded
Status: False
Reason: CustomRunCancelled
```

### Timeout

We recommend `customRun.Timeout` to be set for each retry attempt instead of all retries. That's said, if one `CustomRun` execution fails on `Timeout` and it has remaining retries, the `CustomRun` controller SHOULD NOT set the status of that `CustomRun` as `False`. Instead, it SHOULD initiate another round of execution.

### 4. Retries & RetriesStatus

We recommend you use `customRun.Spec.Retries` if you want to implement the `retry` logic for your `Custom Task`, and archive history `customRun.Status` in `customRun.Status.RetriesStatus`.

Say you started a `CustomRun` by setting the following condition:
```yaml
Status:
Conditions:
- Type: Succeeded
Status: Unknown
Reason: Running
```
Now it failed for some reasons but has remaining retries, instead of setting Conditions as failed on TimedOut:
```yaml
Status:
Conditions:
- Type: Succeeded
Status: False
Reason: Failed
```
We **recommend** you to do archive the failure status to `customRun.Status.retriesStatus`, and keep setting `customRun.Status` as `Unknown`:
```yaml
Status:
Conditions:
- Type: Succeeded
Status: Unknown
Reason: "xxx"
RetriesStatus:
- Conditions:
- Type: Succeeded
Status: False
Reason: Failed
```


## New feature flag `custom-task-version` for migration

You can use `custom-task-version` to control whether `v1alpha1.Run` or `v1beta1.CustomRun` should be created when a `Custom Task` is specified in a `Pipeline`. The feature flag currently supports two values: `v1alpha1` and `v1beta1`.

We'll change its default value per the following timeline:
- v0.43.*: default to `v1alpha1`.
- v0.44.*: switch the default to `v1beta1`
- v0.47.*: remove the feature flag, stop supporting creating `v1alpha1.Run`


[cancel-pr]: https://github.com/tektoncd/pipeline/blob/main/docs/pipelineruns.md#cancelling-a-pipelinerun
[gracefully-cancel-pr]: (https://github.com/tektoncd/pipeline/blob/main/docs/pipelineruns.md#gracefully-cancelling-a-pipelinerun)
109 changes: 98 additions & 11 deletions docs/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,10 @@ weight: 400
- [Specifying matrix](#specifying-matrix)
- [Specifying workspaces](#specifying-workspaces-1)
- [Using `Results`](#using-results-1)
- [Specifying `Timeout`](#specifying-timeout)
- [Specifying `Retries`](#specifying-retries)
- [Limitations](#limitations)
- [Known Custom Tasks](#known-custom-tasks)
- [Code examples](#code-examples)

## Overview
Expand Down Expand Up @@ -1564,17 +1567,18 @@ no `runAfter` can be specified in `finally` tasks.

## Using Custom Tasks

We are promoting Custom Task from `v1alpha1` to `v1beta1`. Starting from `v0.43.0`, Pipeline Controller is able to create either `v1alpha1` or `v1beta1` Custom Task gated by a feature flag `custom-task-version`, defaulting to `v1alpha1`. You can set `custom-task-version` to `v1alpha1` or `v1beta1` to control which version to create.

We'll switch the default value of `custom-task-version` to `v1beta1` in release v0.44.0, and remove the flag as well as the entire alpha version in release v0.47.0. See the [migration doc](migrating-v1alpha1.Run-to-v1beta1.CustomRun.md) for details.

[Custom Tasks](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md)
can implement behavior that doesn't correspond directly to running a workload in a `Pod` on the cluster.
For example, a custom task might execute some operation outside of the cluster and wait for its execution to complete.

A PipelineRun starts a custom task by creating a [`Run`](https://github.com/tektoncd/pipeline/blob/main/docs/runs.md) instead of a `TaskRun`.
A `PipelineRun` starts a custom task by creating a [`Run`](https://github.com/tektoncd/pipeline/blob/main/docs/runs.md)/[`CustomRun`](https://github.com/tektoncd/pipeline/blob/main/docs/customruns.md) instead of a `TaskRun`.
In order for a custom task to execute, there must be a custom task controller running on the cluster
that is responsible for watching and updating `Run`s which reference their type.
If no such controller is running, those `Run`s will never complete and Pipelines using them will time out.

Custom tasks are an **_experimental alpha feature_** and should be expected to change
in breaking ways or even be removed.
that is responsible for watching and updating `Run/CustomRun`s which reference their type.
If no such controller is running, those `Run/CustomRun`s will never complete and Pipelines using them will time out.

### Specifying the target Custom Task

Expand All @@ -1590,7 +1594,7 @@ spec:
kind: Example
```

This creates a `Run` of a custom task of type `Example` in the `example.dev` API group with the version `v1alpha1`.
This creates a `Run/CustomRun` of a custom task of type `Example` in the `example.dev` API group with the version `v1alpha1`.

You can also specify the `name` of a custom task resource object previously defined in the cluster.

Expand All @@ -1612,6 +1616,7 @@ some default behavior for executing unnamed tasks.

### Specifying a Custom Task Spec in-line (or embedded)

**For `v1alpha1.Run`**
```yaml
spec:
tasks:
Expand All @@ -1624,13 +1629,28 @@ spec:
field2: value2
```

If the custom task controller supports the in-line or embedded task spec, this will create a `Run` of a custom task of
**For `v1beta1.CustomRun`**
```yaml
spec:
tasks:
- name: run-custom-task
taskSpec:
apiVersion: example.dev/v1alpha1
kind: Example
customSpec:
field1: value1
field2: value2
```

If the custom task controller supports the in-line or embedded task spec, this will create a `Run/CustomRun` of a custom task of
type `Example` in the `example.dev` API group with the version `v1alpha1`.

If the `taskSpec` is not supported, the custom task controller should produce proper validation errors.

Please take a look at the
[developer guide for custom controllers supporting `taskSpec`.](runs.md#developer-guide-for-custom-controllers-supporting-spec)
developer guide for custom controllers supporting `taskSpec`:
- [guidance for `Run`](runs.md#developer-guide-for-custom-controllers-supporting-spec)
- [guidance for `CustomRun`](customruns.md#developer-guide-for-custom-controllers-supporting-customspec)

`taskSpec` support for `pipelineRun` was designed and discussed in
[TEP-0061](https://github.com/tektoncd/community/blob/main/teps/0061-allow-custom-task-to-be-embedded-in-pipeline.md)
Expand Down Expand Up @@ -1707,12 +1727,79 @@ Consult the documentation of the custom task that you are using to determine whe
If the custom task produces results, you can reference them in a Pipeline using the normal syntax,
`$(tasks.<task-name>.results.<result-name>)`.

### Specifying `Timeout`

#### `v1alpha1.Run`
If the custom task supports it as [we recommended](runs.md#developer-guide-for-custom-controllers-supporting-timeout), you can provide `timeout` to specify the maximum running time of a `CustomRun` (including all retry attempts or other operations).

#### `v1beta1.CustomRun`
If the custom task supports it as [we recommended](customruns.md#developer-guide-for-custom-controllers-supporting-timeout), you can provide `timeout` to specify the maximum running time of one `CustomRun` execution.

```yaml
spec:
tasks:
- name: run-custom-task
timeout: 2s
taskRef:
apiVersion: example.dev/v1alpha1
kind: Example
name: myexample
```

Consult the documentation of the custom task that you are using to determine whether it supports `Timeout`.

### Specifying `Retries`
If the custom task supports it, you can provide `retries` to specify how many times you want to retry the custom task.

```yaml
spec:
tasks:
- name: run-custom-task
retries: 2
taskRef:
apiVersion: example.dev/v1alpha1
kind: Example
name: myexample
```

Consult the documentation of the custom task that you are using to determine whether it supports `Retries`.

### Limitations

Pipelines do not support the following items with custom tasks:
* Pipeline Resources
* [`retries`](#using-the-retries-field)
* [`timeout`](#configuring-the-failure-timeout)

### Known Custom Tasks

We try to list as many known Custom Tasks as possible here so that users can easily find what they want. Please feel free to share the Custom Task you implemented in this table.

#### v1beta1.CustomRun

| Custom Task | Description |
|:---|:--- |
| [Wait Task Beta][wait-task-beta] | Waits a given amount of time before succeeding, specified by an input parameter named duration. Support `timeout` and `retries`. |

#### v1alpha1.Run

| Custom Task | Description |
|:---|:--- |
| [Pipeline Loops][pipeline-loops]| Runs a `Pipeline` in a loop with varying `Parameter` values.
| [Common Expression Language][cel]| Provides Common Expression Language support in Tekton Pipelines.
| [Wait][wait]| Waits a given amount of time, specified by a `Parameter` named "duration", before succeeding.
| [Approvals][approvals]| Pauses the execution of `PipelineRuns` and waits for manual approvals.
| [Pipelines in Pipelines][pipelines-in-pipelines]| Defines and executes a `Pipeline` in a `Pipeline`.
| [Task Group][task-group]| Groups `Tasks` together as a `Task`.
| [Pipeline in a Pod][pipeline-in-pod]| Runs `Pipeline` in a `Pod`.

[pipeline-loops]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/pipeline-loops
[task-loops]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/task-loops
[cel]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/cel
[wait]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/wait-task
[approvals]: https://github.com/automatiko-io/automatiko-approval-task/tree/71da90361dff9444146d52d0a6e2b542d4bf4edc
[task-group]: https://github.com/openshift-pipelines/tekton-task-group/tree/39823f26be8f59504f242a45b9f2e791d4b36e1c
[pipelines-in-pipelines]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/pipelines-in-pipelines
[pipeline-in-pod]: https://github.com/tektoncd/experimental/tree/f60e1cd8ce22ed745e335f6f547bb9a44580dc7c/pipeline-in-pod
[wait-task-beta]: https://github.com/tektoncd/pipeline/tree/a127323da31bcb933a04a6a1b5dbb6e0411e3dc1/test/custom-task-ctrls/wait-task-beta

## Code examples

Expand Down
2 changes: 1 addition & 1 deletion docs/runs.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ weight: 800

# Overview

*We've released the Beta version of `Run`: `CustomRun`, check [this page](customruns.md) for details.*
*We are promoting Custom Task from `v1alpha1.Run` to [`v1beta1.CustomRun`](customruns.md), please refer to the [migration doc](migrating-v1alpha1.Run-to-v1beta1.CustomRun.md) for details.*

A `Run` allows you to instantiate and execute a [Custom
Task](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md),
Expand Down

0 comments on commit 963aa40

Please sign in to comment.