Skip to content

Commit

Permalink
Document PipelineResource's Alpha State
Browse files Browse the repository at this point in the history
During the release of Pipelines' Beta we communicated the
fact that PipelineResources are not being promoted along
with other types but we didn't include any extensive
commentary to support that decision.

This commit adds a description of the problems that remain
with PipelineResources, along with possible next steps
that the project might take with them.
  • Loading branch information
Scott authored and tekton-robot committed Apr 20, 2020
1 parent 9344e4f commit 1346656
Showing 1 changed file with 82 additions and 0 deletions.
82 changes: 82 additions & 0 deletions docs/resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,19 @@ For example:
deployed in a cluster.
- A `Task`'s output can be a jar file to be uploaded to a storage bucket.

> **Note**: PipelineResources have not been promoted to Beta in tandem with Pipeline's
> other CRDs. This means that the level of support for `PipelineResources`
> remains Alpha and there are effectively no guarantees about the type's future.
> There remain a lot of known issues with the type that have caused Tekton's
> developers to reassess it.
>
> For Beta-supported alternatives to PipelineResources see
> [the v1alpha1 to v1beta1 migration guide](./migrating-v1alpha1-to-v1beta1.md#pipelineresources-and-catalog-tasks)
> which lists each PipelineResource type and a suggested option for replacing it.
>
> For more information on why PipelineResources are remaining alpha [see the description
> of their problems, along with next steps, below](#why-arent-pipelineresources-in-beta).
--------------------------------------------------------------------------------

- [Syntax](#syntax)
Expand All @@ -37,6 +50,7 @@ For example:
- [GCS Storage Resource](#gcs-storage-resource)
- [BuildGCS Storage Resource](#buildgcs-storage-resource)
- [Cloud Event Resource](#cloud-event-resource)
- [Why Aren't PipelineResources in Beta?](#why-arent-pipelineresources-in-beta)

## Syntax

Expand Down Expand Up @@ -1048,6 +1062,74 @@ Data,
}
```

## Why Aren't PipelineResources in Beta?

The short answer is that they're not ready to be given a Beta level of support by Tekton's developers. The long answer is, well, longer:

- Their behaviour can be opaque.

They're implemented as a mixture of injected Task Steps, volume configuration and type-specific code in Tekton
Pipeline's controller. This means errors from `PipelineResources` can manifest in quite a few different ways
and it's not always obvious whether an error directly relates to `PipelineResource` behaviour. This problem
is compounded by the fact that, while our docs explain each Resource type's "happy path", there never seems to
be enough info available to explain error cases sufficiently.

- When they fail they're difficult to debug.

Several PipelineResources inject their own Steps before a `Task's` Steps. It's extremely difficult to manually
insert Steps before them to inspect the state of a container before they run.

- There aren't enough of them.

The six types of existing PipelineResources only cover a tiny subset of the possible systems and side-effects we
want to support with Tekton Pipelines.

- Adding extensibility to them makes them really similar to `Tasks`:
- User-definable `Steps`? This is what `Tasks` provide.
- User-definable params? Tasks already have these.
- User-definable "resource results"? `Tasks` have `Task` Results.
- Sharing data between Tasks using PVCs? `workspaces` provide this for `Tasks`.
- They make `Tasks` less reusable.
- A `Task` has to choose the `type` of `PipelineResource` it will accept.
- If a `Task` accepts a `git` `PipelineResource` then it's not able to accept a `gcs` `PipelineResource` from a
`TaskRun` or `PipelineRun` even though both the `git` and `gcs` `PipelineResources` fetch files. They should
technically be interchangeable: all they do is write files from somewhere remote onto disk. Yet with the existing
`PipelineResources` implementation they aren't interchangeable.

They also present challenges from a documentation perspective:

- Their purpose is ambiguous and it's difficult to articulate what the CRD is precisely for.
- Four of the types interact with external systems (git, pull-request, gcs, gcs-build).
- Five of them write files to a Task's disk (git, pull-request, gcs, gcs-build, cluster).
- One tells the Pipelines controller to emit CloudEvents to a specific endpoint (cloudEvent).
- One writes config to disk for a `Task` to use (cluster).
- One writes a digest in one `Task` and then reads it back in another `Task` (image).
- Perhaps the one thing you can say about the `PipelineResource` CRD is that it can create
side-effects for your `Tasks`.

### Next steps

So what are PipelineResources still good for? We think we've identified some of the most important things:

1. You can augment `Task`-only workflows with `PipelineResources` that, without them, can only be done with `Pipelines`.
- For example, let's say you want to checkout a git repo for your Task to test. You have two options. First, you could use a `git` PipelineResource and add it directly to your test `Task`. Second, you could write a `Pipeline` that has a `git-clone` `Task` which checks out the code onto a PersistentVolumeClaim `workspace` and then passes that PVC `workspace` to your test `Task`. For a lot of users the second workflow is totally acceptable but for others it isn't. Some of the most noteable reasons we've heard are:
- Some users simply cannot allocate storage on their platform, meaning `PersistentVolumeClaims` are out of the question.
- Expanding a single `Task` workflow into a `Pipeline` is labor-intensive and feels unnecessary.
2. Despite being difficult to explain the whole CRD clearly each individual `type` is relatively easy to explain.
- For example, users can build a pretty good "hunch" for what a `git` `PipelineResource` is without really reading any docs.
3. Configuring CloudEvents to be emitted by the Tekton Pipelines controller.
- Work is ongoing to get notifications support into the Pipelines controller which should hopefully be able to replace the `cloudEvents` `PipelineResource`.

For each of these there is some amount of ongoing work or discussion. It may be that
`PipelineResources` can be redesigned to fix all of their problems or it could be
that the best features of `PipelineResources` can be extracted for use everywhere in
Tekton Pipelines. So given this state of affairs `PipelineResources` are being kept
out of beta until those questions are resolved.

For Beta-supported alternatives to PipelineResources see
[the v1alpha1 to v1beta1 migration guide](./migrating-v1alpha1-to-v1beta1.md#pipelineresources-and-catalog-tasks)
which lists each PipelineResource type and a suggested option for replacing it.

Except as otherwise noted, the content of this page is licensed under the
[Creative Commons Attribution 4.0 License](https://creativecommons.org/licenses/by/4.0/),
and code samples are licensed under the
Expand Down

0 comments on commit 1346656

Please sign in to comment.