Skip to content

Commit

Permalink
[TEP-0044]: Add links + details on image builds
Browse files Browse the repository at this point in the history
This commit links to the ColocatedPipelineRun design doc and the experimental CustomTask
for discoverability. It also adds details on a design concern related to Pipelines that build
images and then use them in subsequent Tasks.
  • Loading branch information
lbernick authored and tekton-robot committed Jun 13, 2022
1 parent a371071 commit bb5cd25
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 2 deletions.
18 changes: 17 additions & 1 deletion teps/0044-data-locality-and-pod-overhead-in-pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
status: proposed
title: Data Locality and Pod Overhead in Pipelines
creation-date: '2021-01-22'
last-updated: '2022-03-09'
last-updated: '2022-05-26'
authors:
- '@bobcatfish'
- '@lbernick'
Expand Down Expand Up @@ -68,6 +68,10 @@ In addition, some systems support only the ReadWriteOnce [access mode](https://k
for PVCs, which allows the PVC to be mounted on a single node at a time. This means that Pipeline TaskRuns that share data and run in parallel
must run on the same node.

Lastly, Tekton's current implementation requires users to understand that TaskRuns are run as pods, and therefore, they need to provide
persistent storage because pods do not share a filesystem. Even if creating pods and PVCs had no overhead, users would still need to understand
how to provide data storage even for simple examples like a "clone, build, push" Pipeline.

The following issues describe some of these difficulties in more detail:
- [Issue: Difficult to use parallel Tasks that share files using workspace](https://github.com/tektoncd/pipeline/issues/2586):
This issue provides more detail on why it's difficult to share data between parallel tasks using PVCs.
Expand Down Expand Up @@ -284,6 +288,7 @@ We could work around this limitation via a few options:
3. Requiring that hermetic Tasks not execute in parallel with other Tasks run in the same pod.

### Controller role in scheduling TaskRuns

Some solutions to this problem involve allowing a user to configure which TaskRuns they would like to be executed on one pod,
and some solutions allow the controller to determine which TaskRuns should be executed on one pod.

Expand All @@ -296,12 +301,23 @@ which TaskRuns should be scheduled together. A first iteration of this proposal
TaskRuns to be combined together. After experimentation and user feedback, we can explore providing an option that would rely on the
controller to make this decision.

### Pipelines that build images

A frequent Tekton use case is a Pipeline that builds an image, and then runs that image in a subsequent Task.
Running such a Pipeline in a single pod would require Tekton to update the image used in the downstream Tasks
after a build Task has completed. While container images may be updated after pod creation, we must consider how Tekton
would know whether to replace an image, and how the new image could be re-wrapped with the Tekton entrypoint before
the Task begins to execute.

### Additional Design Considerations
- Executing an entire Pipeline in a pod, as compared to executing multiple Tasks in a pod, may pave the way for supporting
[local execution](https://github.com/tektoncd/pipeline/issues/235).

## Design proposal

- [Design doc](https://docs.google.com/document/d/18WTmtTImOo-4ZxIm8Z627WrXv1mkhGf6v1Ft8XxMvYQ/edit#heading=h.tnd2fu4u7702) (not yet accepted)
- [Experimental Custom Task](https://github.com/tektoncd/experimental/tree/main/pipeline-in-pod)

### Summary

The Tekton Pipelines controller will be updated to support running a Pipeline in a pod.
Expand Down
2 changes: 1 addition & 1 deletion teps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ This is the complete list of Tekton teps:
|[TEP-0040](0040-ignore-step-errors.md) | Ignore Step Errors | implemented | 2021-08-11 |
|[TEP-0041](0041-tekton-component-versioning.md) | Tekton Component Versioning | implementable | 2021-04-26 |
|[TEP-0042](0042-taskrun-breakpoint-on-failure.md) | taskrun-breakpoint-on-failure | implemented | 2021-12-10 |
|[TEP-0044](0044-data-locality-and-pod-overhead-in-pipelines.md) | Data Locality and Pod Overhead in Pipelines | proposed | 2022-03-09 |
|[TEP-0044](0044-data-locality-and-pod-overhead-in-pipelines.md) | Data Locality and Pod Overhead in Pipelines | proposed | 2022-05-26 |
|[TEP-0045](0045-whenexpressions-in-finally-tasks.md) | WhenExpressions in Finally Tasks | implemented | 2021-06-03 |
|[TEP-0046](0046-finallytask-execution-post-timeout.md) | Finally tasks execution post pipelinerun timeout | implemented | 2021-12-14 |
|[TEP-0047](0047-pipeline-task-display-name.md) | Pipeline Task Display Name | proposed | 2021-02-10 |
Expand Down

0 comments on commit bb5cd25

Please sign in to comment.