The goal of the Pipeline CRD is to provide k8s-style resources that allow the declaration of CI/CD-style pipelines.
Features the Pipeline CRD will support include:
- Conditional, parallel and distributed execution
- Interaction with CI/CD resources such as source code, artifacts, results, deployments and clusters
The goal of the Pipeline CRD is to fit into and cooperate with the knative ecosystem, specifically:
The CRDs involved are:
High level details of this design:
- Pipelines do not know what will trigger them, they can be triggered by events or by manually creating PipelineRuns
- Tasks can exist and be invoked completely independently of pipelines; they are highly cohesive and loosely coupled
- Test results are a first class concept, being able to navigate test results easily is powerful (e.g. see failures easily, dig into logs, like sponge)
- Tasks can depend on artifacts, output and parameters created by other tasks.
Creating a TaskRun will invoke a Task, running all of the steps until completion
or failure. Creating a TaskRun will require satisfying all of the input requirements of the
Task.
TaskRuns are basically knative Builds with inputs and
outputs.
TaskRuns can be created directly by a user or by a PipelineRun.
Creating a PipelineRun executes the pipeline, creating TaskRuns for each task
in the pipeline.
PipelineRuns tie together a Pipeline and a PipelineParam.
A PipelineRun could be created:
- By a user manually
- In response to an event (e.g. in response to a Github event, possibly processed via knative eventing)
Task is a CRD that knows how to instantiate a Knative Build,
either from a series of steps (i.e. Builders)
or from a BuildTemplate.
It takes Knative Build and adds inputs and outputs. Where these inputs and outputs are provided
from is not known to a task, so they can be provided by a Pipeline or by a user invoking a Task directly.
Tasks are basically knative BuildTemplates
with additional input types and clearly defined outputs.
Pipeline describes a graph of Tasks to execute. It defines the DAG
and expresses how all inputs (including PipelineParams and outputs
from previous Tasks) feed into each Task. It allows for fan in and fan out, and
ordering can be expressed explicitly using prev and next, or it can be inferred
from a Task’s inputs.
Dependencies between parameters or inputs/outputs are expressed as references to k8s objects.
PipelineParams contains parameters for a Pipeline. One Pipeline
can be invoked with many different instances of PipelineParams, which can allow
for scenarios such as running against PRs and against a user’s personal setup.
PipelineParams can control:
- What sources the
Pipelineruns against - Which serviceAccount to use (provided to all tasks)
- What artifact stores are used (e.g. Docker registries)
- Where results are stored (e.g. in GCS)