Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Declare resources in pipeline πŸ˜‡ #415

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ Pipelines are **Typed**:

## Want to start using Pipelines?

- Jump in with [the quickstart!](docs/tutorial.md)
- Jump in with [the tutorial!](docs/tutorial.md)
- [Learn about the Concepts](/docs/Concepts.md)
- [See how to use it](/docs/using.md)
- [Read about how to use it](/docs/using.md)
- Look at [some examples](/examples)

## Want to contribute?
Expand Down
146 changes: 15 additions & 131 deletions docs/Concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,10 @@ High level details of this design:
triggered by events or by manually creating [PipelineRuns](#pipelinerun)
- [Tasks](#task) can exist and be invoked completely independently of
[Pipelines](#pipeline); they are highly cohesive and loosely coupled
- Test results are a first class concept, being able to navigate test results
easily is powerful (e.g. see failures easily, dig into logs, e.g. like
[the Jenkins test analyzer plugin](https://wiki.jenkins.io/display/JENKINS/Test+Results+Analyzer+Plugin))
- [Tasks](#task) can depend on artifacts, output and parameters created by other
tasks.
- [Resources](#pipelineresources) are the artifacts used as inputs and outputs
of TaskRuns.
- [PipelineResources](#pipelineresources) are the artifacts used as inputs and outputs
of Tasks.

## Building Blocks of Pipeline CRDs

Expand All @@ -44,61 +41,30 @@ Below diagram lists the main custom resources created by Pipeline CRDs:

### Task

A `Task` is a collection of sequential steps you would want to run as part of
your continuous integration flow. A Task will run inside a container on your
cluster.
A `Task` is a collection of sequential steps you would want to run as part of your
continuous integration flow. A task will run inside a container on your cluster.

A Task declares:
A `Task` declares:

- [Inputs](#inputs)
- [Outputs](#outputs)
- [Steps](#steps)

#### Inputs

Inputs declare the inputs the Task needs. Every task input resource should
provide name and type (like git, image). It can also provide optionally
`targetPath` to initialize the resource in specific directory. If `targetPath`
is set then the resource will be initialized under `/workspace/targetPath`. If
`targetPath` is not specified then the resource will be initialized
under`/workspace`. The following example demonstrates how git input repository
could be initialized in `GOPATH` to run tests.

```yaml
apiVersion: pipeline.knative.dev/v1alpha1
kind: Task
metadata:
name: task-with-input
namespace: default
spec:
inputs:
resources:
- name: workspace
type: git
targetPath: go/src/github.com/knative/build-pipeline
steps:
- name: unit-tests
image: golang
command: ["go"]
args:
- "test"
- "./..."
workingDir: "/workspace/go/src/github.com/knative/build-pipeline"
env:
- name: GOPATH
value: /workspace/go
```
Declare the inputs the `Task` needs. Every `Task` input resource should provide name
and type (like git, image).

#### Outputs

Outputs declare the outputs task will produce.
Outputs declare the outputs `Task` will produce.

#### Steps

Steps is a sequence of steps to execute. Each step is
[a container image](./using.md#image-contract).

Here is an example simple Task definition which echoes "hello world". The
Here is an example simple `Task` definition which echoes "hello world". The
`hello-world` task does not define any inputs or outputs.

It only has one step named `echo`. The step uses the builder image `busybox`
Expand All @@ -122,21 +88,7 @@ spec:

Examples of `Task` definitions with inputs and outputs are [here](../examples)

##### Step Entrypoint

To get the logs out of a [`Task`](#task), Knative provides its own executable
that wraps the `command` and `args` values specified in the `steps`. This means
that every `Task` must use `command`, and cannot rely on the image's
`entrypoint`.

##### Configure Entrypoint image

To run a step needs to pull an `Entrypoint` image. Knative provides a way for
you to configure the `Entrypoint` image in case it is hard to pull in your
environment. To do that you can edit the `image`'s value in a configmap named
[`config-entrypoint`](./../config/config-entrypoint.yaml).

#### ClusterTask
#### Cluster Task

A `ClusterTask` is similar to `Task` but with a cluster-wide scope. Cluster
Tasks are available in all namespaces, typically used to conveniently provide
Expand Down Expand Up @@ -171,7 +123,7 @@ spec:
name: hello-world
```

Examples of pipelines with more complex DAGs are [here](../examples/)
Examples of more complex `Pipelines` are [in our examples dir](../examples/).

#### PipelineResources

Expand All @@ -185,10 +137,10 @@ For example:
deployed in a cluster.
- A Task's output can be a jar file to be uploaded to a storage bucket.

Read more on `PipelineResources` and their types [here](./using.md)
Read more on PipelineResources and their types [here](./using.md#creating-pipelineresources).

`PipelineResources` in a Pipeline are the set of objects that are going to be
used as inputs and outputs of a `TaskRun`.
used as inputs and outputs of a `Task`.

#### Runs

Expand All @@ -206,76 +158,8 @@ requirements of the `Task`.

`TaskRun` definition includes `inputs`, `outputs` for `Task` referred in spec.

Input resource includes name and reference to pipeline resource and optionally
`paths`. The `paths` are used by `TaskRun` as the resource's new source paths
i.e., copy the resource from specified list of paths. `TaskRun` expects the
folder and contents to be already present in specified paths. The `paths`
feature could be used to provide extra files or altered version of existing
resource before execution of steps.

Output resource includes name and reference to pipeline resource and optionally
`paths`. The `paths` are used by `TaskRun` as the resource's new destination
paths i.e., copy the resource entirely to specified paths. `TaskRun` will be
responsible for creating required directories and copying contents over. The
`paths` feature could be used to inspect the results of taskrun after execution
of steps.

The `paths` feature for input and output resource is heavily used to pass same
version of resources across tasks in context of pipelinerun.

In the following example, task and taskrun are defined with input resource,
output resource and step which builds war artifact. After execution of Taskrun
(`volume-taskrun`), `custom` volume has the entire resource `java-git-resource`
(including the war artifact) copied to the destination path
`/custom/workspace/`.

```yaml
apiVersion: pipeline.knative.dev/v1alpha1
kind: Task
metadata:
name: volume-task
namespace: default
spec:
generation: 1
inputs:
resources:
- name: workspace
type: git
steps:
- name: build-war
image: objectuser/run-java-jar #https://hub.docker.com/r/objectuser/run-java-jar/
command: jar
args: ["-cvf", "projectname.war", "*"]
volumeMounts:
- name: custom-volume
mountPath: /custom
```

```yaml
apiVersion: pipeline.knative.dev/v1alpha1
kind: TaskRun
metadata:
name: volume-taskrun
namespace: default
spec:
taskRef:
name: volume-task
inputs:
resources:
- name: workspace
resourceRef:
name: java-git-resource
outputs:
resources:
- name: workspace
paths:
- /custom/workspace/
resourceRef:
name: java-git-resource
volumes:
- name: custom-volume
emptyDir: {}
```
Input and output resources include the PipelineResource's name in the `Task`
spec and a reference to the actual `PipelineResource` that should be used.

`TaskRuns` can be created directly by a user or by a
[PipelineRun](#pipelinerun).
Expand Down
Loading