API: References and Specifications

[jerop]

Tekton Pipelines has the following fields:

• TaskRef - references to Tasks and Custom Tasks
• TaskSpec - embedded specifications of Tasks and CustomTasks
• PipelineRef - references to Pipelines
• PipelineSpec - embedded specifications of Pipelines

We have discussed replacing them with ref and spec, especially when we discussed supporting Pipelines in Pipelines. We descoped this work from the TEPs where it came up, so opening this issue to track this discussion and gather more feedback.

Below is the most promising API for using ref and spec - thanks to @afrittoli!

This example below demonstrates the API change to Pipelines, but the same can be applied to other types.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: clone-scan-notify
spec:
  tasks:

    # task using reference
    - name: pt-0
      ref:
         apiVersion: tekton.dev/v1beta1
         kind: Task
         name: git-clone

    # task using specification
    - name: pt-1
      spec:
         apiVersion: tekton.dev/v1beta1
         kind: Task
         spec:
           params: (...)
           steps:
             - (...)

    # pipeline using reference
    - name: pt-2
      ref:
         apiVersion: tekton.dev/v1beta1
         kind: Pipeline
         name: golang-pipeline

    # pipeline using specification
    - name: pt-3
      spec:
         apiVersion: tekton.dev/v1beta1
         kind: Pipeline
         spec:
           params: (...)
           tasks:
             - (...)

    # custom task using reference
    - name: pt-4
      ref:
         apiVersion: example.dev/v1
         kind: Example
         name: example-pipeline

    # custom task using specification
    - name: pt-5
      spec:
         apiVersion: example.dev/v1
         kind: Example
         spec:
           field1: (...)
           field2: (...)
             
    # anything using remote resolution (kind is specified in resource field)
    - name: pt-6
      ref:
        resolver: git
        resource: (...)

Notes

• We could default apiVersion such that it matches the top level value - this way apiVersion is required for Custom Tasks only
• ChildReferences already takes this shape - where it lists the references with apiVersion and kind - so this change will align the specifications with the status
• We can support both the existing API and this API while we migrate to this new version, but we should go into V1 with this API if we decide this is the direction we want to go

/kind feature
/area api

[jerop]

@tektoncd/core-maintainers please take a look

[abayer]

My biggest concern with this is that tasks[0].spec will have to be a runtime.RawExtension, since it could be a spec for a Task, a custom task, or (eventually) a Pipeline. We're stuck with runtime.RawExtension for custom tasks, but I'm not a fan of having to do that same kind of conversion for every PipelineTask.

I definitely lean more towards having distinct .taskSpec, .customTaskSpec (instead of overloading .taskSpec.spec as we do now for custom tasks), and (eventually) .pipelineSpec fields on PipelineTask. That said, switching to .ref from .taskRef etc makes sense to me, with defaulting as mentioned for apiVersion, and I'd say defaulting kind to Task in PipelineTask.Ref as well.

[pritidesai]

My biggest concern with this is that tasks[0].spec will have to be a runtime.RawExtension, since it could be a spec for a Task, a custom task, or (eventually) a Pipeline. We're stuck with runtime.RawExtension for custom tasks, but I'm not a fan of having to do that same kind of conversion for every PipelineTask.

💯

It's an implementation detail but I prefer avoiding runtime.RawExtension for other types of resources if possible.

I definitely lean more towards having distinct .taskSpec, .customTaskSpec (instead of overloading .taskSpec.spec as we do now for custom tasks), and (eventually) .pipelineSpec fields on PipelineTask. That said, switching to .ref from .taskRef etc makes sense to me, with defaulting as mentioned for apiVersion, and I'd say defaulting kind to Task in PipelineTask.Ref as well.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: clone-scan-notify
spec:
  tasks:
    # task using reference
    - name: pt-0
      ref:
         apiVersion: tekton.dev/v1beta1
         kind: Task
         name: git-clone

I like using kind this way but can we avoid introducing apiVersion here for task and pipeline? The problem with apiVersion here is it could mean that pipeline supports mixed versions, pipeline could be v1 and task could be v1beta1. We have not supported this kind of combination AFAIR. If we avoid apiVersion, it leads us to just two fields:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: clone-scan-notify
spec:
  tasks:
    # task using reference
    - name: pt-0
      ref:
         kind: Task
         name: git-clone

isn't ref is too short here? 😞

And, for the spec, we are again going back to spec.tasks[0].spec.spec 🙃

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: clone-scan-notify
spec:
  tasks:

    # task using specification
    - name: pt-1
      spec:
         apiVersion: tekton.dev/v1beta1
         kind: Task
         spec:
           params: (...)
           steps:
             - (...)

instead, its cleaner to be more descriptive.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[jerop]

/lifecycle frozen

[Yongxuanzhang]

comment for tracking 👀