Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement implicit parameter resolution. #4127

Merged
merged 1 commit into from
Sep 15, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/200-role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ rules:
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get"]
resourceNames: ["config-logging", "config-observability", "config-leader-election"]
resourceNames: ["config-logging", "config-observability", "config-leader-election", "feature-flags"]
wlynch marked this conversation as resolved.
Show resolved Hide resolved
- apiGroups: [""]
resources: ["secrets"]
verbs: ["list", "watch"]
Expand Down
2 changes: 2 additions & 0 deletions config/webhook.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@ spec:
value: config-observability
- name: CONFIG_LEADERELECTION_NAME
value: config-leader-election
- name: CONFIG_FEATURE_FLAGS_NAME
value: feature-flags
- name: WEBHOOK_SERVICE_NAME
value: tekton-pipelines-webhook
- name: WEBHOOK_SECRET_NAME
Expand Down
69 changes: 69 additions & 0 deletions docs/pipelineruns.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,6 +246,75 @@ case is when your CI system autogenerates `PipelineRuns` and it has `Parameters`
provide to all `PipelineRuns`. Because you can pass in extra `Parameters`, you don't have to
go through the complexity of checking each `Pipeline` and providing only the required params.

#### Implicit Parameters

**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**

When using an inlined spec, parameters from the parent `PipelineRun` will be
available to any inlined specs without needing to be explicitly defined. This
allows authors to simplify specs by automatically propagating top-level
parameters down to other inlined resources.

```yaml
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: echo-
spec:
params:
- name: MESSAGE
value: "Good Morning!"
pipelineSpec:
tasks:
- name: echo-message
taskSpec:
steps:
- name: echo
image: ubuntu
script: |
#!/usr/bin/env bash
echo "$(params.MESSAGE)"
```
wlynch marked this conversation as resolved.
Show resolved Hide resolved

On creation, this will resolve to a fully-formed spec and will be returned back
to clients to avoid ambiguity:

```yaml
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: echo-
spec:
params:
- name: MESSAGE
value: Good Morning!
pipelineSpec:
params:
- name: MESSAGE
type: string
tasks:
- name: echo-message
params:
- name: MESSAGE
value: $(params.MESSAGE)
taskSpec:
params:
- name: MESSAGE
type: string
spec: null
steps:
- name: echo
image: ubuntu
script: |
#!/usr/bin/env bash
echo "$(params.MESSAGE)"
```

Note that all implicit Parameters will be passed through to inlined resources
(i.e. PipelineRun -> Pipeline -> Tasks) even if they are not used.
Extra parameters passed this way should generally be safe (since they aren't
actually used), but may result in more verbose specs being returned by the API.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks a bunch @wlynch for adding this note 🙏


### Specifying custom `ServiceAccount` credentials

You can execute the `Pipeline` in your `PipelineRun` with a specific set of credentials by
Expand Down
63 changes: 63 additions & 0 deletions docs/taskruns.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,69 @@ spec:

**Note:** If a parameter does not have an implicit default value, you must explicitly set its value.

#### Implicit Parameters

**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**

When using an inlined `taskSpec`, parameters from the parent `TaskRun` will be
available to the `Task` without needing to be explicitly defined.

```yaml
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
generateName: hello-
spec:
params:
- name: message
value: "hello world!"
taskSpec:
# There are no explicit params defined here.
# They are derived from the TaskRun params above.
steps:
- name: default
image: ubuntu
script: |
echo $(params.message)
```
wlynch marked this conversation as resolved.
Show resolved Hide resolved

On creation, this will resolve to a fully-formed spec and will be returned back
to clients to avoid ambiguity:

```yaml
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
generateName: hello-
spec:
params:
- name: message
value: "hello world!"
taskSpec:
params:
- name: message
type: string
steps:
- name: default
image: ubuntu
script: |
echo $(params.message)
```

Note that all implicit Parameters will be passed through to inlined resource,
even if they are not used. Extra parameters passed this way should generally
be safe (since they aren't actually used), but may result in more verbose specs
being returned by the API.

#### Extra Parameters

**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**

You can pass in extra `Parameters` if needed depending on your use cases. An example use
case is when your CI system autogenerates `TaskRuns` and it has `Parameters` it wants to
provide to all `TaskRuns`. Because you can pass in extra `Parameters`, you don't have to
go through the complexity of checking each `Task` and providing only the required params.

### Specifying `Resources`

If a `Task` requires [`Resources`](tasks.md#specifying-resources) (that is, `inputs` and `outputs`) you must
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: echo-
spec:
pipelineSpec:
tasks:
- name: echo-message
taskSpec:
steps:
- name: echo
image: ubuntu
script: |
#!/usr/bin/env bash
echo "$(params.MESSAGE)"
params:
- name: MESSAGE
value: "Good Morning!"
15 changes: 15 additions & 0 deletions examples/v1beta1/taskruns/alpha/implicit-params.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
generateName: hello-
spec:
params:
- name: message
value: "hello world!"
taskSpec:
# There are no explicit params defined here. They are derived from the TaskRun.
steps:
- name: default
image: ubuntu
script: |
echo $(params.message)
172 changes: 172 additions & 0 deletions pkg/apis/pipeline/v1beta1/param_context.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
// Copyright 2021 The Tekton Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package v1beta1
wlynch marked this conversation as resolved.
Show resolved Hide resolved

wlynch marked this conversation as resolved.
Show resolved Hide resolved
import (
"context"
"fmt"

"github.com/tektoncd/pipeline/pkg/apis/config"
)

// paramCtxKey is the unique identifier for referencing param information from
// a context.Context. See [context.Context.Value](https://pkg.go.dev/context#Context)
// for more details.
var paramCtxKey struct{}

// paramCtxVal is the data type stored in the param context.
// This maps param names -> ParamSpec.
type paramCtxVal map[string]ParamSpec

// AddContextParams adds the given Params to the param context. This only
// preserves the fields included in ParamSpec - Name and Type.
func AddContextParams(ctx context.Context, in []Param) context.Context {
if in == nil {
return ctx
}

if config.FromContextOrDefaults(ctx).FeatureFlags.EnableAPIFields != "alpha" {
return ctx
}

out := paramCtxVal{}
// Copy map to ensure that contexts are unique.
v := ctx.Value(paramCtxKey)
if v != nil {
for n, cps := range v.(paramCtxVal) {
out[n] = cps
}
}
for _, p := range in {
// The user may have omitted type data. Fill this in in to normalize data.
if v := p.Value; v.Type == "" {
if len(v.ArrayVal) > 0 {
p.Value.Type = ParamTypeArray
}
if v.StringVal != "" {
p.Value.Type = ParamTypeString
}
}
out[p.Name] = ParamSpec{
Name: p.Name,
Type: p.Value.Type,
}
}
return context.WithValue(ctx, paramCtxKey, out)
}

// AddContextParamSpec adds the given ParamSpecs to the param context.
func AddContextParamSpec(ctx context.Context, in []ParamSpec) context.Context {
if in == nil {
return ctx
}

if config.FromContextOrDefaults(ctx).FeatureFlags.EnableAPIFields != "alpha" {
return ctx
}

out := paramCtxVal{}
// Copy map to ensure that contexts are unique.
v := ctx.Value(paramCtxKey)
if v != nil {
for n, ps := range v.(paramCtxVal) {
out[n] = ps
}
}
for _, p := range in {
cps := ParamSpec{
Name: p.Name,
Type: p.Type,
Description: p.Description,
Default: p.Default,
}
out[p.Name] = cps
}
return context.WithValue(ctx, paramCtxKey, out)
}

// GetContextParams returns the current context parameters overlayed with a
// given set of params. Overrides should generally be the current layer you
// are trying to evaluate. Any context params not in the overrides will default
// to a generic pass-through param of the given type (i.e. $(params.name) or
// $(params.name[*])).
func GetContextParams(ctx context.Context, overlays ...Param) []Param {
pv := paramCtxVal{}
v := ctx.Value(paramCtxKey)
if v == nil && len(overlays) == 0 {
return nil
}
if v != nil {
pv = v.(paramCtxVal)
}
out := make([]Param, 0, len(pv))

// Overlays take precedence over any context params. Keep track of
// these and automatically add them to the output.
overrideSet := make(map[string]Param, len(overlays))
for _, p := range overlays {
overrideSet[p.Name] = p
out = append(out, p)
}

// Include the rest of the context params.
for _, ps := range pv {
// Don't do anything for any overlay params - these are already
// included.
if _, ok := overrideSet[ps.Name]; ok {
continue
}

// If there is no overlay, pass through the param to the next level.
// e.g. for strings $(params.name), for arrays $(params.name[*]).
p := Param{
Name: ps.Name,
}
if ps.Type == ParamTypeString {
p.Value = ArrayOrString{
Type: ParamTypeString,
StringVal: fmt.Sprintf("$(params.%s)", ps.Name),
}
} else {
p.Value = ArrayOrString{
Type: ParamTypeArray,
ArrayVal: []string{fmt.Sprintf("$(params.%s[*])", ps.Name)},
}
}
out = append(out, p)
}

return out
}

// GetContextParamSpecs returns the current context ParamSpecs.
func GetContextParamSpecs(ctx context.Context) []ParamSpec {
v := ctx.Value(paramCtxKey)
if v == nil {
return nil
}

pv := v.(paramCtxVal)
out := make([]ParamSpec, 0, len(pv))
for _, ps := range pv {
out = append(out, ParamSpec{
Name: ps.Name,
Type: ps.Type,
Description: ps.Description,
Default: ps.Default,
})
}
return out
}
Loading