Skip to content
This repository has been archived by the owner on Sep 30, 2024. It is now read-only.

Commit

Permalink
Document dynamic changesetTemplates and steps.outputs (#17286)
Browse files Browse the repository at this point in the history 
* Document dynamic changesetTemplates and outputs

* Fix broken links

* Rework templating docs to use tables and better examples

* Update doc/campaigns/references/campaign_spec_yaml_reference.md

Co-authored-by: Erik Seliger <erikseliger@me.com>

* Apply suggestions from code review

Co-authored-by: Adam Harvey <aharvey@sourcegraph.com>

Co-authored-by: Erik Seliger <erikseliger@me.com>
Co-authored-by: Adam Harvey <aharvey@sourcegraph.com>
  • Loading branch information
@mrnugget @eseliger @LawnGnome
3 people authored Jan 19, 2021
1 parent b32224c commit 749f6de
Show file tree
Hide file tree
Showing 5 changed files with 231 additions and 34 deletions.
12 changes: 12 additions & 0 deletions doc/_resources/assets/docsite.css
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ The document content can set class names and IDs on elements (for example, Markd
--warning-badge-color: #f59f00;
--critical-badge-color: #f03e3e;
--experimental-color: #b200f8;
--feature-color: #38757F;

--table-row-bg-1: var(--body-bg);
}
Expand Down Expand Up @@ -432,6 +433,16 @@ body > #page > main > #content {
background-color: var(--experimental-color);
}

.badge-feature {
color: var(--feature-color);
background-color: var(--note-color);
}

.badge-note {
color: #000000;
background-color: var(--note-color);
}

.badge-warning {
color: #ffffff;
background-color: var(--warning-badge-color);
Expand Down Expand Up @@ -564,6 +575,7 @@ body > #page > main > #content {
.markdown-body aside.experimental {
border-color: var(--experimental-color);
}

.markdown-body table {
display: block;
width: 100%;
Expand Down
2 changes: 1 addition & 1 deletion doc/campaigns/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,5 +99,5 @@ Create a campaign by specifying a search query to get a list of repositories and

- [Requirements](references/requirements.md)
- [Campaign spec YAML reference](references/campaign_spec_yaml_reference.md)
- <span class="badge badge-experimental">Experimental</span> [Campaign spec templating](references/campaign_spec_templating.md)
- [Campaign spec templating](references/campaign_spec_templating.md)
- [Troubleshooting](references/troubleshooting.md)
141 changes: 115 additions & 26 deletions doc/campaigns/references/campaign_spec_templating.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,82 +5,118 @@
.markdown-body pre.chroma { font-size: 0.75em; }
</style>

<aside class="experimental">
<span class="badge badge-experimental">Experimental</span> This feature is experimental and might change in the future. It's available in Sourcegraph 3.22 with <a href="https://github.com/sourcegraph/src-cli">Sourcegraph CLI</a> 3.21.5 and later.
</aside>

## Overview

[Certain fields](#fields-with-template-support) in a [campaign spec YAML](campaign_spec_yaml_reference.md) can include template variables to create even more powerful and performant campaigns.
[Certain fields](#fields-with-template-support) in a [campaign spec YAML](campaign_spec_yaml_reference.md) support templating to create even more powerful and performant campaigns.

Templating in a campaign spec uses the delimiters `${{` and `}}`. Inside the delimiters, [template variables](#template-variables) and [template helper functions](#template-helpers-functions) may be used to produce a text value.

Template variables in a campaign spec all have this form: `${{ <variable> }}`. They are evaluated before the execution of each entry in `steps` and allow accessing not only data from search results but also from previous steps.
### Example campaign spec

Here is an example excerpt of a campaign spec that uses the template variables:
Here is an excerpt of a campaign spec that uses templating:

```yaml
on:
- repositoriesMatchingQuery: lang:go fmt.Sprintf("%d", :[v]) patterntype:structural -file:vendor

steps:
- run: comby -in-place 'fmt.Sprintf("%d", :[v])' 'strconv.Itoa(:[v])' ${{ join repository.search_result_paths " " }}
# ^ templating starts here
container: comby/comby
- run: goimports -w ${{ join previous_step.modified_files " " }}
# ^ templating starts here
container: unibeautify/goimports
```
In this case, `${{ repository.search_result_paths }}` will be replaced with the relative-to-root-dir file paths of each search resulted yielded by `repositoriesMatchingQuery`. By using the [template helper function](#template-helper-functions) `join`, an argument list of whitespace-separated values is constructed. Before the step is executed the final `run` value would look close to this:
Before executing the first `run` command, `repository.search_result_paths` will be replaced with the relative-to-root-dir file paths of each search result yielded by `repositoriesMatchingQuery`. By using the [template helper function](#template-helper-functions) `join`, an argument list of whitespace-separated values is constructed.

The final `run` value, that will be executed, will look similar to this:

```yaml
run: comby -in-place 'fmt.Sprintf("%d", :[v])' 'strconv.Itoa(:[v])' cmd/src/main.go internal/fmt/fmt.go
```

The result is that `comby` only search and replaces in those files, instead of having to search through the complete repository.

The `${{ previous_step.modified_files }}` in the second step will be replaced by the list of files that the previous `comby` step modified. The final `run` value will look like this, if `comby` modified both of these files:
Before the second step is executed `previous_step.modified_files` will be replaced with the list of files that the previous `comby` step modified. It will look similar to this:

```yaml
run: goimports -w cmd/src/main.go internal/fmt/fmt.go
```

See "[Examples](#examples)" for more examples of how to use and leverage templating in campaign specs.

## Fields with template support

Template variables are supported in the following fields:
Templating is supported in the following fields:

- [`steps.run`](campaign_spec_yaml_reference.md#steps-run)
- [`steps.env`](campaign_spec_yaml_reference.md#steps-run) values
- [`steps.files`](campaign_spec_yaml_reference.md#steps-run) values
- [`steps.outputs.<name>.value`](campaign_spec_yaml_reference.md#steps-outputs)

Additionally, with Sourcegraph 3.24 and [Sourcegraph CLI](../../cli/index.md) 3.24 or later:

- [`changesetTemplate.title`](campaign_spec_yaml_reference.md#changesettemplate-title)
- [`changesetTemplate.body`](campaign_spec_yaml_reference.md#changesettemplate-body)
- [`changesetTemplate.branch`](campaign_spec_yaml_reference.md#changesettemplate-branch)
- [`changesetTemplate.commit.message`](campaign_spec_yaml_reference.md#changesettemplate-commit-message)
- [`changesetTemplate.commit.author.name`](campaign_spec_yaml_reference.md#changesettemplate-commit-author)
- [`changesetTemplate.commit.author.email`](campaign_spec_yaml_reference.md#changesettemplate-commit-author)

## Template variables

The following template variables are available:
Template variables are the names that are defined and accessible when using templating syntax in a given context.

- `${{ repository.search_result_paths }}`
Depending on the context in which templating is used, different variables are available.

Unique list of file paths relative to the repository root directory in which the search results of the `repositoriesMatchingQuery`s have been found.
- `${{ repository.name }}`
For example: in the context of `steps` the template variable `previous_step` is available, but not in the context of `changesetTemplate`.

Full name of the repository in which the step is being executed.
- `${{ previous_step.modified_files }}`
### `steps` context

List of files that have been modified by the previous step in `steps`. Empty if no files have been modified.
- `${{ previous_step.added_files }}`
The following template variables are available in the fields under `steps`.

List of files that have been added by the previous step in `steps`. Empty if no files have been added.
- `${{ previous_step.deleted_files }}`
They are evaluated before the execution of each entry in `steps`, except for the `step.*` variables, which only contain values _after_ the step has executed.

List of files that have been deleted by the previous step in `steps`. Empty if no files have been deleted.
- `${{ previous_step.stdout }}`
| Template variable | Description |
| --- | --- |
| `repository.search_result_paths` | Unique list of file paths relative to the repository root directory in which the search results of the `repositoriesMatchingQuery`s have been found. |
| `repository.name` | Full name of the repository in which the step is being executed. |
| `previous_step.modified_files` | List of files that have been modified by the previous step in `steps`. Empty list if no files have been modified. |
| `previous_step.added_files` | List of files that have been added by the previous step in `steps`. Empty list if no files have been added. |
| `previous_step.deleted_files` | List of files that have been deleted by the previous step in `steps`. Empty list if no files have been deleted. |
| `previous_step.stdout` | The complete output of the previous step on standard output. |
| `previous_step.stderr` | The complete output of the previous step on standard error. |
| `step.modified_files` | Only in `steps.outputs`: List of files that have been modified by the just-executed step. Empty list if no files have been modified. </br><i><small>Requires Sourcegraph 3.24 and [Sourcegraph CLI](../../cli/index.md) 3.24 or later</small></i>. |
| `step.added_files` | Only in `steps.outputs`: List of files that have been added by the just-executed step. Empty list if no files have been added. </br><i><small>Requires Sourcegraph 3.24 and [Sourcegraph CLI](../../cli/index.md) 3.24 or later</small></i>. |
| `step.deleted_files` | Only in `steps.outputs`: List of files that have been deleted by the just-executed step. Empty list if no files have been deleted. </br><i><small>Requires Sourcegraph 3.24 and [Sourcegraph CLI](../../cli/index.md) 3.24 or later</small></i>. |
| `step.stdout` | Only in `steps.outputs`: The complete output of the just-executed step on standard output.</br><i><small>Requires Sourcegraph 3.24 and [Sourcegraph CLI](../../cli/index.md) 3.24 or later</small></i>. |
| `step.stderr` | Only in `steps.outputs`: The complete output of the just-executed step on standard error. </br><i><small>Requires Sourcegraph 3.24 and [Sourcegraph CLI](../../cli/index.md) 3.24 or later</small></i>. |

The complete output of the previous step on standard output.
- `${{ previous_step.stderr }}`
### `changesetTemplate` context

The complete output of the previous step on standard error.
> NOTE: Templating in `changsetTemplate` is only supported in Sourcegraph 3.24 and [Sourcegraph CLI](../../cli/index.md) 3.24 or later.

The following template variables are available in the fields under `changesetTemplate`.

They are evaluated after the execution of all entries in `steps`.

| Template variable | Description |
| --- | --- |
| `repository.search_result_paths` | Unique list of file paths relative to the repository root directory in which the search results of the `repositoriesMatchingQuery`s have been found. |
| `repository.name` | Full name of the repository in which the step is being executed. |
| `steps.modified_files` | List of files that have been modified by the `steps`. Empty list if no files have been modified. |
| `steps.added_files` | List of files that have been added by the `steps`. Empty list if no files have been added. |
| `steps.deleted_files` | List of files that have been deleted by the `steps`. Empty list if no files have been deleted. |
| `outputs.<name>` | Value of an [`output`](campaign_spec_yaml_reference.md#steps-outputs) set by `steps`. If the [`outputs.<name>.format`](campaign_spec_yaml_reference.md#steps-outputs-format) is `yaml` or `json` and the `value` a data structure (i.e. array, object, ...), then subfields can be accessed too. See "[Examples](#examples)" below. |

## Template helper functions

- `${{ join repository.search_result_paths "\n" }}`
- `${{ split repository.name "/" }}`

The features of Go's [`text/template`](https://golang.org/pkg/text/template/) package are also available, including conditionals and loops, since it is the underlying templating engine.

## Examples

Pass the exact list of search result file paths to a command:
Expand Down Expand Up @@ -135,11 +171,64 @@ steps:

If you need to escape the `${{` and `}}` delimiters you can simply render them as string literals:


```yaml
steps:
- run: cp /tmp/escaped.txt .
container: alpine:3
files:
/tmp/escaped.txt: ${{ "${{" }} ${{ "}}" }}
```

Accessing the `outputs` set by `steps` in subsequent `steps` and the `changesetTemplate`:

```yaml
steps:
- run: echo "Hello there!"
container: alpine:3
outputs:
myFriendlyMessage:
value: "${{ step.stdout }}"
- run: echo "We have access to the output here: ${{ outputs.myFriendlyMessage }}"
container: alpine:3
outputs:
stepTwoOutput:
otherMessage: "here too: ${{ outputs.myFriendlyMessage }}"
changesetTemplate:
# [...]
body: |
The first step left us the following message: ${{ outputs.myFriendlyMessage }}
The second step left this one: ${{ outputs.otherMessage }}
```

Using the [`steps.outputs.<name>.format`](campaign_spec_yaml_reference.md#steps-outputs-name-format) field, it's possible to parse the value of an output as JSON or YAML and access it as a data structure instead of just text:

```yaml
steps:
- run: cat .goreleaser.yml
container: alpine:3
outputs:
goreleaserConfig:
value: "${{ step.stdout }}"
# The step's output is parsed as YAML, making it accessible as a YAML
# object in the other templating fields.
format: yaml
goreleaserConfigExists:
# We can use the power of Go's text/template engine to dynamically produce complex values
value: "exists: ${{ gt (len step.stderr) 0 }}"
format: yaml
changesetTemplate:
# [...]
# Since templating fields use Go's `text/template` and `goreleaserConfig` was
# parsed as YAML we can iterate over every field:
body: |
This repository has a `gorelaserConfig`: ${{ outputs.goreleaserConfigExists.exists }}.
The `goreleaser.yml` defines the following `before.hooks`:
${{ range $index, $hook := outputs.goreleaserConfig.before.hooks }}
- `${{ $hook }}`
${{ end }}
```
Loading

0 comments on commit 749f6de

Please sign in to comment.