Adds release tasks and pipeline for triggers

The release tasks and pipeline are based on the pipeline
project's release pipeline. The generated `release.yaml` file
is published at `gcr.io/tekton-releases/triggers/latest.release.yaml`.

Signed-off-by: Dibyo Mukherjee <dibyo@google.com>

tekton/README.md

@@ -0,0 +1,118 @@
+# Tekton Repo CI/CD
+
+_Why does Tekton triggers have a folder called `tekton`? Cuz we think it would be cool
+if the `tekton` folder were the place to look for CI/CD logic in most repos!_
+
+We use Tekton Pipelines to build, test and release Tekton Triggers!
+
+This directory contains the
+[`Tasks`](https://github.com/tektoncd/pipeline/blob/master/docs/tasks.md) and
+[`Pipelines`](https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md)
+that we use.
+
+The Pipelines and Tasks in this folder are used for:
+
+1. [Manually creating official releases from the official cluster](#create-an-official-release)
+
+To start from scratch and use these Pipelines and Tasks:
+
+1. [Install Tekton v0.3.1](https://github.com/tektoncd/pipeline/blob/master/tekton/README.md#install-tekton)
+1. [Setup the Tasks and Pipelines](https://github.com/tektoncd/pipeline/blob/master/tekton/README.md#setup)
+1. [Create the required service account + secrets](https://github.com/tektoncd/pipeline/blob/master/tekton/README.md#service-account-and-secrets)
+
+## Create an official release
+
+Official releases are performed from [the `prow` cluster](https://github.com/tektoncd/plumbing#prow)
+[in the `tekton-releases` GCP project](https://github.com/tektoncd/plumbing/blob/master/gcp.md).
+This cluster
+[already has the correct version of Tekton installed](#install-tekton).
+
+To make a new release:
+
+1. (Optionally) [Apply the latest versions of the Tasks + Pipelines](#setup)
+2. (If you haven't already) [Install `tkn`](https://github.com/tektoncd/cli#installing-tkn)
+2. [Run the Pipeline](#run-the-pipeline)
+3. Create the new tag and release in GitHub
+   ([see one of way of doing that here](https://github.com/tektoncd/pipeline/issues/530#issuecomment-477409459)).
+   _TODO(tektoncd/pipeline#530): Automate as much of this as possible with Tekton._
+4. Add an entry to [the README](../README.md) at `HEAD` for docs and examples for the new release
+   ([README.md#read-the-docs](README.md#read-the-docs)).
+5. Update the new release in GitHub with the same links to the docs and examples, see
+   [v0.1.0](https://github.com/tektoncd/pipeline/releases/tag/v0.1.0) for example.
+
+### Run the Pipeline
+
+To use [`tkn`](https://github.com/tektoncd/cli) to run the `publish-tekton-pipelines` `Task` and create a release:
+
+1. Pick the revision you want to release and update the
+   [`resources.yaml`](./resources.yaml) file to add a
+   `PipelineResoruce` for it, e.g.:
+
+   ```yaml
+   apiVersion: tekton.dev/v1alpha1
+   kind: PipelineResource
+   metadata:
+     name: tekton-triggers-vX-Y-
+   spec:
+     type: git
+     params:
+     - name: url
+       value: https://github.com/tektoncd/triggers
+     - name: revision
+       value: vX.Y.Z-invalid-tags-boouuhhh # REPLACE with the commit you'd like to build from
+   ```
+  
+3. To run against your own infrastructure (if you are running
+   [in the production cluster](https://github.com/tektoncd/plumbing#prow) the default account should
+   already have these creds, this is just a bonus - plus `release-right-meow` might already exist in the
+   cluster!), also setup the required credentials for the `release-right-meow` service account, either:
+
+   - For
+     [the GCP service account `release-right-meow@tekton-releases.iam.gserviceaccount.com`](#production-service-account)
+     which has the proper authorization to release the images and yamls in
+     [our `tekton-releases` GCP project](https://github.com/tektoncd/plumbing#prow)
+   - For
+     [your own GCP service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts)
+     if running against your own infrastructure
+
+
+2. [Connect to the production cluster](https://github.com/tektoncd/plumbing#prow):
+
+  ```bash
+  gcloud container clusters get-credentials prow --zone us-central1-a --project tekton-releases
+  ```
+
+6. Run the `release-pipeline` (assuming you are using the production cluster and
+   [all the Tasks and Pipelines already exist](#setup)):
+	
+   ```shell
+   # Create the resoruces - i.e. set the revision that you wan to build from
+   kubectl apply -f tekton/resources.yaml
+
+   # Change thie environment variable to the verison you would like to use.
+   # Be careful: due to #983 it is possible to overwrite previous releases.
+   export VERSION_TAG=v0.X.Y
+
+   tkn pipeline start \
+		--param=versionTag=${VERSION_TAG} \
+		--serviceaccount=release-right-meow \
+		--resource=source-repo=tekton-triggers-git \
+		--resource=bucket=tekton-bucket \
+		--resource=builtEventListenerSinkImage=event-listener-sink-image \
+    --resource=builtControllerImage=triggers-controller-image \
+		--resource=builtWebhookImage=triggers-webhook-image \
+		triggers-release
+   ```
+
+_TODO(tektoncd/pipeline#569): Normally we'd use the image `PipelineResources` to control which
+image registry the images are pushed to. However since we have so many images,
+all going to the same registry, we are cheating and using a parameter for the
+image registry instead._
+
+## Supporting scripts and images
+
+Some supporting scripts have been written using Python 2.7:
+
+- [koparse](./koparse) - Contains logic for parsing `release.yaml` files created
+  by `ko`
+

tekton/account.yaml

@@ -0,0 +1,4 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: release-right-meow

tekton/koparse/koparse.py

@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+
+"""
+koparse.py parses release.yaml files from `ko`
+
+The `ko` tool (https://github.com/google/go-containerregistry/tree/master/cmd/ko)
+builds images and embeds the full names of the built images in the resulting
+yaml files.
+
+This script does two things:
+
+* Parses those image names out of the release.yaml, including their digests, and
+  outputs those to stdout
+* Verifies the list of built images against an expected list, to be sure that all
+  expected images were built (and no extra images were built)
+"""
+
+import argparse
+import re
+import sys
+from typing import List
+
+
+DIGEST_MARKER = "@sha256"
+
+
+class ImagesMismatchError(Exception):
+    def __init__(self, missing: List[str], extra: List[str]):
+        self.missing = missing
+        self.extra = extra
+
+    def __str__(self):
+        errs = []
+        if self.missing:
+            errs.append("Images %s were expected but missing." % self.missing)
+        if self.extra:
+            errs.append("Images %s were present but not expected." %
+                        self.extra)
+        return " ".join(errs)
+
+
+class BadActualImageFormatError(Exception):
+    def __init__(self, image: str):
+        self.image = image
+
+    def __str__(self):
+        return "Format of image %s was unexpected, did not contain %s" % (self.image, DIGEST_MARKER)
+
+
+def parse_release(base: str, path: str) -> List[str]:
+    """Extracts built images from the release.yaml at path
+
+    Args:
+        base: The built images will be expected to start with this string,
+            other images will be ignored
+        path: The path to the file (release.yaml) that will contain the built images
+    Returns:
+        list of the images parsed from the file
+    """
+    images = []
+    with open(path) as f:
+        for line in f:
+            match = re.search(base + ".*" + DIGEST_MARKER + ":[0-9a-f]*", line)
+            if match:
+                images.append(match.group(0))
+    return images
+
+
+def compare_expected_images(expected: List[str], actual: List[str]) -> None:
+    """Ensures that the list of actual images includes only the expected images
+
+    Args:
+        expected: A list of all of the names of images that are expected to have
+            been built, including the path to the image without the digest
+        actual: A list of the names of the built images, including the path to the
+            image and the digest
+    """
+    for image in actual:
+        if DIGEST_MARKER not in image:
+            raise BadActualImageFormatError(image)
+
+    actual_no_digest = [image.split(DIGEST_MARKER)[0] for image in actual]
+
+    missing = set(expected) - set(actual_no_digest)
+    extra = set(actual_no_digest) - set(expected)
+
+    if missing or extra:
+        raise ImagesMismatchError(list(missing), list(extra))
+
+
+if __name__ == "__main__":
+    arg_parser = argparse.ArgumentParser(
+        description="Parse expected built images from a release.yaml created by `ko`")
+    arg_parser.add_argument("--path", type=str, required=True,
+                            help="Path to the release.yaml")
+    arg_parser.add_argument("--base", type=str, required=True,
+                            help="String prefix which is used to find images within the release.yaml")
+    arg_parser.add_argument("--images", type=str, required=True, nargs="+",
+                            help="List of all images expected to be built, without digests")
+    args = arg_parser.parse_args()
+
+    try:
+        images = parse_release(args.base, args.path)
+        compare_expected_images(args.images, images)
+    except (IOError, BadActualImageFormatError) as e:
+        sys.stderr.write("Error determining built images: %s\n" % e)
+        sys.exit(1)
+    except (ImagesMismatchError) as e:
+        sys.stderr.write("Expected images did not match: %s\n" % e)
+        with open(args.path) as f:
+            sys.stderr.write(f.read())
+        sys.exit(1)
+
+    print("\n".join(images))

tekton/koparse/test_koparse.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3.6
+
+import os
+import unittest
+
+import koparse
+
+
+IMAGE_BASE = "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/"
+PATH_TO_TEST_RELEASE_YAML = os.path.join(os.path.dirname(
+    os.path.abspath(__file__)), "test_release.yaml")
+PATH_TO_WRONG_FILE = os.path.join(os.path.dirname(
+    os.path.abspath(__file__)), "koparse.py")
+BUILT_IMAGES = [
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/kubeconfigwriter@sha256:68453f5bb4b76c0eab98964754114d4f79d3a50413872520d8919a6786ea2b35",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/creds-init@sha256:67448da79e4731ab534b91df08da547bc434ab08e41d905858f2244e70290f48",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init@sha256:7d5520efa2d55e1346c424797988c541327ee52ef810a840b5c6f278a9de934a",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/nop@sha256:3784d6b8f73043a29d2c1d6196801bee46fe808fbb94ba4fd21ca52dce503183",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/bash@sha256:d55917ef5c92627027e3755bfc577fbfa2fb783cccfb13a98632cb6ba6088cd6",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/gsutil@sha256:421a261436e16af4057b4a069fdae8a5aca6e37269952209ad9932a774aa0003",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/controller@sha256:bdc6f22a44944c829983c30213091b60f490b41f89577e8492f6a2936be0df41",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/webhook@sha256:cca7069a11aaf0d9d214306d456bc40b2e33e5839429bf07c123ad964d495d8a",
+]
+EXPECTED_IMAGES = [
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/kubeconfigwriter",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/creds-init",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/nop",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/bash",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/gsutil",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/controller",
+    "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/webhook",
+]
+
+
+class TestKoparse(unittest.TestCase):
+
+    def test_parse_release(self):
+        images = koparse.parse_release(IMAGE_BASE, PATH_TO_TEST_RELEASE_YAML)
+        self.assertListEqual(images, BUILT_IMAGES)
+
+    def test_parse_release_no_file(self):
+        with self.assertRaises(IOError):
+            koparse.parse_release(IMAGE_BASE, "whoops")
+
+    def test_parse_release_wrong_contents(self):
+        images = koparse.parse_release(IMAGE_BASE, PATH_TO_WRONG_FILE)
+        self.assertEqual(images, [])
+
+    def test_compare_expected_images(self):
+        koparse.compare_expected_images(EXPECTED_IMAGES, BUILT_IMAGES)
+
+    def test_compare_expected_images_bad_format(self):
+        with self.assertRaises(koparse.BadActualImageFormatError):
+            koparse.compare_expected_images(EXPECTED_IMAGES, EXPECTED_IMAGES)
+
+    def test_compare_expected_images_missing(self):
+        extra_expected = (EXPECTED_IMAGES[:] +
+                          ["gcr.io/knative-releases/something-else"])
+        with self.assertRaises(koparse.ImagesMismatchError):
+            koparse.compare_expected_images(extra_expected, BUILT_IMAGES)
+
+    def test_compare_expected_images_too_many(self):
+        extra_actual = (BUILT_IMAGES[:] +
+                        ["gcr.io/knative-releases/something-else@sha256:somedigest"])
+        with self.assertRaises(koparse.ImagesMismatchError):
+            koparse.compare_expected_images(EXPECTED_IMAGES, extra_actual)
+
+
+if __name__ == "__main__":
+    unittest.main()