Print step output into a single buffer

[mrnugget]

This is a side-quest to EM2 https://github.com/sourcegraph/sourcegraph/issues/24421 because we realised that we'll make things harder for ourselves if we separate stdout/stderr into separate buffers.

We'd have to zip them up together in the UI but that's hard without having additional timing information.

But timing information is also a bit overkill (at least for now) so we thought we'd use a single buffer.

That's what this PR here contains.

Stdout and stderr are logged into a single buffer, each line prefixed with stdout: and stderr: respectively.

That works because the process.PipeOutput function only writes lines (not chunks) to the passed in writers. So it works even if commands print half a line on stdout, then half a line on stderr, and only then the rest.

Example

Steps

steps:
  - run: |
      echo "this is step 1" >> README.txt
      echo "this is step 1 stdout"
      echo "this is step 1 stderr" 1>&2
      echo -n "this is step 1 stdout half a line..."
      echo -n "this is step 1 stderr half a line..." 1>&2
      echo -n "...this the other half of step 1 stdout line"
      echo -n "...this the other half of step 1 stderr line" 1>&2
    container: alpine:3
  - run: |
      echo "this is step 2" >> README.md
      echo "this is step 2 stdout"
      echo "this is step 2 stderr" 1>&2
    container: alpine:3
  - run: |
      echo "this is step 3" >> README.md
      echo "this is step 3 stdout"
      echo "this is step 3 stderr" 1>&2
    container: alpine:3
    outputs:
      myOutput:
        value: "my-output.txt"
  - run: |
      echo "this is step 4" >> README.md
      cat README.md
      echo "previous_step.modified_files=${{ previous_step.modified_files }}" >> README.md
      ls -lrt
    container: alpine:3
  - run: echo "this is step 5" >> ${{ outputs.myOutput }}
    container: alpine:3

The resulting TASK_STEP JSON lines that contain output:

{"operation":"TASK_STEP","timestamp":"2021-09-17T09:27:45.928Z","status":"PROGRESS","metadata":{"out":"stdout: this is step 1 stdout\nstdout: \nstderr: this is step 1 stderr\nstderr: \n","step":0,"taskID":"65qO4Zovx7V"}}
{"operation":"TASK_STEP","timestamp":"2021-09-17T09:27:45.947Z","status":"PROGRESS","metadata":{"out":"stderr: this is step 1 stderr half a line......this the other half of step 1 stdout line\nstderr: \nstdout: this is step 1 stdout half a line......this the other half of step 1 stdout line\nstdout: \n","step":0,"taskID":"65qO4Zovx7V"}}
{"operation":"TASK_STEP","timestamp":"2021-09-17T09:27:47.357Z","status":"PROGRESS","metadata":{"out":"stderr: this is step 2 stderr\nstderr: \nstdout: this is step 2 stdout\nstdout: \n","step":1,"taskID":"65qO4Zovx7V"}}
{"operation":"TASK_STEP","timestamp":"2021-09-17T09:27:48.835Z","status":"PROGRESS","metadata":{"out":"stderr: this is step 3 stderr\nstderr: \nstdout: this is step 3 stdout\nstdout: \n","step":2,"taskID":"65qO4Zovx7V"}}
{"operation":"TASK_STEP","timestamp":"2021-09-17T09:27:50.235Z","status":"PROGRESS","metadata":{"out":"stdout: # automation-testing\nstdout: \nstdout: This repository is used to test opening and closing pull request with Automation\nstdout: \nstdout: \nstdout: \nstdout: (c) Copyright Sourcegraph 2013-2020.\nstdout: \nstdout: (c) Copyright Sourcegraph 2013-2020.\nstdout: \nstdout: (c) Copyright Sourcegraph 2013-2020.this is step 2\nstdout: \nstdout: this is step 3\nstdout: \nstdout: this is step 4\nstdout: \nstdout: total 56\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000            17 Sep 17 09:27 test.md\nstdout: \nstdout: drwxrwxrwx    4 1000     1000          4096 Sep 17 09:27 project2\nstdout: \nstdout: drwxrwxrwx    2 1000     1000          4096 Sep 17 09:27 project1\nstdout: \nstdout: drwxrwxrwx    2 1000     1000          4096 Sep 17 09:27 gopkg\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000             0 Sep 17 09:27 foobar\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000            17 Sep 17 09:27 file_without_newline_at_eof.txt\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000           434 Sep 17 09:27 file_with_multiple_lines.txt\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000            23 Sep 17 09:27 file3.txt\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000            23 Sep 17 09:27 file2.txt\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000            23 Sep 17 09:27 file1.txt\nstdout: \nstdout: drwxrwxrwx    3 1000     1000          4096 Sep 17 09:27 examples\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000           706 Sep 17 09:27 circle-ci.yml\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000           153 Sep 17 09:27 Dockerfile\nstdout: \nstdout: -rw-r--r--    1 root     root            15 Sep 17 09:27 README.txt\nstdout: \nstdout: -rw-rw-rw-    1 1000     1000           299 Sep 17 09:27 README.md\nstdout: \n","step":3,"taskID":"65qO4Zovx7V"}}