Speed up CI pipeline

[KellyAH]

A continuation of #1053

Problem:

The CI pipeline currently runs as 1 job that executes everything sequentially:

1. builds the app
2. runs all tests

Fix:

Break up the pipeline so after the app is built, unit, and feature tests run in parallel.

1. build app
2. --> run unit tests
   --> run browser tests

TODO:

• Cleanup yml
  • add names - add names to clarify github actions #1054
  • do we need brew? - maybe - to run bin/setup (Zee note: I don't think we need brew! We can take the bin/setup stuff out and replace them with github-actions specific prep!)
  • do we need inotify watchers?
• Break out separate sequential jobs - (passing run)
  • build - (can start rails app)
  • test-rspec - (enviro + run unit tests)
  • test-features - (browser enviro + run browser tests)
• Make test jobs run in parallel if build job is successful
• Fix Warning: The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
• DRY up the jobs
  • extract repeated "services" sections into shared YAML block (see https://joshdevlin.com/blog/yaml-repeating-sections/) not supported by GitHub workflows
  • maybe look into reusing the steps that are part of "setup" somehow (either via YAML shared block or as a reusable workflow, or some other way)
  • maybe look into removing the extra health check options for postgres -- are those still needed?
  • look into updating the postgres version in CI -- make it match production and dev docker-compose version

[KellyAH]

note: needs: makes test job only execute if build job is successful.

[KellyAH]

note: build job currently has the same steps that test job does. but this is a WIP and I'll see if I can make setup a reusable step to DRY it up. Since we aren't using docker-compose to build the full stack i dunno it it's possible to build the app once and then share it between the different jobs. each job runs on its own VM so I think it's like each job is running on a fresh machine.

[zspencer]

It looks like you're already setting up caching, which should make downstream containers "speedrun" the bundle/yarn/brew processes!

So the most likely vector of decomposition would be to gather the duplicative bits into a job or pulling out a workflow but I'm not really fluent enough in Github Action's domain specific language to know which pathway is most likely to be shortest-to-successful.

[KellyAH]

Currently, all tests are running via bin/test script.

In order to decompose the test job into separate jobs test-features and test-rspec that will run in parallel, the pipe is gonna need a different command to execute tests.

I think these are the appropriate commands:

• Run test-rspec - bundle exec rspec
• Run test-features - yarn run test (which invokes package.json script test)

[zspencer]

You could also split the bin/test script into bin/test-features and bin/test-rspec and then call those directly!

[KellyAH]

maybe. But we use bin/test script for local test runs to conveniently run all the tests. If I split bin/test out into 2 new scripts, and we want to preserve the ability to easily run all tests that I'd have to... 🤔 make bin/test invoke bin/test-features and bin/test-rspec. Could it be an abstraction overkill?

[zspencer]

I think it's one of those tradeoffs where neither are advantageous over the other. Using bin/rspec or bin/rake for ruby and yarn run test for cucumber are totally fine.

[zspencer]

This is looking really good! It looks like the biggest time-savings will be removing homebrew from the CI pipeline; as that is adding 1.5m just to get homebrew installed, and another 4~5m to install the dependencies.

[KellyAH]

This is looking really good! It looks like the biggest time-savings will be removing homebrew from the CI pipeline; as that is adding 1.5m just to get homebrew installed, and another 4~5m to install the dependencies.

I think we need brew for overmind and other dependencies.
Else overmind is not installed via bin/setup and rails doesn't start with bin/run. See failing run when brew was removed.

[zspencer]

What if we... didn't use overmind or other brew-based installation bits in CI?

[KellyAH]

I'm blocked on this. Waiting on a dev ops friend to advise me.

[zspencer]

RSpec in < 1m! This is the happiest I've ever been!

[anaulin]

Getting very close. Things seem to be working in parallel, much faster than before (no brew installations), but we still have a couple of smallish thing to cleanup (see checkboxes in PR description).

[zspencer]

4m!!! That's soooo fast!

[KellyAH]

Thanks @anaulin and @zspencer for helping this work get unstuck and closer to the finish line 🏁 🐎 . I'll see if I can tackle the DRY tasks this week.

[anaulin]

One thing I'm wondering about is that initial "install and cache dependencies" step: it does a bundle install and a yarn install, and that's it. The two test steps also have to then do bundle install and yarn install, and I don't know if the caches are shared between the different steps.

It might be better to just remove the separate "install" step, and just run the two different test sets in parallel.

Thoughts?

[KellyAH]

One thing I'm wondering about is that initial "install and cache dependencies" step: it does a bundle install and a yarn install, and that's it. The two test steps also have to then do bundle install and yarn install, and I don't know if the caches are shared between the different steps.

It might be better to just remove the separate "install" step, and just run the two different test sets in parallel.

Thoughts?

🤔 What happens if we leave the "bundle install and a yarn install" steps in the setup job and remove them from the 2 parallel test jobs?

🗒️ According to GHA cache doc caching seems to be shared between jobs.

Use caching when you want to reuse files that don't change often between jobs or workflow runs, such as build dependencies from a package management system.

[anaulin]

What happens if we leave the "bundle install and a yarn install" steps in the setup job and remove them from the 2 parallel test jobs?

I removed the yarn install steps from the test-_* jobs, and things still seem to work. I don't think I can remove the bundle install step without removing the "setup ruby" step, but I think that's ok, because it is takes about 5s (since things are cached):

[zspencer]

So, is this ready to merge then?! It looks amazing! I want it!!!

[anaulin]

GitHub doesn't support YAML block reuse via anchors just yet:

So no "services" block reuse for us 🤷🏼‍♀️

[anaulin]

@zspencer i think this is good enough to merge (if we want to do more tightening, we can always do that in follow-up PRs)

[zspencer]

YESSSSS

[KellyAH]

Thanks so much for taking this over the finish line @anaulin 🏁 🏎️ I ran outta steam at the end and this past week has been packed with family projects. Thanks again! 🙇‍♀️

[KellyAH]

wow the time-saving is amazing. before this change, pipeline took ~7m 47s, now it's 3m 37s. 🌟

[zspencer]

This is the happiest I have ever been!