Move Travis CI to GitHub Workflows

[DasSkelett]

Replacement of #2994, re-pushed to a branch directly in the KSP-CKAN/CKAN repo to be able to test the workflows with this PR.

Motivation

While Travis is a great tool, it has its difficult times. Currently almost every CI run fails in one instance because of some network problems.
We already discussed this in the Discord a bit, and I think no one was really bound to Travis and the move to GitHub Workflows was generally supported.

Changes

This PR removes the .travis.yml file from the repository, so Travis is no longer triggered on merges.

Instead, there are now three workflow files according to the documentation:

1. The standard build workflow.

   • It is triggered with pull requests, to be more precise, with the opened, synchronize and reopened events. This is adjustable.
   • It contains to so-called jobs: I called them build and build_NetCore.
   • The build job:

     • The build job contains a matrix: a list of Mono versions to use and a list of build configurations. It does one pass for each possible configuration + mono combination (just like Travis did).
       The mono versions in the list are 5.20, 6.4, 6.6 and 6.8. So I removed 5.16 and 6.0 and added 6.8 again compared to our current Travis config. I'm happy to change the versions, this was basically just what I filled in to have something for this workflow.

     • It first removes the preinstalled Mono version (currently 6.4 on Ubuntu 18.04 workers, see this page)
       Then it adds the Mono repository for the Mono version of this pass, and installs mono-complete. This step, as on Travis, takes the longest (~10 minutes). Unfortunately apt packages are not cacheable.

     • Then it installs libcurl4-openssl-dev and xvfb.

     • Afterwards it tries to restore two caches for NuGet packages I've set up. They are keyed with the hash of the packages.config and all .csproj files, so every change to those files (potentially changing our dependencies) invalidates them. In this case the packages are just downloaded as usual before build.

     • The it's build time! Same command as before, nothing's changed there.

     • The ./build test+only command has a xvfb-run prefixed, this is needed because I couldn't get xvfb to run as a service as it did in Travis. But I don't think that's a problem.

     • If anything failed before, we get a nice message in the Discord. Note: This runs for PRs! So yes, we will get notifications for CI fails in PRs. This is easily changeable if it's too spammy.

   • The build_NetCore job:
     • No Mono versions specified, because we are building with dotnet. But the configuration matrix is there and contains Debug_NetCore and Release_NetCore.
     • We use an action maintained by GitHub itself to setup the NetCore environment with the right version, currently 3.1.1 (runtime) / 3.1.101 (SDK).
     • Same es before: Restore caches, build, test, send failure to Discord.

2. The deploy workflow

   • It runs only on pushes to master.
   • It runs with only one Mono version, the one we use for releases too. This simplifies a lot, and those nasty ifs for the deployment like in Travis are not needed. Currently Mono 5.20, not changed due to the well-known issues...
   • Same as before: Caches, build, test.
   • Plus, build the NetKAN Docker container and push it to the Hub. Fortunately we decided to implement this entirely in Cake, so we only have to set the right secrets (see below).
   • To push ckan.exe and netkan.exe to S3 I needed to do a bit more. Travis has a built in S3 support. Fortunately there was a third-party GH Action that claims to do this. I'm not sure if we want to keep the --delete option:

   "--delete permanently deletes files in the S3 bucket that are not present in the latest version of your repository/build."

   • Finally the Discord notification again. We will get this one regardless of job status, but this is not a problem, because this one is only run for master-pushes for one single Mono version.
     I couldn't really test this workflow, because it needs secrets set for the Docker Hub, to redeploy our Infra services and for the S3.

3. The release workflow:

   • Triggered when new client releases are created.
   • Installs one single Mono version, this is used to compile the release binaries.
   • Builds the macOS, deb and rpm packages in release configuration.
   • Get some data from the GH API needed to upload the files: upload_url and tag_name (i.e. the CKAN version)
   • Now the next part is unfortunately pretty repetitive, because we need to call the upload-release-asset action for every single file we want to upload...
     It uploads the ckan.exe, AutoUpdater.exe, ckan_*.deb, CKAN.dmg and ckan-*.rpm.
   • When everything is done uploading, we get a notification 🎉

If you want to see the log output of a release run after creating a new CKAN release: https://github.com/DasSkelett/CKAN/runs/428138462?check_suite_focus=true

Secrets

We (as in someone with admin right in this repository) need to set some secrets for this to work:

• DISCORD_WEBHOOK (pinging @Olympic1)
• DOCKERHUB_PASSWORD (pinging @techman83)
• DOCKERHUB_USERNAME maybe? Was not a secret in Travis, but could make sense.
• AWS_SECRET_ACCESS_KEY(pinging @techman83)
• AWS_ACCESS_KEY_ID maybe? Was not a secret in Travis, but could make sense.

Future additions

We can extend this more in the future if we want.
It's really easy to upload artifacts to GitHub during workflows. This could replace the S3 (or exist alongside) and we would have the artifacts more accessible here in the repository. We could upload an artifact during the deploy workflow (the one that is run after merges/pushes to master).
They are also downloadable via the API: https://developer.github.com/v3/actions/artifacts/

[HebaruSan]

Well, I ❤️ this. Where are our build experts to approve it?

[techman83]

I think this is brilliant! I can sort out the dockerhub username/password. We might want to move the KEY_ID and Dockerhub Username into Secrets for convenience and ease of updating if required.

For S3, we'll need to regenerate the creds. So probably makes sense to add this into our stack with explicit permissions and then revoke the old ones. I'll be able to do this over the next day or 2, just need to collect up the paths they have access to.

[techman83]

I've configured all the secrets. So uploading to Dockerhub will work and the redeploy wil work. The real rub is that the S3 link isn't a part of KSP-CKANs AWS tenancy, it's @pjf 's private one from memory. We should consider what we want to do there.

http://ckan-travis.s3-website-us-east-1.amazonaws.com/

As far as the project is concerned, it is mainly used by the metadata CI and giving out test builds. We can give the builder creds access to a bucket in our tenancy and use that or convert to using GitHubs artifacts.

[techman83]

This all seems reasonable to me from my skim. I'm yet to use github actions personally, but we can always tweak things as needed.

[HebaruSan]

Oh shoot, that was the old branch, DasSkelett/gh-workflow instead of upstream/gh-workflow. I hope this can be fixed...