ETCM-[165, 166]: Publish the RLP and Crypto libraries to Sonatype.

[aakoshh]

Description

Add steps to the CI pipeline to publish the mantis-rlp and mantis-crypto artifacts to Sonatype so they can be reused in other projects.

On the develop branch, the version is expected to be -SNAPSHOT, so it can overwrite previous versions.
On the master branch the version is set to release by removing -SNAPSHOT, so it can only be released once.

After that it's expected that develop should have a bump in version, but that's currently not part of the automation.

Proposed Solution

This is based of what we're currently doing in scalanet: instead of a version file I added a version.sbt file which could be used by sbt-release to bump and commit a new version, as described here, but currently it's just updated from the publish.sh script for the release, it's not committed and pushed back to Github. See the SBT guide for the Sonatype settings.

Ended up using this plugin: https://github.com/olafurpg/sbt-ci-release
What needs to be published is set int the publish.sh script using environment variables used by the plugin.

Its default assumption is that it will do a release if the build is tagged in git, and push a snapshot if it's just a normal commit. Tagging is faked in the publish script at the moment, but if the release strategy were to change, it would be easy to modify it that way.

To make it work we added the following environment variables to the build:

• GPG_KEY: GPG key used to sign the published artifacts, in Base64 format (see the sbt-ci-release plugin for the command to produce it)
• GPG_KEY_ID: ID of the GPG key, to tell the SBT plugin which key to use
• GPG_PASSPHRASE: Pass phrase to import the GPG key
• OSS_USERNAME: Sonatype username
• OSS_PASSWORD: Sonatype password

Important Changes Introduced

Added -SNAPSHOT to the version, which is stored in version.sbt.

Testing

I temporarily left the name of this branch as a conditional, which resulted in the SNAPSHOT versions being published.

See the logs on Buildkite.

[aakoshh]

@KonradStaniec can you add the required credentials to the build? I don't know the values.

[aakoshh]

It would be nice if the pipeline.nix could use wait steps so publishing only happens if everything worked fine. I should see if that works.

[aakoshh]

Apparently we should try to get a Mantis specific Sonatype account and PGP key.

[aakoshh]

@dzajkowski @KonradStaniec this should be ready for approval now.

[aakoshh]

@KonradStaniec kindly reminded me that we'd need this published for Scala 2.13 as well as 2.12, which I completely forgot.

[aakoshh]

I wonder how nix likes cross building. In any case I noticed that my PR has a ./nix/pkgs/mantis.nix which looks like it's currently missing on develop (there's one at ./nix/mantis.nix instead); I just did what the README told me to update the dependencies, not sure if it's still correct.

[aakoshh]

It turns out it's because without this the --pinentry-mode loopback doesn't work. When I removed that from the command the plugin printed, and tried it in the console, it worked with the --passphrase flag, so it's not an issue of surrounding with ' or having a bad password.

[dzajkowski]

oh this duplication with build.sbt will come back to bite us....

[jvdp]

It shouldn't be too hard to move more of this logic into sbt.

[aakoshh]

You'd think so, right! I spent hours trying to figure out with a code like this:

scapegoatVersion in ThisBuild := (scalaBinaryVersion.value match {
  case "2.12" => "1.4.5"
  case "2.13" => println("Still 2.13"); "1.4.7"
})

I start sbt, type ++ 2.12.10 and see "Still 2.13" printed every time, then I type scalaBinaryVersion and it says 2.12.10, really frustrating.

[aakoshh]

If you want to give it a go, you should be able to try wit sbt +compile for example (or at least sbt +publishLocal), the + should trigger cross building, and then it fails because it tries to grab the 1.4.7 version for 2.12.10, even when I moved the scapegoatVersion under node.

If I removed in ThisBuild then it wants to grab 1.0.0. No idea what I'm missing, this should work.

[dzajkowski]

Thanks for aligning with build changes!