Proposal for cryptographically signing integration packages

[akshay-saraswat]

There are lots of components that would need changes for package signing and validation in Kibana.

Acceptance Criteria:

• Drafting a document to share the proposed changes and areas.
• Creating github issues for those changes and linking them to this META issue.

[mtojek]

Let me sketch the MVP.

I based my research on the GPG signing workflow: https://www.gnupg.org/gph/en/manual/x135.html . The biggest inconvenience is the archiving logic which creates .zip archives on the fly. This will change if we decide to switch to store only .zip files in the Package Storage.

1. Only Elastic can sign packages.
2. Private keys are internal and can’t escape the CI.
3. While building the release we have to generate -.sig files in the CI environment.
4. Public key is stored in the Package Storage and distributed together with its Docker images.
5. Signature files (.sig) are stored in the signatures/ directory in Package Storage.
6. Package Registry exposes additional property “signature”.
7. After Kibana downloads the .zip package it verifies the package signature.

Comments:

1. Package .zip archives are created dynamically, which means that the file content may slightly differ if we change the Go version or adjust the archive serving logic. We can acknowledge this as it will be fixed when we switch to static .zip files instead of the dynamic option.
2. Another option would be adopting a different signature algorithm, similar to content addressing: https://docs.ipfs.io/concepts/content-addressing/
   signature = hash(signature(f1) + signature(f2) + … + signature(fn)), where fn are alphanum sorted files in the archive.
   2.1 Kibana must be aware of this signing algorithm
   2.2 It isn’t GPG anymore
   2.3 We can change implementation/compression rate of archiver

Let me know what you think.

cc @jsoriano @jlind23

[jlind23]

@akshay-saraswat signing is also a good point for the registry design. What should be signed? Who should have access to those keys?

[jsoriano]

Some thoughts.

This will change if we decide to switch to store only .zip files in the Package Storage.

Maybe we can wait to this. Signing whole archives would be definitely easier and less error-prone than something like content addressing. But one point in favour of content addressing is that it may allow to include the signature in the package itself (as I think is done in android packages for example).

1. Only Elastic can sign packages.

But only because only Elastic can publish packages so far :) Any package developer should be able to sign packages, with their own keys.

3. While building the release we have to generate -.sig files in the CI environment.

Ideally building packages should generate both the package files and the signature when a key is configured. Maybe there should be also a way to sign packages already built. These actions should be supported by elastic-package.
In our case this process should be probably automated in CI, yes.

4. Public key is stored in the Package Storage and distributed together with its Docker images.

Not sure about this. Keys distribution should be probably independent from the distribution of the images. e.g. Kibana could have Elastic keys pre-installed, so when installing a package it can check the validity of the signature. Other keys could be installed from trusted key servers.
If a docker image is compromised it could include both malicious packages and the public keys used to sign them. This is more difficult to exploit if Kibana installs the keys independently, it doesn't need to accept packages signed with keys distributed with the packages themselves.

signing is also a good point for the registry design.

I think the registry shouldn't have any responsibility in this process, apart of serving the signatures if they are separated files.

What should be signed? Who should have access to those keys?

I think developers of packages should have their private keys, and all the content in the packages should be signed (except the signature itself if we do something like content addressing and include it in the package file).

[mtojek]

Design overview

0 - means a prerequisite

Changes in Package Registry #760 (done)

0. Package Registry expects that we distribute packages as .zip artifacts together with .sig files.
1. File .sig contains a hash - signature.
2. Package Registry API shows a signature for a package (if .sig file exists).

Changes in Package Spec

If we don't bundle .sig files together with packages, no need for spec updates.

Changes in Elastic Package

0. Build packages as .zip files. - Build zipped package elastic-package#550
   1. elastic-package publish will sign built packages. PRs to Package Storage will contain .zip packages and .sig files.
1. elastic-package build will provide the --sign flag, which will force the builder to create .sig files.

Changes in Integrations

1. CI will use predefined signing key during the "publish" build --sign operation.

Changes in Package Storage

1. Signatures will be stored alongside .zip artifacts.

[jlind23]

We do have to package sig files with package for air gapped environnement right? Do package registry validate or invalidate the signature authenticity? What if as an open source open I push a package with my own sig file?

[mtojek]

We do have to package sig files with package for air gapped environnement right?

Yes, we'll include them in every Docker distribution.

Do package registry validate or invalidate the signature authenticity?

Based on @jsoriano's reply above, the Package Registry should be "stupid" as there is a risk of being compromised. In this case I would leave signature verification to Kibana.

What if as an open source open I push a package with my own sig file?

It's an interesting question, but I guess we aren't even close to that point. Let's first fully support community packages (third-party owners) and then we can talk about own signatures :)

[mtojek]

I'm working on the signing logic in elastic-package. I did some research and ProtonMail's library should do the trick: https://github.com/ProtonMail/gopenpgp#detached-signatures-for-plain-text-messages

[mtojek]

The implementation is done, so let me resolve in favor of elastic/elastic-package#583 .