[Umbrella] Bill of Materials

[puerco]

We intend to ensure the quality and integrity of the artifacts produced on each release cut by adding a Bill of Materials (BOM). The BOM will be published in SPDX and will include integrity and licensing information for the artifacts we produce. Work on this area will lead to close other outstanding issues (linked here).

Following our road-mapping session, this umbrella issue will track the development to create the BOM.

Make krel aware of binary artifacts expected from the release process:

Note: These items are postponed as we delayed the supported platforms effort to 1.23+

• Read the data from the proposed (Add machine readable description of platforms #1836) machine-readable platform map
• Bootstrap the Release Process state object with files expected as output, crossing the platform data and options specified in the run.

Verify/process binary artifacts as the release process advances from stage to stage

• Verify binaries to ensure the platform is correct (Add verification logic to ensure all Golang binaries in core images are of the same arch #1521 Release process artifact verification #2160 )
  This step involves:
  • Creating a new API in our tools to check the binaries (@puerco Binary package: Base module for binary analysis #1856)
  • Integrating it into the release process to check compiled artifacts (Release process artifact verification #2160)
• Verify tagging in binaries (issue Add a check to ensure release binaries have the correct tag versions before publishing #1898) (Release process artifact verification #2160)
  We need to ensure that binaries are correctly tagged with the corresponding semver tag and commit sha

Write SPDX manifest(s). Output should include data about:

• Naked binaries (Generate the first SBOM protoype from the Kubernetes release process #2095)
• Images (Provide links to k8s control plane images #1384) (Generate the first SBOM protoype from the Kubernetes release process #2095)
• Third-party licences (Improve license transparency for vendor/ in Kubernetes kubernetes#94976) .
  To accomplish this one we will need to:
  • Write or use a license parsing tool to obtain licenses of external modules. (Add k8s.io/release/pkg/license package #1874 @puerco)
  • Integrate the scanner into the release process scanning the vendor/ directory in k/k to get the licenses that will be added to the BOM Scan all dependencies licensing information and include them in the SBOM (Generate the first SBOM protoype from the Kubernetes release process #2095)

Publish the SPDX manifests with the other release artifacts:

• Generate the BOM in one or more formats supported by SPDX: tag, RDF, etc. (Generate the first SBOM protoype from the Kubernetes release process #2095)
• Write them to a subdirectory in the release bucket (Generate the first SBOM protoype from the Kubernetes release process #2095)
• Upload manifests as assets in the GitHub release page Note: In later discussions we chose to publish the documents only via https for now and not relay on the GH release page.
• Publish the SBOM documents via sbom.k8s.io (SBOM domain redirects: nginx + DNS config k8s.io#2447)

Make our tools available community-wide

• Build a general-purpose tool to generate SPDX compliant SBOMs (bom: A utility to generate SPDX compliant Bills of Materials #2066)
• Support defining complex SBOM compositions from a yaml file (Fixes to SBOM libraries #2096)
• Add bom utility documentation (Full set of bom READMEs and documentation #2109)
• Write an in-depth guide to generating SPDX bill of materials (Full set of bom READMEs and documentation #2109)

/cc @hasheddan @xmudrii @markyjackson-taulia

[nishakm]

cc @rnjudge and @kestewart for Tern

[tsteenbe]

@puerco If you need a tool to generate SPDX for Go and other languages, have a look at OSS Review Toolkit. I am both ORT and SPDX maintainer, happy to answer any questions you may have.

[puerco]

Thank you @tsteenbe I certainly will take a look at it :)

[lasomethingsomething]

/area release-eng

[justaugustus]

Connecting some threads based on an OSS supply chain convo I had earlier in the week with folks across VMware, Red Hat, and Google (@ncdc, @dlorenc, @wattsteve, @lukehinds, @kimsterv, @jonjohnsonjr to name a few):

• https://github.com/opencontainers/image-spec/blob/master/spec.md
• https://github.com/opencontainers/distribution-spec/blob/master/spec.md
• OCI artifact manifest, Phase 1-Reference Types opencontainers/artifacts#29
• [Presentation] Rekor tamper-resist ledgar cncf/tag-security#467 (comment)
• https://rekor.dev/
• https://rekor.dev/docs/plugable_types/

@saschagrunert @hasheddan @puerco -- Let's please leverage Rekor here as we build this out.

Of course, there's more to consider and supply chain concerns will be core to everything we do in RelEng, but the bill of materials is a great starting point.

cc: @kubernetes/sig-security-leads

Project Rekor xref: sigstore/rekor#156, sigstore/rekor#144

[lukehinds]

Rekor could in particular be beneficial for 'Publish the SPDX manifests with the other release artifacts' - this would provide two key benefits off the top of my head.

1. immutability, making the BOMs tamper resistant.
2. Public and assured transparency, e.g. more eyes to monitor the BOMs for anything untoward

I will read up on SPDX (tagging @bobcallaway for types expertise). Do you know if there will be some sort of signature at BOM generation time?

[puerco]

@lukehinds the first iteration of the SPDX BOM will not be signed as we have yet to set a key management mechanism before we start signing the manifests and other release artifacts. We hope to have it resolved soon(ish) as this is the main issue blocking us from generating and hosting debs and rpms in community infra.

[justaugustus]

Noting this issue in cncf/foundation about short-form copyright headers w/ SPDX identifiers: cncf/foundation#143

[saschagrunert]

@puerco is this one done?

[saschagrunert]

@puerco can we close this one as done?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[reylejano]

/remove-lifecycle stale

[puerco]

The SBOM project is done. We are now producing SBOMs with all k8s releases:

https://sbom.k8s.io/v1.23.0/release
https://sbom.k8s.io/v1.23.0/source

And we have spun off the took into its own repository: https://sigs.k8s.io/bom

Thanks for all your help and support, everybody! 🥳

/close

[k8s-ci-robot]

@puerco: Closing this issue.

In response to this:

The SBOM project is done. We are now producing SBOMs with all k8s releases:

https://sbom.k8s.io/v1.23.0/release
https://sbom.k8s.io/v1.23.0/source

And we have spun off the took into its own repository: https://sigs.k8s.io/bom

Thanks for all your help and support, everybody! 🥳

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.