Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[POC] Provisioner for SBOM #13171

Merged
merged 8 commits into from
Jan 21, 2025
Merged

[POC] Provisioner for SBOM #13171

merged 8 commits into from
Jan 21, 2025

Conversation

devashish-patel
Copy link
Contributor

@devashish-patel devashish-patel commented Sep 25, 2024

Example templates:

JSON:

{
  "builders": [
    {
      "type": "docker",
      "image": "ubuntu:20.04",
      "commit": true
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "apt-get update -y",
        "apt-get install -y curl",
        "bash -c \"$(curl -sSL https://install.mondoo.com/sh)\""
      ]
    },
    {
      "type": "shell",
      "inline": [
        "cnquery sbom --output cyclonedx-json --output-target /tmp/sbom_cyclonedx.json"
      ]
    },
    {
      "type": "hcp-sbom",
      "source": "/tmp/sbom_cyclonedx.json"
    }
  ]
}

HCL:

packer {
  required_plugins {
    docker = {
      version = ">= 1.0.0"
      source  = "github.com/hashicorp/docker"
    }
  }
}

source "docker" "ubuntu" {
  image  = "ubuntu:20.04"
  commit = true
}

build {
  sources = ["source.docker.ubuntu"]

  provisioner "shell" {
    inline = [
      "apt-get update -y",
      "apt-get install -y curl",
      "bash -c \"$(curl -sSL https://install.mondoo.com/sh)\""
    ]
  }

  provisioner "shell" {
    inline = [
      //"cnquery sbom --output cyclonedx-json | tee /tmp/sbom_cyclonedx.json",
      "cnquery sbom --output cyclonedx-json --output-target /tmp/sbom_cyclonedx.json",
      "cnquery sbom --output spdx-json --output-target /tmp/sbom_spdx.json",
    ]
  }

  provisioner "hcp-sbom" {
      source      = "/tmp/sbom_cyclonedx.json"
      destination = "./sbom/sbom_cyclonedx.json"
  }

  provisioner "hcp-sbom" {
    source      = "/tmp/sbom_spdx.json"
    destination = "./sbom/sbom_spdx.json"
  }
}

Copy link
Contributor

@lbajolet-hashicorp lbajolet-hashicorp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a bunch of comments on the code, I think we can simplify the download to have it done once only, technically once we've copied the file locally for Packer, we can copy it to the user-specified destination (if specified). That or we can factorise the code for downloading since it's very similar.
I'll let you address those comments and do another pass of review after that.

packer/provisioner.go Outdated Show resolved Hide resolved
packer/provisioner.go Outdated Show resolved Hide resolved
packer/provisioner.go Outdated Show resolved Hide resolved
packer/provisioner.go Outdated Show resolved Hide resolved
packer/provisioner.go Outdated Show resolved Hide resolved
packer/build.go Outdated Show resolved Hide resolved
provisioner/hcp_sbom/provisioner.go Outdated Show resolved Hide resolved
provisioner/hcp_sbom/provisioner.go Outdated Show resolved Hide resolved
provisioner/hcp_sbom/provisioner.go Outdated Show resolved Hide resolved
provisioner/hcp_sbom/provisioner.go Outdated Show resolved Hide resolved
provisioner/hcp_sbom/validate.go Outdated Show resolved Hide resolved
provisioner/hcp_sbom/validate.go Outdated Show resolved Hide resolved
provisioner/hcp_sbom/provisioner.go Outdated Show resolved Hide resolved
packer/provisioner.go Outdated Show resolved Hide resolved
website/content/docs/provisioners/hcp-sbom.mdx Outdated Show resolved Hide resolved
website/content/docs/provisioners/hcp-sbom.mdx Outdated Show resolved Hide resolved
website/content/docs/provisioners/hcp-sbom.mdx Outdated Show resolved Hide resolved
website/content/docs/provisioners/hcp-sbom.mdx Outdated Show resolved Hide resolved
website/content/docs/provisioners/hcp-sbom.mdx Outdated Show resolved Hide resolved
`destination` option in the provisioner.

Currently, we support `CycloneDX` and `SPDX` SBOM formats in `JSON`.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBA: Add more details about max number of files allowed to download, and if we are going to add the file name field!

@@ -662,6 +661,23 @@ func (bucket *Bucket) completeBuild(
return packerSDKArtifacts, fmt.Errorf("build failed, not uploading artifacts")
}

artifacts, err := bucket.doCompleteBuild(ctx, buildName, packerSDKArtifacts, buildErr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Copy link
Contributor

@JenGoldstrich JenGoldstrich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can consider this verbally approved, will make it as approved once we have tested it using the internal SDK instead of the locally generated one. Good work on this Lucas and Dev!

Since this will fail locally currently without the generated SDK. I checked this branch out and ran make test and make ci-lint, both passed.

@lbajolet-hashicorp lbajolet-hashicorp force-pushed the poc/provisioner-hcp-sbom branch 2 times, most recently from 7b430bc to 3b1f4cc Compare January 21, 2025 14:42
@lbajolet-hashicorp lbajolet-hashicorp marked this pull request as ready for review January 21, 2025 16:31
@lbajolet-hashicorp lbajolet-hashicorp requested review from a team as code owners January 21, 2025 16:31
devashish-patel and others added 8 commits January 21, 2025 16:23
The hcp-sbom provisioner is a provisioner that acts essentially like a
download-only file provisioner, which also verifies the file downloaded
is a SPDX/CycloneDX JSON-encoded SBOM file, and sets up its upload to
HCP Packer later on.
Since packer now supports keeping track of SBOMs produced during a
build, we add the code to integrate those changes into the internal/hcp
package, so we do upload them on build completion.
When a build cannot be completed without errors, the build state was
left as running, unless the build explicitly failed, which meant that
HCP Packer would be responsible for changing the status after the
heartbeats for the build stopped being sent for two 5m periods.

This commit changes this behaviour, by explicitly marking the build as
failed if something did not work while trying to complete a build on HCP
Packer, even if the local Packer core build succeeded before that.
In the current state, a Packer build that succeeds but fails to push its
metadata to HCP for reasons other than a lack of artifact will always
succeed from the perspective of a user invoking `packer build`.

This can be a bit misleading, as users may expect their artifacts to
appear on HCP Packer if their build succeeded on Packer Core, so this
commit changes this behaviour, instead reporting HCP errors as a real
error if the build failed, so packer returns a non-zero error code if
this happens.
Since the protos for uploading an SBOM for a build have been changed to
use an enumeration instead of a plain string with the latest revisions
to the HCP Packer SBOM support feature, we update how we reference those
values for the SBOM format to use that enum instead.
@lbajolet-hashicorp lbajolet-hashicorp force-pushed the poc/provisioner-hcp-sbom branch 2 times, most recently from 2213e5e to e2543f9 Compare January 21, 2025 21:32
Copy link
Contributor

@JenGoldstrich JenGoldstrich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @lbajolet-hashicorp and @devashish-patel for your work here, I am glad we're getting this out for our users :), it's been a long one and I think we have work to be proud of here

@JenGoldstrich JenGoldstrich dismissed lbajolet-hashicorp’s stale review January 21, 2025 21:49

Lucas asked me to in Zoom

@lbajolet-hashicorp lbajolet-hashicorp merged commit 2e60923 into main Jan 21, 2025
63 checks passed
@lbajolet-hashicorp lbajolet-hashicorp deleted the poc/provisioner-hcp-sbom branch January 21, 2025 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants