Skip to content

Latest commit

 

History

History
232 lines (173 loc) · 11.5 KB

0096-remove-stacks-mixins.md

File metadata and controls

232 lines (173 loc) · 11.5 KB

Meta

Summary

NOTE: This proposal is part of a larger initiative to reduce complexity originally outlined in #167

This RFC proposes that we remove the stack ID concept from the project and replace it with existing constructs in the container image ecosystem such as operating system name, operating system version, and architecture.

This RFC also proposes that we remove the "mixin" concept from the project entirely, eliminating strict validation of all facets of base-image-to-buildpack compatibility in favor of a more optimistic approach.

This proposal prioritizes simplicity and flexibility over the avoidance of all failure scenarios by allowing more potentially successful builds to execute.

Motivation

The "stack" and "mixin" concepts add unnecessary complexity to the project and make it difficult for new users and contributors to understand how buildpacks work. Compatibility guarantees that are strongly enforced by the stack contract could be replaced with metadata validations and warnings.

What it is

Summary of changes:

  • Remove mixins
  • Replace stack metadata (including stack IDs) with canonical OS metadata.

How it Works

Base Image Metadata

Instead of a stack ID, runtime and build-time base images must contain the following canonicalized metadata:

  • OS (e.g., "linux", $GOOS), specified as os in the base image config
  • Architecture (e.g., "arm", $GOARCH), specified as architecture in the base image config
  • Architecture Variant (optional) (e.g., "v6", $GOARM), specified as variant in the base image config
  • Distribution (optional) (e.g., "ubuntu", $ID), specified as a label io.buildpacks.base.distro.name
  • Version (optional) (e.g., "18.04", $VERSION_ID), specified as a label io.buildpacks.base.distro.version

Additionally, the runtime base may contain the following metadata:

  • Target ID (optional) (e.g., "minimal"), specified as a label io.buildpacks.base.id

OS, Architecture, and Architecture Variant must be valid identifiers as defined in the OCI Image specification.

Target ID is an identifier specified on the runtime base image that must be provided to buildpacks as CNB_TARGET_ID during the build process. This allows buildpacks to change their behavior if a run image is selected (e.g., distroless) that has special properties outside of OS, architecture, etc.

For Linux-based images, each field should be canonicalized against values specified in /etc/os-release ($ID and $VERSION_ID). The os.version field in a base image config may contain combined distribution and version information, but it is not used by the lifecycle.

For Windows-based images, Distribution should be empty. Version should be the suggested value of os.version in the OCI spec (e.g., 10.0.14393.1066).

The stacks list in buildpack.toml is replaced by a targets list, where each entry corresponds to a different buildpack image that is exported into a manifest index. Each entry may contain multiple valid values for Distribution and/or Version, but only a single OS, Architecture, and Variant. If the targets list is empty and /bin/build is present, a target with os = "linux" and arch = "amd64" is assumed by tools reading buildpack.toml. If the targets list is empty and /bin/build.bat or /bin/build.exe is present, a target with os = "windows" and arch = "amd64" is assumed by tools reading buildpack.toml.

App image builds fail if the build image and selected run image have mismatched metadata. We may introduce flags or additional labels to skip this validation (e.g., for cross-compilation or minimal runtime base images). An image without a specified Distribution is compatible with images specifying any Distribution. An image specifying a Distribution without a Version is compatible with images specifying any Versions of that Distribution.

When an app image is rebased, rebaser must fail if the new run image and previous run image have mismatched metadata. This check can be skipped for Distribution and Version by passing a new --force flag to rebaser.

Example: buildpack.toml targets table

[[targets]]
os = "linux"
arch = "amd64"
[[targets.distributions]]
name = "ubuntu"
versions = ["18.04", "20.04"]

[[targets]]
os = "linux"
arch = "amd64"
[[targets.distros]]
name = "ubuntu"
version = "16.04"

[[targets]]
os = "linux"
arch = "arm"
variant = "v6"
[[targets.distros]]
name = "ubuntu"
version = "16.04"

Runtime Metadata

To allow different runtime base images to be used, and to support cross-compilation in the future, buildpacks may need access to the runtime base image's target metadata. The following environment variables will be available to buildpacks directly in the build-time environment (not via /platform/env):

  • CNB_TARGET_OS
  • CNB_TARGET_ARCH
  • CNB_TARGET_VARIANT
  • CNB_TARGET_DISTRO_NAME
  • CNB_TARGET_DISTRO_VERSION
  • CNB_TARGET_ID (optional ID, if present on runtime base image io.buildpacks.id label)

Mixins

Mixins are no longer used. If an SBoM is available, platforms may warn when, e.g., a rebase operation would change the available packages.

Example: CycloneDX SBoM

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.3",
  "version": 1,
  "components": [
    {
      "type": "library",
      "name": "curl",
      "version": "1.2.3",
      "purl": "pkg:deb/ubuntu/curl@1.2.3"
    },
    {
      "type": "library", 
      "name": "libcurl",
      "version": "4.5.6",
      "purl": "pkg:deb/ubuntu/libcurl@4.5.6"
    },
    ...
  ]
}

Validations

Buildpack base image metadata specified in buildpack.toml's targets list are validated against the runtime and build-time base images.

Runtime and build-time base image packages are no longer validated against each other.

Platforms may choose to add additional validation. For example, if an SBoM is available, pack rebase may fail if packages are removed from the new runtime base image. This check may be skipped by passing a new --force flag to pack rebase. However, this validation is not enforced by the specification.

Migration

All existing labels and environment variables for stacks and mixins may be preserved on a base image until all users have migrated to the new format. These labels will be deprecated (but allowed) for the forseeable future. If the newly-specified field values are missing, the lifecycle and pack may used existing, standardized stack IDs (i.e., io.buildpacks.stacks.*) to determine the values of the missing fields, as long as the lifecycle and pack provide a warning for the user. More concretely, when using a stack with io.buildpacks.stacks.bionic lifecycle will assume the following metadata is set if these fields are empty:

config.os = "linux"
config.architecture = "amd64"
io.buildpacks.base.distro.name = "ubuntu"
io.buildpacks.base.distro.version = "18.04"

Moving forward it's encouraged for buildpack authors to support both [[stacks]] and [[targets]] sections in buildpack.toml for maximum compatibility. In order to ease this process for those using the io.buildpacks.stacks.bionic, lifecycle will translate any section that sets this as one of the stacks:

[[stacks]]
id = "io.buildpacks.stacks.bionic

to

[[targets]]
os = "linux"
arch = "amd64"
[[targets.distros]]
name = "ubuntu"
versions = ["18.04"]

Drawbacks

  • Involves breaking changes when existing metadata is removed.
  • CNB_TARGET_* env vars assume a single target -- may require more breaking changes to support parallel, single-container cross-compilation in the future.

Alternatives

  • Keep stacks.
  • Make the above changes but keep some existing terminology: stacks, build-image, run-image.
  • Continue to allow buildpacks to specify package requirements (e.g., by PURL instead of mixins)

Unresolved Questions

  • How will migration work? Can we make new base images compatible with older buildpacks? Can we make newer buildpacks compatible with older stacks?
  • What should builder.toml (and similar stack-dependent config files) look like? What should assets look like? Note: these could be decided in subsequent subteam RFCs / spec PRs.
  • How will this affect the UX for pack buidpack package? For example package.toml contains a conflicting platform table. This should be resolved in a separate subteam RFC.
  • Are we planning to export an image for each target? Or does pack buildpack package require a target? Suggestion: All targets will live in an immutable OCI Manifest Index, so all targets should be exported at the same time.

Spec. Changes (OPTIONAL)

This RFC requires extensive changes to all specifications.

Amended

Summary

rename x86_64 -> amd64 in keeping with all other usages of arch. descriptors.

Motivation

This is how we do it everywhere else, this is the way.

Amended

Meta

  • Name: Rename Docker labels and buildpack.toml table names
  • Start Date: 2024-04-08
  • Author(s): @edmorley
  • Amendment Pull Request: rfcs#310

Summary

Changes were made to the Docker label and buildpack.toml table names between when this RFC was written and the changes were made to the spec in spec#365, which have now been backported to the RFC:

  • The io.buildpacks.distributions.* Docker labels were renamed to io.buildpacks.base.distro.*.
  • The io.buildpacks.id Docker label was renamed to io.buildpacks.base.id.
  • The buildpack.toml table [[targets.distributions]] was renamed to [[targets.distros]] and the versions field within it renamed to version (along with its type changing from an array to a string).

Motivation

To prevent use of the wrong Docker label or buildpack.toml table names, if users base their implementations on the RFC rather than reading the spec.