From 0af5a0f788a2e44c657963d374ce8bfe0b6d5b59 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Wed, 13 Jan 2021 17:45:54 -0600 Subject: [PATCH 01/12] Add RFC for prepare phase Signed-off-by: Joe Kutner --- text/0000-prepare-phase.md | 104 +++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 text/0000-prepare-phase.md diff --git a/text/0000-prepare-phase.md b/text/0000-prepare-phase.md new file mode 100644 index 000000000..bfefec311 --- /dev/null +++ b/text/0000-prepare-phase.md @@ -0,0 +1,104 @@ +# Meta +[meta]: #meta +- Name: Prepare Phase +- Start Date: 2021-01-13 +- Author(s): [@jkutner](github.com/jkutner/) +- RFC Pull Request: (leave blank) +- CNB Pull Request: [buildpacks/spec#176](https://github.com/buildpacks/spec/pull/176) +- CNB Issue: (leave blank) +- Supersedes: N/A + +# Summary +[summary]: #summary + +This is a proposal for a new Lifecycle phase, called "prepare", that would run before all other phases. + +# Definitions +[definitions]: #definitions + +* __stack descriptor__ - an enhanced version of the [`stack.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#stacktoml-toml) +* __project descriptor__ - the [`project.toml`](https://github.com/buildpacks/spec/blob/main/extensions/project-descriptor.md) extension specification + +# Motivation +[motivation]: #motivation + +The prepare phase would support the following features and capabilities: +* [Stack buildpacks](https://github.com/buildpacks/rfcs/pull/111), which require a phase to read run-image mixins validation prior to detection +* [Inline buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0048-inline-buildpack.md), which require parsing of the `project.toml` in the lifecycle +* [Lifecycle configuration](https://github.com/buildpacks/rfcs/pull/128) + +# What it is +[what-it-is]: #what-it-is + +The prepare phase will run before all other phase, and prepare the execution environment for a buildpack build. This phase will have access to secrets and credentials used to access registries and other services. + +## Responsibilities + +* Stack validation, to ensure that a new run-image is campatible with the previous app image +* Retrive run-image mixins, which will be use dby subsequent phases +* Validation of registry credentials, to avoid a long build that fails during export phase +* Parsing the project descriptor and performance various operations based on its contents, include: + - downloading buildpacks + - creating ephemeral buildpacks + - applying include and exclude rules + - modifying environment variables + +## Inputs + +* Log level +* Run-image +* Stack ID +* Project descriptor (optional) +* App source code + +## Output + +* Exit status +* Info-level logs to `stdout` +* Error-level logs to `stderr` +* Stack descriptor +* Analysis metadata +* Buildpacks (derived from inline buildpacks in project descriptor, or buildpacks in project descriptor that are not present in the builder) +* Order definition (derived from buildpacks configuration in project descriptor) +* Lifecycle configuration (derived from configuration in project descriptor) +* Mutated app source code (applying include and exclude rules in project descriptor) + +# How it Works +[how-it-works]: #how-it-works + +The prepare phase would be implemented as the `/cnb/lifecycle/preparer` binary. A platform MUST execute this phases either by invoking the `/cnb/lifecycle/preparer` binary or by executing `/cnb/lifecycle/creator`. + +The `preparer` binary will have access to the [`Keychain`](https://github.com/buildpacks/lifecycle/blob/main/auth/env_keychain.go), and MUST NOT execute arbitrary code provided by either the buildpack user or buildpack author. + +The [logic in the `analyzer` phase that reads image metadata and outputs an `analyzed.toml`](https://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L34-L40) would be moved into the `preparer`. + +The [logic in `pack` that parses a `project.toml`](https://github.com/buildpacks/pack/blob/main/project/project.go) would be copied or moved into the `preparer`. + +The app source code (which may be provided to the prepare either as a directory, volume, or tarball) would be mutated (either by copying it to a new location, or making changes directory against it). The `preparer` may delete files to apply the include and exclude rules from `project.toml`. + +# Drawbacks +[drawbacks]: #drawbacks + +* Yet another lifecycle phase + +# Alternatives +[alternatives]: #alternatives + +- [Reverse the order of analyze and detect phases](https://github.com/buildpacks/spec/pull/172) + +# Prior Art +[prior-art]: #prior-art + +- [Tekton prepare step](https://github.com/tektoncd/catalog/blob/11a17cfe87779099b0b61be3f1e496dfa79646b3/task/buildpacks-phases/0.1/buildpacks-phases.yaml#L61-L78) + +# Unresolved Questions +[unresolved-questions]: #unresolved-questions + +- Do we still need `analyzer`, or can the remaining parts of analyze phase be rolled into restore phase? +- Does `pack` still need to parse `project.toml`, or is there value in reading it early on (before lifecycle runs)? +- Should we create a shared library for `project.toml` parsing? + +# Spec. Changes +[spec-changes]: #spec-changes + +See [buildpacks/spec PR #176](https://github.com/buildpacks/spec/pull/176) From 288a096d81e7581261952743ea07e59014c2c11b Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Wed, 13 Jan 2021 17:58:38 -0600 Subject: [PATCH 02/12] Update text/0000-prepare-phase.md Signed-off-by: Joe Kutner Co-authored-by: Terence Lee --- text/0000-prepare-phase.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-prepare-phase.md b/text/0000-prepare-phase.md index bfefec311..e180d5a7f 100644 --- a/text/0000-prepare-phase.md +++ b/text/0000-prepare-phase.md @@ -30,7 +30,7 @@ The prepare phase would support the following features and capabilities: # What it is [what-it-is]: #what-it-is -The prepare phase will run before all other phase, and prepare the execution environment for a buildpack build. This phase will have access to secrets and credentials used to access registries and other services. +The prepare phase will run before all other phases, and prepare the execution environment for a buildpack build. This phase will have access to secrets and credentials used to access registries and other services. ## Responsibilities From 4547fe1ce602877db24f09e5b08bc9713c979be0 Mon Sep 17 00:00:00 2001 From: Joe Kutner Date: Thu, 14 Jan 2021 09:22:38 -0600 Subject: [PATCH 03/12] Added an alternative to prepare phase Signed-off-by: Joe Kutner --- text/0000-prepare-phase.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-prepare-phase.md b/text/0000-prepare-phase.md index e180d5a7f..1b1b4bfd9 100644 --- a/text/0000-prepare-phase.md +++ b/text/0000-prepare-phase.md @@ -85,6 +85,7 @@ The app source code (which may be provided to the prepare either as a directory, [alternatives]: #alternatives - [Reverse the order of analyze and detect phases](https://github.com/buildpacks/spec/pull/172) +- Split analyze into two parts (one to write `analyzed.toml` and one to analyze layers). The first (`analyzed.toml`) would be moved before the detect phase, and the second part (analyzing layers) would be merged into the restore phase. # Prior Art [prior-art]: #prior-art From 8c0541f588f245570e6ebddf2c815a89f817f7e9 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 14 Jan 2021 13:56:41 -0600 Subject: [PATCH 04/12] Updating based on feedback. Move analyze phase instead of new phase. Signed-off-by: Jesse Brown --- ...re-phase.md => 0000-move-analyze-phase.md} | 38 +++++++++---------- 1 file changed, 19 insertions(+), 19 deletions(-) rename text/{0000-prepare-phase.md => 0000-move-analyze-phase.md} (57%) diff --git a/text/0000-prepare-phase.md b/text/0000-move-analyze-phase.md similarity index 57% rename from text/0000-prepare-phase.md rename to text/0000-move-analyze-phase.md index 1b1b4bfd9..0bb683a24 100644 --- a/text/0000-prepare-phase.md +++ b/text/0000-move-analyze-phase.md @@ -1,17 +1,17 @@ # Meta [meta]: #meta -- Name: Prepare Phase +- Name: Move analyze phase - Start Date: 2021-01-13 -- Author(s): [@jkutner](github.com/jkutner/) +- Author(s): [@jkutner](github.com/jkutner/) [@jabrown85](github.com/jabrown85) - RFC Pull Request: (leave blank) -- CNB Pull Request: [buildpacks/spec#176](https://github.com/buildpacks/spec/pull/176) +- CNB Pull Request: [buildpacks/spec#172](https://github.com/buildpacks/spec/pull/172) - CNB Issue: (leave blank) - Supersedes: N/A # Summary [summary]: #summary -This is a proposal for a new Lifecycle phase, called "prepare", that would run before all other phases. +This is a proposal to re-order and adjust Lifecycle phases. Specifically, moving "analyze" before "detect". # Definitions [definitions]: #definitions @@ -22,7 +22,7 @@ This is a proposal for a new Lifecycle phase, called "prepare", that would run b # Motivation [motivation]: #motivation -The prepare phase would support the following features and capabilities: +Doing this would support the following features and capabilities: * [Stack buildpacks](https://github.com/buildpacks/rfcs/pull/111), which require a phase to read run-image mixins validation prior to detection * [Inline buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0048-inline-buildpack.md), which require parsing of the `project.toml` in the lifecycle * [Lifecycle configuration](https://github.com/buildpacks/rfcs/pull/128) @@ -30,18 +30,19 @@ The prepare phase would support the following features and capabilities: # What it is [what-it-is]: #what-it-is -The prepare phase will run before all other phases, and prepare the execution environment for a buildpack build. This phase will have access to secrets and credentials used to access registries and other services. +The analyze phase will now run before the detect phase. The analyze phase will have access to secrets and credentials used to access registries and other services, as it does today. Analyze will no longer require a [`group.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#grouptoml-toml). It will do this by splitting off some of the responsibilites into the restore phase. ## Responsibilities * Stack validation, to ensure that a new run-image is campatible with the previous app image -* Retrive run-image mixins, which will be use dby subsequent phases +* Retrive run-image mixins, which will be used by subsequent phases * Validation of registry credentials, to avoid a long build that fails during export phase * Parsing the project descriptor and performance various operations based on its contents, include: - downloading buildpacks - creating ephemeral buildpacks - applying include and exclude rules - modifying environment variables + - producing an [`order.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml) to be consumed by later phases ## Inputs @@ -57,35 +58,35 @@ The prepare phase will run before all other phases, and prepare the execution en * Info-level logs to `stdout` * Error-level logs to `stderr` * Stack descriptor -* Analysis metadata +* Analysis metadata [`analyzed.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#analyzedtoml-toml) * Buildpacks (derived from inline buildpacks in project descriptor, or buildpacks in project descriptor that are not present in the builder) -* Order definition (derived from buildpacks configuration in project descriptor) +* Buildpacks order [`order.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml) * Lifecycle configuration (derived from configuration in project descriptor) * Mutated app source code (applying include and exclude rules in project descriptor) # How it Works [how-it-works]: #how-it-works -The prepare phase would be implemented as the `/cnb/lifecycle/preparer` binary. A platform MUST execute this phases either by invoking the `/cnb/lifecycle/preparer` binary or by executing `/cnb/lifecycle/creator`. +A platform MUST execute the analyze phase either by invoking the `/cnb/lifecycle/detector` binary or by executing `/cnb/lifecycle/creator`. -The `preparer` binary will have access to the [`Keychain`](https://github.com/buildpacks/lifecycle/blob/main/auth/env_keychain.go), and MUST NOT execute arbitrary code provided by either the buildpack user or buildpack author. +The `analyzer` binary will have access to the [`Keychain`](https://github.com/buildpacks/lifecycle/blob/main/auth/env_keychain.go), and MUST NOT execute arbitrary code provided by either the buildpack user or buildpack author. -The [logic in the `analyzer` phase that reads image metadata and outputs an `analyzed.toml`](https://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L34-L40) would be moved into the `preparer`. +The [logic in the `analyzer` phase that reads image metadata and outputs an `analyzed.toml`](https://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L34-L40) would be remain. -The [logic in `pack` that parses a `project.toml`](https://github.com/buildpacks/pack/blob/main/project/project.go) would be copied or moved into the `preparer`. +The [logic in `pack` that parses a `project.toml`](https://github.com/buildpacks/pack/blob/main/project/project.go) would be copied or moved into the `analyzer`. -The app source code (which may be provided to the prepare either as a directory, volume, or tarball) would be mutated (either by copying it to a new location, or making changes directory against it). The `preparer` may delete files to apply the include and exclude rules from `project.toml`. +The [logic in the `analyzer` phase that analyzes layers](hhttps://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L54-L116) would be moved to the `restorer`. `restorer` already takes in `group.toml` as a flag. # Drawbacks [drawbacks]: #drawbacks -* Yet another lifecycle phase +* Platform maintainers will need to update the order of their container execution and also update flags for `analyzer`, `detector`, and `restorer`. +* Lifecycle will now take on the responsibility of processing `project.toml` # Alternatives [alternatives]: #alternatives -- [Reverse the order of analyze and detect phases](https://github.com/buildpacks/spec/pull/172) -- Split analyze into two parts (one to write `analyzed.toml` and one to analyze layers). The first (`analyzed.toml`) would be moved before the detect phase, and the second part (analyzing layers) would be merged into the restore phase. +- [Introduce Prepare Phase](https://github.com/buildpacks/rfcs/blob/4547fe1ce602877db24f09e5b08bc9713c979be0/text/0000-prepare-phase.md) (this same rfc, previous version) # Prior Art [prior-art]: #prior-art @@ -95,11 +96,10 @@ The app source code (which may be provided to the prepare either as a directory, # Unresolved Questions [unresolved-questions]: #unresolved-questions -- Do we still need `analyzer`, or can the remaining parts of analyze phase be rolled into restore phase? - Does `pack` still need to parse `project.toml`, or is there value in reading it early on (before lifecycle runs)? - Should we create a shared library for `project.toml` parsing? # Spec. Changes [spec-changes]: #spec-changes -See [buildpacks/spec PR #176](https://github.com/buildpacks/spec/pull/176) +See [buildpacks/spec PR #172](https://github.com/buildpacks/spec/pull/172) From 58cf956302ffacafeab24b919de436cb05f800a5 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 14 Jan 2021 14:23:58 -0600 Subject: [PATCH 05/12] suggestions from Emily Signed-off-by: Jesse Brown --- text/0000-move-analyze-phase.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/text/0000-move-analyze-phase.md b/text/0000-move-analyze-phase.md index 0bb683a24..e49fc3ba1 100644 --- a/text/0000-move-analyze-phase.md +++ b/text/0000-move-analyze-phase.md @@ -16,7 +16,6 @@ This is a proposal to re-order and adjust Lifecycle phases. Specifically, moving # Definitions [definitions]: #definitions -* __stack descriptor__ - an enhanced version of the [`stack.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#stacktoml-toml) * __project descriptor__ - the [`project.toml`](https://github.com/buildpacks/spec/blob/main/extensions/project-descriptor.md) extension specification # Motivation @@ -35,13 +34,13 @@ The analyze phase will now run before the detect phase. The analyze phase will h ## Responsibilities * Stack validation, to ensure that a new run-image is campatible with the previous app image -* Retrive run-image mixins, which will be used by subsequent phases +* Retrieve identifier (imageID or digest), stack ID, and mixins, which will be used by subsequent phases * Validation of registry credentials, to avoid a long build that fails during export phase * Parsing the project descriptor and performance various operations based on its contents, include: - downloading buildpacks - creating ephemeral buildpacks - applying include and exclude rules - - modifying environment variables + - adding environment variables to /env - producing an [`order.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml) to be consumed by later phases ## Inputs @@ -51,14 +50,15 @@ The analyze phase will now run before the detect phase. The analyze phase will h * Stack ID * Project descriptor (optional) * App source code +* Previous Image +* Destination tag(s) ## Output * Exit status * Info-level logs to `stdout` * Error-level logs to `stderr` -* Stack descriptor -* Analysis metadata [`analyzed.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#analyzedtoml-toml) +* Analysis metadata [`analyzed.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#analyzedtoml-toml), including run-image information. * Buildpacks (derived from inline buildpacks in project descriptor, or buildpacks in project descriptor that are not present in the builder) * Buildpacks order [`order.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml) * Lifecycle configuration (derived from configuration in project descriptor) @@ -77,6 +77,8 @@ The [logic in `pack` that parses a `project.toml`](https://github.com/buildpacks The [logic in the `analyzer` phase that analyzes layers](hhttps://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L54-L116) would be moved to the `restorer`. `restorer` already takes in `group.toml` as a flag. +The app source code (which may be provided to the prepare either as a directory, volume, or tarball) would be mutated (either by copying it to a new location, or making changes directly). The `analyzer` may delete files to apply the include and exclude rules from `project.toml`. + # Drawbacks [drawbacks]: #drawbacks @@ -98,6 +100,7 @@ The [logic in the `analyzer` phase that analyzes layers](hhttps://github.com/bui - Does `pack` still need to parse `project.toml`, or is there value in reading it early on (before lifecycle runs)? - Should we create a shared library for `project.toml` parsing? +- How should `analyzed.toml` be changed to include run-image information (mixins) # Spec. Changes [spec-changes]: #spec-changes From d0d712c28a065e93005ba1ca3de76b8eaff8d994 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 14 Jan 2021 15:13:57 -0600 Subject: [PATCH 06/12] added few missing inputs that are needed Signed-off-by: Jesse Brown --- text/0000-move-analyze-phase.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/text/0000-move-analyze-phase.md b/text/0000-move-analyze-phase.md index e49fc3ba1..df0fe9a23 100644 --- a/text/0000-move-analyze-phase.md +++ b/text/0000-move-analyze-phase.md @@ -52,6 +52,9 @@ The analyze phase will now run before the detect phase. The analyze phase will h * App source code * Previous Image * Destination tag(s) +* gid +* uid +* log-level ## Output From 986f4190b96dff57fce95135da6a3b7f2499fe6f Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 14 Jan 2021 16:45:49 -0600 Subject: [PATCH 07/12] suggestions from Emily Signed-off-by: Jesse Brown --- text/0000-move-analyze-phase.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-move-analyze-phase.md b/text/0000-move-analyze-phase.md index df0fe9a23..07baa9c54 100644 --- a/text/0000-move-analyze-phase.md +++ b/text/0000-move-analyze-phase.md @@ -48,7 +48,8 @@ The analyze phase will now run before the detect phase. The analyze phase will h * Log level * Run-image * Stack ID -* Project descriptor (optional) +* [stack.toml](https://github.com/buildpacks/spec/blob/main/platform.md#stacktoml-toml) +* Project descriptor * App source code * Previous Image * Destination tag(s) From 829dc4f747ddb0f04a959621e3f4ab139c785f61 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Tue, 19 Jan 2021 14:37:25 -0600 Subject: [PATCH 08/12] Suggestions from Natalie Signed-off-by: Jesse Brown --- text/0000-move-analyze-phase.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/text/0000-move-analyze-phase.md b/text/0000-move-analyze-phase.md index 07baa9c54..b68325392 100644 --- a/text/0000-move-analyze-phase.md +++ b/text/0000-move-analyze-phase.md @@ -36,7 +36,7 @@ The analyze phase will now run before the detect phase. The analyze phase will h * Stack validation, to ensure that a new run-image is campatible with the previous app image * Retrieve identifier (imageID or digest), stack ID, and mixins, which will be used by subsequent phases * Validation of registry credentials, to avoid a long build that fails during export phase -* Parsing the project descriptor and performance various operations based on its contents, include: +* Parsing the project descriptor and performing various operations based on its contents, include: - downloading buildpacks - creating ephemeral buildpacks - applying include and exclude rules @@ -56,6 +56,10 @@ The analyze phase will now run before the detect phase. The analyze phase will h * gid * uid * log-level +* Path to output analyzed.toml +* Layers Directory +* Skip Layers +* Daemon ## Output @@ -71,7 +75,7 @@ The analyze phase will now run before the detect phase. The analyze phase will h # How it Works [how-it-works]: #how-it-works -A platform MUST execute the analyze phase either by invoking the `/cnb/lifecycle/detector` binary or by executing `/cnb/lifecycle/creator`. +A platform MUST execute the analyze phase either by invoking the `/cnb/lifecycle/analyzer` binary or by executing `/cnb/lifecycle/creator`. The `analyzer` binary will have access to the [`Keychain`](https://github.com/buildpacks/lifecycle/blob/main/auth/env_keychain.go), and MUST NOT execute arbitrary code provided by either the buildpack user or buildpack author. From 4b835b4534e3da13413ee599362911b401b0b518 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Tue, 19 Jan 2021 15:01:08 -0600 Subject: [PATCH 09/12] More suggestions from Natalie Signed-off-by: Jesse Brown --- text/0000-move-analyze-phase.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-move-analyze-phase.md b/text/0000-move-analyze-phase.md index b68325392..e3a1c9405 100644 --- a/text/0000-move-analyze-phase.md +++ b/text/0000-move-analyze-phase.md @@ -31,7 +31,7 @@ Doing this would support the following features and capabilities: The analyze phase will now run before the detect phase. The analyze phase will have access to secrets and credentials used to access registries and other services, as it does today. Analyze will no longer require a [`group.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#grouptoml-toml). It will do this by splitting off some of the responsibilites into the restore phase. -## Responsibilities +## New Responsibilities in analzyer * Stack validation, to ensure that a new run-image is campatible with the previous app image * Retrieve identifier (imageID or digest), stack ID, and mixins, which will be used by subsequent phases From 3da941740dfcf3f0c92b528940a4b786124bbc1a Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Wed, 20 Jan 2021 15:01:45 -0600 Subject: [PATCH 10/12] Remove project.toml parsing from proposal We can add these in later or in another optional phase. Signed-off-by: Jesse Brown --- text/0000-move-analyze-phase.md | 20 +------------------- 1 file changed, 1 insertion(+), 19 deletions(-) diff --git a/text/0000-move-analyze-phase.md b/text/0000-move-analyze-phase.md index e3a1c9405..57ee74819 100644 --- a/text/0000-move-analyze-phase.md +++ b/text/0000-move-analyze-phase.md @@ -36,12 +36,6 @@ The analyze phase will now run before the detect phase. The analyze phase will h * Stack validation, to ensure that a new run-image is campatible with the previous app image * Retrieve identifier (imageID or digest), stack ID, and mixins, which will be used by subsequent phases * Validation of registry credentials, to avoid a long build that fails during export phase -* Parsing the project descriptor and performing various operations based on its contents, include: - - downloading buildpacks - - creating ephemeral buildpacks - - applying include and exclude rules - - adding environment variables to /env - - producing an [`order.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml) to be consumed by later phases ## Inputs @@ -49,16 +43,13 @@ The analyze phase will now run before the detect phase. The analyze phase will h * Run-image * Stack ID * [stack.toml](https://github.com/buildpacks/spec/blob/main/platform.md#stacktoml-toml) -* Project descriptor -* App source code * Previous Image +* Cache Image * Destination tag(s) * gid * uid -* log-level * Path to output analyzed.toml * Layers Directory -* Skip Layers * Daemon ## Output @@ -67,10 +58,6 @@ The analyze phase will now run before the detect phase. The analyze phase will h * Info-level logs to `stdout` * Error-level logs to `stderr` * Analysis metadata [`analyzed.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#analyzedtoml-toml), including run-image information. -* Buildpacks (derived from inline buildpacks in project descriptor, or buildpacks in project descriptor that are not present in the builder) -* Buildpacks order [`order.toml`](https://github.com/buildpacks/spec/blob/main/platform.md#ordertoml-toml) -* Lifecycle configuration (derived from configuration in project descriptor) -* Mutated app source code (applying include and exclude rules in project descriptor) # How it Works [how-it-works]: #how-it-works @@ -85,13 +72,10 @@ The [logic in `pack` that parses a `project.toml`](https://github.com/buildpacks The [logic in the `analyzer` phase that analyzes layers](hhttps://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L54-L116) would be moved to the `restorer`. `restorer` already takes in `group.toml` as a flag. -The app source code (which may be provided to the prepare either as a directory, volume, or tarball) would be mutated (either by copying it to a new location, or making changes directly). The `analyzer` may delete files to apply the include and exclude rules from `project.toml`. - # Drawbacks [drawbacks]: #drawbacks * Platform maintainers will need to update the order of their container execution and also update flags for `analyzer`, `detector`, and `restorer`. -* Lifecycle will now take on the responsibility of processing `project.toml` # Alternatives [alternatives]: #alternatives @@ -106,8 +90,6 @@ The app source code (which may be provided to the prepare either as a directory, # Unresolved Questions [unresolved-questions]: #unresolved-questions -- Does `pack` still need to parse `project.toml`, or is there value in reading it early on (before lifecycle runs)? -- Should we create a shared library for `project.toml` parsing? - How should `analyzed.toml` be changed to include run-image information (mixins) # Spec. Changes From 5479f531207e783cef9ec7285a5e92289e433530 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Wed, 20 Jan 2021 15:06:05 -0600 Subject: [PATCH 11/12] Removing more project.toml references Signed-off-by: Jesse Brown --- text/0000-move-analyze-phase.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/text/0000-move-analyze-phase.md b/text/0000-move-analyze-phase.md index 57ee74819..5118c5425 100644 --- a/text/0000-move-analyze-phase.md +++ b/text/0000-move-analyze-phase.md @@ -13,18 +13,12 @@ This is a proposal to re-order and adjust Lifecycle phases. Specifically, moving "analyze" before "detect". -# Definitions -[definitions]: #definitions - -* __project descriptor__ - the [`project.toml`](https://github.com/buildpacks/spec/blob/main/extensions/project-descriptor.md) extension specification - # Motivation [motivation]: #motivation Doing this would support the following features and capabilities: * [Stack buildpacks](https://github.com/buildpacks/rfcs/pull/111), which require a phase to read run-image mixins validation prior to detection -* [Inline buildpacks](https://github.com/buildpacks/rfcs/blob/main/text/0048-inline-buildpack.md), which require parsing of the `project.toml` in the lifecycle -* [Lifecycle configuration](https://github.com/buildpacks/rfcs/pull/128) +* Validating registry access for all images that are used can happen prior to `detector` or `builder` phases, providing faster failures for end users. # What it is [what-it-is]: #what-it-is @@ -68,8 +62,6 @@ The `analyzer` binary will have access to the [`Keychain`](https://github.com/bu The [logic in the `analyzer` phase that reads image metadata and outputs an `analyzed.toml`](https://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L34-L40) would be remain. -The [logic in `pack` that parses a `project.toml`](https://github.com/buildpacks/pack/blob/main/project/project.go) would be copied or moved into the `analyzer`. - The [logic in the `analyzer` phase that analyzes layers](hhttps://github.com/buildpacks/lifecycle/blob/main/analyzer.go#L54-L116) would be moved to the `restorer`. `restorer` already takes in `group.toml` as a flag. # Drawbacks From 7eb541e73d4cc3db40a51221312200966ab21a86 Mon Sep 17 00:00:00 2001 From: Jesse Brown Date: Thu, 21 Jan 2021 13:14:27 -0600 Subject: [PATCH 12/12] Add cache-dir back In order to determine if cache should be restored/analyzed at all if the stack change. Future use. Signed-off-by: Jesse Brown --- text/0000-move-analyze-phase.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-move-analyze-phase.md b/text/0000-move-analyze-phase.md index 5118c5425..e39b04bfd 100644 --- a/text/0000-move-analyze-phase.md +++ b/text/0000-move-analyze-phase.md @@ -40,6 +40,7 @@ The analyze phase will now run before the detect phase. The analyze phase will h * Previous Image * Cache Image * Destination tag(s) +* Cache Dir * gid * uid * Path to output analyzed.toml