Skip to content

Commit

Permalink
BUILD-273 Volumes override by Build and BuildRun
Browse files Browse the repository at this point in the history 
Build and BuildRun objects can now override volumes defined by
BuildStrategy in case they are overridable.
Only overridable volumes are allowed to be overriden.
  • Loading branch information
Alice Rum committed Apr 5, 2022
1 parent abc2df8 commit 02163e6
Show file tree
Hide file tree
Showing 9 changed files with 6,513 additions and 37 deletions.
4,811 changes: 4,803 additions & 8 deletions deploy/crds/shipwright.io_buildruns.yaml
View file

Large diffs are not rendered by default.

1,551 changes: 1,551 additions & 0 deletions deploy/crds/shipwright.io_builds.yaml
View file

Large diffs are not rendered by default.

75 changes: 62 additions & 13 deletions deploy/crds/shipwright.io_buildstrategies.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1720,8 +1720,7 @@ spec:
CSI driver is meant to be used that way - see the documentation
of the driver for more information. \n A pod can use both
types of ephemeral volumes and persistent volumes at the
same time. \n This is a beta feature and only available
when the GenericEphemeralVolume feature gate is enabled."
same time."
properties:
volumeClaimTemplate:
description: "Will be used to create a stand-alone PVC
Expand Down Expand Up @@ -1769,15 +1768,61 @@ spec:
description: 'This field can be used to specify
either: * An existing VolumeSnapshot object
(snapshot.storage.k8s.io/VolumeSnapshot) *
An existing PVC (PersistentVolumeClaim) *
An existing custom resource that implements
data population (Alpha) In order to use custom
resource types that implement data population,
the AnyVolumeDataSource feature gate must
be enabled. If the provisioner or an external
controller can support the specified data
source, it will create a new volume based
on the contents of the specified data source.'
An existing PVC (PersistentVolumeClaim) If
the provisioner or an external controller
can support the specified data source, it
will create a new volume based on the contents
of the specified data source. If the AnyVolumeDataSource
feature gate is enabled, this field will always
have the same contents as the DataSourceRef
field.'
properties:
apiGroup:
description: APIGroup is the group for the
resource being referenced. If APIGroup
is not specified, the specified Kind must
be in the core API group. For any other
third-party types, APIGroup is required.
type: string
kind:
description: Kind is the type of resource
being referenced
type: string
name:
description: Name is the name of resource
being referenced
type: string
required:
- kind
- name
type: object
dataSourceRef:
description: 'Specifies the object from which
to populate the volume with data, if a non-empty
volume is desired. This may be any local object
from a non-empty API group (non core object)
or a PersistentVolumeClaim object. When this
field is specified, volume binding will only
succeed if the type of the specified object
matches some installed volume populator or
dynamic provisioner. This field will replace
the functionality of the DataSource field
and as such if both fields are non-empty,
they must have the same value. For backwards
compatibility, both fields (DataSource and
DataSourceRef) will be set to the same value
automatically if one of them is empty and
the other is non-empty. There are two important
differences between DataSource and DataSourceRef:
* While DataSource only allows two specific
types of objects, DataSourceRef allows any
non-core object, as well as PersistentVolumeClaim
objects. * While DataSource ignores disallowed
values (dropping them), DataSourceRef preserves
all values, and generates an error if a disallowed
value is specified. (Alpha) Using this field
requires the AnyVolumeDataSource feature gate
to be enabled.'
properties:
apiGroup:
description: APIGroup is the group for the
Expand All @@ -1800,8 +1845,12 @@ spec:
type: object
resources:
description: 'Resources represents the minimum
resources the volume should have. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources'
resources the volume should have. If RecoverVolumeExpansionFailure
feature is enabled users are allowed to specify
resource requirements that are lower than
previous value but must still be higher than
capacity recorded in the status field of the
claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources'
properties:
limits:
additionalProperties:
Expand Down
75 changes: 62 additions & 13 deletions deploy/crds/shipwright.io_clusterbuildstrategies.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1720,8 +1720,7 @@ spec:
CSI driver is meant to be used that way - see the documentation
of the driver for more information. \n A pod can use both
types of ephemeral volumes and persistent volumes at the
same time. \n This is a beta feature and only available
when the GenericEphemeralVolume feature gate is enabled."
same time."
properties:
volumeClaimTemplate:
description: "Will be used to create a stand-alone PVC
Expand Down Expand Up @@ -1769,15 +1768,61 @@ spec:
description: 'This field can be used to specify
either: * An existing VolumeSnapshot object
(snapshot.storage.k8s.io/VolumeSnapshot) *
An existing PVC (PersistentVolumeClaim) *
An existing custom resource that implements
data population (Alpha) In order to use custom
resource types that implement data population,
the AnyVolumeDataSource feature gate must
be enabled. If the provisioner or an external
controller can support the specified data
source, it will create a new volume based
on the contents of the specified data source.'
An existing PVC (PersistentVolumeClaim) If
the provisioner or an external controller
can support the specified data source, it
will create a new volume based on the contents
of the specified data source. If the AnyVolumeDataSource
feature gate is enabled, this field will always
have the same contents as the DataSourceRef
field.'
properties:
apiGroup:
description: APIGroup is the group for the
resource being referenced. If APIGroup
is not specified, the specified Kind must
be in the core API group. For any other
third-party types, APIGroup is required.
type: string
kind:
description: Kind is the type of resource
being referenced
type: string
name:
description: Name is the name of resource
being referenced
type: string
required:
- kind
- name
type: object
dataSourceRef:
description: 'Specifies the object from which
to populate the volume with data, if a non-empty
volume is desired. This may be any local object
from a non-empty API group (non core object)
or a PersistentVolumeClaim object. When this
field is specified, volume binding will only
succeed if the type of the specified object
matches some installed volume populator or
dynamic provisioner. This field will replace
the functionality of the DataSource field
and as such if both fields are non-empty,
they must have the same value. For backwards
compatibility, both fields (DataSource and
DataSourceRef) will be set to the same value
automatically if one of them is empty and
the other is non-empty. There are two important
differences between DataSource and DataSourceRef:
* While DataSource only allows two specific
types of objects, DataSourceRef allows any
non-core object, as well as PersistentVolumeClaim
objects. * While DataSource ignores disallowed
values (dropping them), DataSourceRef preserves
all values, and generates an error if a disallowed
value is specified. (Alpha) Using this field
requires the AnyVolumeDataSource feature gate
to be enabled.'
properties:
apiGroup:
description: APIGroup is the group for the
Expand All @@ -1800,8 +1845,12 @@ spec:
type: object
resources:
description: 'Resources represents the minimum
resources the volume should have. More info:
https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources'
resources the volume should have. If RecoverVolumeExpansionFailure
feature is enabled users are allowed to specify
resource requirements that are lower than
previous value but must still be higher than
capacity recorded in the status field of the
claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources'
properties:
limits:
additionalProperties:
Expand Down
5 changes: 5 additions & 0 deletions pkg/apis/build/v1alpha1/build_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,11 @@ type BuildSpec struct {
// Env contains additional environment variables that should be passed to the build container
// +optional
Env []corev1.EnvVar `json:"env,omitempty"`

// Volumes contains volume Overrides of the BuildStrategy volumes in case those are allowed
// to be overriden. Must only contain volumes that exist in the corresponding BuildStrategy
// +optional
Volumes []BuildVolume `json:"volumes,omitempty"`
}

// StrategyName returns the name of the configured strategy, or 'undefined' in
Expand Down
5 changes: 5 additions & 0 deletions pkg/apis/build/v1alpha1/buildrun_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ type BuildRunSpec struct {
//
// +optional
Env []corev1.EnvVar `json:"env,omitempty"`

// Volumes contains volume Overrides of the BuildStrategy volumes in case those are allowed
// to be overriden. Must only contain volumes that exist in the corresponding BuildStrategy
// +optional
Volumes []BuildVolume `json:"volumes,omitempty"`
}

// BuildRunRequestedState defines the buildrun state the user can provide to override whatever is the current state.
Expand Down
14 changes: 14 additions & 0 deletions pkg/apis/build/v1alpha1/zz_generated.deepcopy.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pkg/reconciler/buildrun/resources/taskrun.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ func GenerateTaskSpec(

// Add volumes from the strategy to generated task spec
// will have to add buildrun volumes here for merging later, too
volumes, err := volumes.TaskSpecVolumes(existingVolumeMounts, buildVolumes, nil)
volumes, err := volumes.TaskSpecVolumes(existingVolumeMounts, buildVolumes, build.Spec.Volumes, buildRun.Spec.Volumes)
if err != nil {
return nil, err
}
Expand Down
12 changes: 10 additions & 2 deletions pkg/volumes/volumes.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,19 @@ func NewStepVolumeMount(name string, readOnly bool) StepVolumeMount {
func TaskSpecVolumes(
existingVolumeMounts []StepVolumeMount,
strategyVolumes []buildv1alpha1.BuildVolume,
buildVolumes []buildv1alpha1.BuildVolume,
buildrunVolumes []buildv1alpha1.BuildVolume,
) ([]corev1.Volume, error) {
res := []corev1.Volume{}

volumes, err := MergeBuildVolumes(strategyVolumes, buildrunVolumes)
// first we merge build volumes into the strategy ones, next we merge
// build run volumes into result of the previous merge.
// eventual list of volumes will be added to the generated TaskSpec object
volumes, err := MergeBuildVolumes(strategyVolumes, buildVolumes)
if err != nil {
return nil, err
}
volumes, err = MergeBuildVolumes(volumes, buildrunVolumes)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -79,7 +87,7 @@ func TaskSpecVolumes(
// MergeBuildVolumes merges Build Volumes from one list into the other. It only allows to merge those that have property
// Overridable set to true. In case it is empty or false, it is not allowed to be overridden, so Volume cannot be merged
// Merging in this context means copying the VolumeSource from one object to the other.
func MergeBuildVolumes(new []buildv1alpha1.BuildVolume, into []buildv1alpha1.BuildVolume) ([]buildv1alpha1.BuildVolume, error) {
func MergeBuildVolumes(into []buildv1alpha1.BuildVolume, new []buildv1alpha1.BuildVolume) ([]buildv1alpha1.BuildVolume, error) {
if len(new) == 0 && len(into) == 0 {
return []buildv1alpha1.BuildVolume{}, nil
}
Expand Down

0 comments on commit 02163e6

Please sign in to comment.