Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TEP-0089] SPIRE for non-falsifiable provenance. Setup the test environment. #6553

Merged
merged 1 commit into from
May 4, 2023
Merged

[TEP-0089] SPIRE for non-falsifiable provenance. Setup the test environment. #6553

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions test/e2e-common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,69 @@ function install_pipeline_crd_version() {
verify_pipeline_installation
}

# Add the provided spiffeId to the spire-server.
function spire_apply() {
jerop marked this conversation as resolved.
Show resolved Hide resolved
if [ $# -lt 2 -o "$1" != "-spiffeID" ]; then
echo "spire_apply requires a spiffeID as the first arg" >&2
exit 1
fi
echo "Checking if spiffeID $2 already exists..."
show=$(kubectl exec -n spire deployment/spire-server -- \
/opt/spire/bin/spire-server entry show $1 $2)
if [ "$show" != "Found 0 entries" ]; then
# delete to recreate
entryid=$(echo "$show" | grep "^Entry ID" | cut -f2 -d:)
echo "Deleting previously existing spiffeID $2 ..."
kubectl exec -n spire deployment/spire-server -- \
/opt/spire/bin/spire-server entry delete -entryID $entryid
fi
echo "Adding spiffeID $2 to spire-server."
kubectl exec -n spire deployment/spire-server -- \
/opt/spire/bin/spire-server entry create "$@"
}

function install_spire() {
jerop marked this conversation as resolved.
Show resolved Hide resolved
echo ">> Deploying Spire"
DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"

echo "Creating SPIRE namespace..."
kubectl create ns spire

echo "Applying SPIFFE CSI Driver configuration..."
kubectl apply -f "$DIR"/testdata/spire/spiffe-csi-driver.yaml

echo "Deploying SPIRE server"
kubectl apply -f "$DIR"/testdata/spire/spire-server.yaml

echo "Deploying SPIRE agent"
kubectl apply -f "$DIR"/testdata/spire/spire-agent.yaml

wait_until_pods_running spire || fail_test "SPIRE did not come up"

spire_apply \
-spiffeID spiffe://example.org/ns/spire/node/example \
-selector k8s_psat:cluster:example-cluster \
-selector k8s_psat:agent_ns:spire \
-selector k8s_psat:agent_sa:spire-agent \
-node
spire_apply \
-spiffeID spiffe://example.org/ns/tekton-pipelines/sa/tekton-pipelines-controller \
-parentID spiffe://example.org/ns/spire/node/example \
-selector k8s:ns:tekton-pipelines \
-selector k8s:pod-label:app:tekton-pipelines-controller \
-selector k8s:sa:tekton-pipelines-controller \
-admin
}

function patch_pipeline_spire() {
kubectl patch \
deployment tekton-pipelines-controller \
-n tekton-pipelines \
--patch-file "$DIR"/testdata/patch/pipeline-controller-spire.json

verify_pipeline_installation
}

function verify_pipeline_installation() {
# Make sure that everything is cleaned up in the current namespace.
delete_pipeline_resources
Expand Down
13 changes: 13 additions & 0 deletions test/e2e-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,18 @@ install_pipeline_crd

failed=0

function add_spire() {
local gate="$1"
if [ "$gate" == "alpha" ] ; then
DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
printf "Setting up environment for alpha features"
install_spire
patch_pipeline_spire
kubectl apply -n tekton-pipelines -f "$DIR"/testdata/spire/config-spire.yaml
failed=0
fi
}

function set_feature_gate() {
local gate="$1"
local resolver="false"
Expand Down Expand Up @@ -91,6 +103,7 @@ function run_e2e() {
fi
}

add_spire "$PIPELINE_FEATURE_GATE"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this line come after set_feature_gate "$PIPELINE_FEATURE_GATE"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order does not matter as we are not depending on whether spire feature is enabled or not to install SPIRE, which is what add_spire() does.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But add_spire only installs spire when the alpha feature flag is enabled right? set_feature_gate "$PIPELINE_FEATURE_GATE" is the place where we set alpha feature flag for integration test. If add_spire is called before set_feature_gate, it seems to me the if condition in add_spire will never be met?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$PIPELINE_FEATURE_GATE is an env variable which is passed in into the e2e-tests.sh script (or rather set before e2e-tests.sh). This can be alpha or stable.
We set the feature flags based on this env variable.
Both add_spire and set_feature_gate depend on this env variable. set_feature_gate sets the feature_flags config based on this env variable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aha you're right, my bad. I don't why I ignored the fact the add_spire uses $PIPELINE_FEATURE_GATE as a parameter :/

Thank you for explaining it.

set_feature_gate "$PIPELINE_FEATURE_GATE"
set_result_extraction_method "$RESULTS_FROM"
run_e2e
Expand Down
57 changes: 57 additions & 0 deletions test/testdata/patch/pipeline-controller-spire.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
{
"spec":{
"template":{
"spec":{
"$setElementOrder/containers":[
{
"name":"tekton-pipelines-controller"
}
],
"$setElementOrder/volumes":[
{
"name":"config-logging"
},
{
"name":"config-registry-cert"
},
{
"name":"spiffe-workload-api"
}
],
"containers":[
{
"$setElementOrder/volumeMounts":[
{
"mountPath":"/etc/config-logging"
},
{
"mountPath":"/etc/config-registry-cert"
},
{
"mountPath":"/spiffe-workload-api"
}
],
"name":"tekton-pipelines-controller",
"volumeMounts":[
{
"mountPath":"/spiffe-workload-api",
"name":"spiffe-workload-api",
"readOnly":true
}
]
}
],
"volumes":[
{
"csi":{
"driver":"csi.spiffe.io",
"readOnly":true
},
"name":"spiffe-workload-api"
}
]
}
}
}
}

17 changes: 17 additions & 0 deletions test/testdata/spire/config-spire.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: config-spire
namespace: tekton-pipelines
labels:
app.kubernetes.io/instance: default
app.kubernetes.io/part-of: tekton-pipelines
data:
# spire-trust-domain specifies the SPIRE trust domain to use.
spire-trust-domain: "example.org"
# spire-socket-path specifies the SPIRE agent socket for SPIFFE workload API.
spire-socket-path: "unix:///spiffe-workload-api/spire-agent.sock"
# spire-server-addr specifies the SPIRE server address for workload/node registration.
spire-server-addr: "spire-server.spire.svc.cluster.local:8081"
# spire-node-alias-prefix specifies the SPIRE node alias prefix to use.
spire-node-alias-prefix: "/tekton-node/"
20 changes: 20 additions & 0 deletions test/testdata/spire/spiffe-csi-driver.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: "csi.spiffe.io"
spec:
# Only ephemeral, inline volumes are supported. There is no need for a
# controller to provision and attach volumes.
attachRequired: false

# Request the pod information which the CSI driver uses to verify that an
# ephemeral mount was requested.
podInfoOnMount: true

# Don't change ownership on the contents of the mount since the Workload API
# Unix Domain Socket is typically open to all (i.e. 0777).
fsGroupPolicy: None

# Declare support for ephemeral volumes only.
volumeLifecycleModes:
- Ephemeral
Loading