Libvirt test e2e failing due to lack of node-role.kubernetes.io/worker label

[wainersm]

Today I ran the latest (commit 79d309d) libvirt e2e test but it failed as:

<OMITTED>

Wait at least one worker be Ready
Wait system pods be running or completed

time="2023-06-21T15:37:22-03:00" level=info msg="Deploy the Cloud API Adaptor"
time="2023-06-21T15:37:22-03:00" level=info msg="Install the controller manager"
Wait for the cc-operator-controller-manager deployment be available
time="2023-06-21T15:38:29-03:00" level=info msg="Customize the overlay yaml file"
time="2023-06-21T15:38:32-03:00" level=info msg="Install the cloud-api-adaptor"
Wait for the cc-operator-daemon-install DaemonSet be available
F0621 15:39:32.565344   78821 env.go:369] Setup failure: timed out waiting for the condition
FAIL    github.com/confidential-containers/cloud-api-adaptor/test/e2e   795.612s
FAIL
make: *** [Makefile:91: test-e2e] Error 1

It turned out the cc-operator-daemon-install daemonSet didn't get available because the worker node hadn't the node-role.kubernetes.io/worker label:

$ kubectl get nodes
NAME                   STATUS   ROLES           AGE    VERSION
peer-pods-ctlplane-0   Ready    control-plane   129m   v1.27.3
peer-pods-worker-0     Ready    <none>          127m   v1.27.3

In order to solve that problem, it will be needed to apply the node label after the cluster is created (see https://github.com/confidential-containers/cloud-api-adaptor/blob/main/test/provisioner/provision_libvirt.go#L86).

The Azure implementation also apply the label to the node (see https://github.com/confidential-containers/cloud-api-adaptor/blob/main/test/provisioner/provision_azure.go#L308) however it uses kubectl. It would be nice if we could have a Go implementation instead, that new method could be hosted in https://github.com/confidential-containers/cloud-api-adaptor/blob/main/test/e2e/common.go

How to reproduce the problem:

$ git clone https://github.com/confidential-containers/cloud-api-adaptor
$ cd cloud-api-adaptor
$ TEST_PROVISION=yes CLOUD_PROVIDER=libvirt make test-e2e

[bpradipt]

Related issues
#780
confidential-containers/operator#195

[bookinabox]

I'll take a look

[bookinabox]

I'm still getting:

F0705 14:29:45.303677 52645 env.go:369] Setup failure: timed out waiting for the condition FAIL github.com/confidential-containers/cloud-api-adaptor/test/e2e 995.619s FAIL

after adding the worker label in createCluster. Unsure if this is an issue on my end, but will investigate further.

[wainersm]

I'm still getting:

F0705 14:29:45.303677 52645 env.go:369] Setup failure: timed out waiting for the condition FAIL github.com/confidential-containers/cloud-api-adaptor/test/e2e 995.619s FAIL

after adding the worker label in createCluster. Unsure if this is an issue on my end, but will investigate further.

It is not in your end, I also get this error. Actually I've a fix that I am testing right now. I will have it posted today.

[wainersm]

@bookinabox please have a look at #1147

[bookinabox]

I'm still getting:
F0705 14:29:45.303677 52645 env.go:369] Setup failure: timed out waiting for the condition FAIL github.com/confidential-containers/cloud-api-adaptor/test/e2e 995.619s FAIL
after adding the worker label in createCluster. Unsure if this is an issue on my end, but will investigate further.

It is not in your end, I also get this error. Actually I've a fix that I am testing right now. I will have it posted today.

I changed the timing and it seems to still time out. How did you diagnose the problem initially? Is there a more verbose logging I can try to dig through?

edit: looks like for me even setting the timeout to 10+ minutes still times out. I suspect I might have a different issue.