KubernetesPipelineTest.errorPod flake revealed bug in Reaper

[jglick]

While checking why #1415 did not automatically deploy, I saw that the trunk commit failed with a flake in KubernetesPipelineTest.errorPod

Expected: a string containing "jnlp -- terminated (1)"
     but: was "Started
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Created Pod: kubernetes kubernetes-plugin-test/error-pod-1-rpsj1-z7xj2-7nnm8
kubernetes-plugin-test/error-pod-1-rpsj1-z7xj2-7nnm8 Container jnlp was terminated (Exit Code: 1, Reason: Error)
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Queue task was cancelled
org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 65b59291-4c40-4b7c-925a-ea0448f6dbcc
Finished: ABORTED
"

I found that the same failure could be reproduced easily:

diff --git src/test/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/KubernetesPipelineTest.java src/test/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/KubernetesPipelineTest.java
index f9ffdace..3df21eb8 100644
--- src/test/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/KubernetesPipelineTest.java
+++ src/test/java/org/csanchez/jenkins/plugins/kubernetes/pipeline/KubernetesPipelineTest.java
@@ -505,6 +505,7 @@ public class KubernetesPipelineTest extends AbstractKubernetesPipelineTest {
     public void errorPod() throws Exception {
         r.waitForMessage("jnlp -- terminated (1)", b);
         r.waitForMessage("Foo", b);
+        Thread.sleep(5_000);
         b.doKill();
     }
 

Turns out the eager hard-kill was preventing the normal event sequence from being observed:

[Pipeline] // node
[Pipeline] }

- jnlp -- terminated (1)
-----Logs-------------
Foo

[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Hard kill!
Finished: ABORTED

The log lines being asserted were introduced in #1050. It appears that #1118 prevented these from being shown in some cases, but due to the odd structure of the test this regression was not caught. Fixing the production code to print failing pod log lines before cancelling the queue item (which will abort the node step and thus also podTemplate), and improving the test to wait for the build to terminate naturally.

[Vlatombe]

LGTM