Jib Ubuntu and MacOS Continuous Builds are failing

[mpeddada1]

As observed by @zhumin8, the jib-ubuntu and jib-macos continious tests have been consistently failing.

The failure appears to originate from the jib-cli integration tests, particularly:

com.google.cloud.tools.jib.cli.WarCommandTest > testWar_jetty FAILED
    java.lang.AssertionError: expected:<Hello world> but was:<null>
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:120)
        at org.junit.Assert.assertEquals(Assert.java:146)
        at com.google.cloud.tools.jib.api.HttpRequestTester.verifyBody(HttpRequestTester.java:35)
        at com.google.cloud.tools.jib.cli.WarCommandTest.testWar_jetty(WarCommandTest.java:99)

Test Fusion Link

[mpeddada1]

More observations: These checks started failing on Feb 2, 2024, starting with this PR

[chanseokoh]

IIRC, the Jetty tests were already failing when I tried to file the PR.

[mpeddada1]

Hm I see, that is good to know. Thanks for the input @chanseokoh. Will try running these locally to see what may have changed.

[mpeddada1]

Observations:

• Running docker image with jetty 11.x base image with docker run -p8080:8080 exploded-war results in successful build. The startup logs show the following:

2024-03-19 22:18:24.981:INFO :oejs.Server:main: jetty-11.0.11; built: 2022-06-21T21:42:55.454Z; git: 58487315cb75e0f5c81cc6fa50096cbeb3b9554e; jvm 11.0.16+8
2024-03-19 22:18:25.033:INFO :oejdp.ScanningAppProvider:main: Deployment monitor [file:///var/lib/jetty/webapps/]
2024-03-19 22:18:25.723:INFO :oejss.DefaultSessionIdManager:main: Session workerName=node0
2024-03-19 22:18:25.761:INFO :oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@6f46426d{ROOT,/,file:///var/lib/jetty/webapps/ROOT/,AVAILABLE}{/var/lib/jetty/webapps/ROOT}
2024-03-19 22:18:25.803:INFO :oejs.AbstractConnector:main: Started ServerConnector@4b6690c0{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2024-03-19 22:18:25.820:INFO :oejs.Server:main: Started Server@49438269{STARTING}[11.0.11,sto=5000] @1780ms

• Running docker image with jetty 10.0.8 base image with docker run -p8080:8080 exploded-war results in:

java.lang.NoClassDefFoundError: jakarta/servlet/http/HttpServlet
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
	at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
	at org.eclipse.jetty.webapp.WebAppClassLoader.foundClass(WebAppClassLoader.java:594)
	at org.eclipse.jetty.webapp.WebAppClassLoader.loadAsResource(WebAppClassLoader.java:567)
	at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:502)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at org.eclipse.jetty.util.Loader.loadClass(Loader.java:59)
	at org.eclipse.jetty.servlet.BaseHolder.doStart(BaseHolder.java:97)

• Running the docker image with jetty:latest results in the following logs (with both jakarta.servlet:jakarta.servlet-api:5.0.0 and jakarta.servlet:jakarta.servlet-api:6.0.0). Note the deployment logic missing from the logs:

2024-03-19 02:03:10.216:INFO :oejs.Server:main: jetty-12.0.7; built: 2024-02-29T21:19:41.771Z; git: c89aca8fd34083befd79f328a3b8b6ffff04347e; jvm 21.0.2+13-LTS
2024-03-19 02:03:10.319:INFO :oejs.AbstractConnector:main: Started ServerConnector@49e53c76{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2024-03-19 02:03:10.365:INFO :oejs.Server:main: Started oejs.Server@50a638b5{STARTING}[12.0.7,sto=5000] @1908ms

Jetty 12 release notes: https://github.com/jetty/jetty.project/releases/tag/jetty-12.0.0

[mpeddada1]

According to https://eclipse.dev/jetty/documentation/jetty-12/operations-guide/index.html, in order to deploy the web app, we need: java -jar $JETTY_HOME/start.jar --add-modules=ee10-deploy. Setting this as the entrypoint worked locally.