KSQL on 6.0.x branch fails to start if OpenSSL is not installed on the host.

[JumaX]

Describe the bug
A clear and concise description of what the bug is.

Problem:

If you install Confluent Platform 6.0 on any supported Linux distribution which does not have OpenSSL installed, it will fail to start.

To Reproduce
Steps to reproduce the behavior, include:

Install any package from the 6.0.x nightly build on any supported Linux platform without OpenSSL installed. Configure KSQL and start it.

Expected behavior
A clear and concise description of what you expected to happen.

I would expect KSQL to list out OpenSSL as a hard requirement. I would also expect to fail with a much more graceful error indicating that OpenSSL is missing.

Actual behaviour
A clear and concise description of what actually happens, including:

1. CLI output
2. Error messages
3. KSQL logs

Error:

2020-07-08 13:13:18,809 WARN Failed to initialize a channel. Closing: id: 0xa9069359, L:/172.28.0.9:8088 - R:/172.28.0.9:47238 (io.netty.channel.ChannelInitializer:97)
java.lang.NoClassDefFoundError: org/eclipse/jetty/alpn/ALPN$Provider
at io.netty.handler.ssl.JettyAlpnSslEngine.newServerEngine(JettyAlpnSslEngine.java:60)
at io.netty.handler.ssl.JdkAlpnApplicationProtocolNegotiator$AlpnWrapper.wrapSslEngine(JdkAlpnApplicationProtocolNegotiator.java:141)
at io.netty.handler.ssl.JdkSslContext.configureAndWrapEngine(JdkSslContext.java:360)
at io.netty.handler.ssl.JdkSslContext.newEngine(JdkSslContext.java:330)
at io.vertx.core.net.impl.SSLHelper.createEngine(SSLHelper.java:538)
at io.vertx.core.http.impl.HttpServerChannelInitializer.initChannel(HttpServerChannelInitializer.java:105)
at io.vertx.core.http.impl.HttpServerImpl$1.initChannel(HttpServerImpl.java:215)
at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129)
at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112)
at io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:964)
at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:613)
at io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:46)
at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1475)
at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1127)
at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:654)
at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:503)
at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:416)
at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:475)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.alpn.ALPN$Provider
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 25 more

Additional context

It would appear that we expect OpenSSL to be installed by default. However if it is not, KSQL will fail. We also do not list out OpenSSL as a hard requirement in our documentation.

It's possible for KSQL to use native JDK libs for this functionality as well, just with lower performance.

The proposed solution is to document that OpenSSL is recommended, however if it is not installed, we will detect this, and fall back to the JDK libs.

[purplefox]

Netty default behaviour https://netty.io/wiki/requirements-for-4.x.html for choosing an SSL engine is to first look for openSSL, and if not there to fall back to using the JDKs built in SSL engine. openSSL is recommended for better performance.

However, with JDK < 1.8.0_252 OpenJDK did not contain an ALPN implementation (this is needed for TLS with HTTP2) so, for ALPN the Jetty ALPN agent can be used to provide one.

Currently ksqlDB is using Vert.x 3.9.0 which depends on an earlier version of Netty which does not have the code that drops the ALPN check.

Possible solutions to this issue are:

1. Upgrade ksqlDB to use Vert.x 3.9.1 - this will make the reported issue go away as long as the user is using OpenJDK.1.8.0_252 or later. I think it's reasonable to expect that users are using the latest version of 1.8.0. I'd also recommend that we log a warning on startup if openSSL is not installed so the user is aware of this and can install it for better performance.
2. Add the Jetty ALPN agent to the classpath as described here https://github.com/jetty-project/jetty-alpn-agent - this requires us to change the bootclasspath for the ksqlDB and to ship the Jetty ALPN libraries.
3. Require users to install OpenSSL. This could be reasonable too. Most Linux distros come with OpenSSL installed already. It's unlikely a user would not want to use OpenSSL (unless they don't care about performance).

I'd recommend we go with 1 for now, and if we see a lot of customer issues we can look at 2.

[vcrfxia]

Hey @purplefox , assigning this to you for now since you mentioned taking a stab at it sometime next week. If you're swamped I can pick it up as well -- let me know.