Application deployed to k8s won't start with dependency on micronaut-tracing-opentelemetry-jdbc

[miguelaferreira]

Expected Behavior

My expectation was that the application deployed to a k8s cluster would start just like it does locally. Producing this output:

   __  __ _                                  _   
  |  \/  (_) ___ _ __ ___  _ __   __ _ _   _| |_
  | |\/| | |/ __| '__/ _ \| '_ \ / _` | | | | __|
  | |  | | | (__| | | (_) | | | | (_| | |_| | |_
  |_|  |_|_|\___|_|  \___/|_| |_|\__,_|\__,_|\__|
  09:45:28.274 [main] INFO  i.m.c.DefaultApplicationContext$RuntimeConfiguredEnvironment - Established active environments: [k8s, cloud]
  09:45:29.427 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
  09:45:30.049 [main] INFO  com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection conn0: url=jdbc:h2:mem:devDb user=SA
  09:45:30.053 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed.
  09:45:30.961 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 4743ms. Server Running: http://jdbc-tracing:8080

Actual Behaviour

The application does not start and complaints about a missing bean.

  __  __ _                                  _   
  |  \/  (_) ___ _ __ ___  _ __   __ _ _   _| |_
  | |\/| | |/ __| '__/ _ \| '_ \ / _` | | | | __|
  | |  | | | (__| | | (_) | | | | (_| | |_| | |_
  |_|  |_|_|\___|_|  \___/|_| |_|\__,_|\__,_|\__|
  09:44:25.329 [main] INFO  i.m.c.DefaultApplicationContext$RuntimeConfiguredEnvironment - Established active environments: [k8s, cloud]
  09:44:25.519 [main] ERROR io.micronaut.runtime.Micronaut - Error starting Micronaut server: No bean of type [io.micronaut.context.event.ApplicationEventPublisher<io.micronaut.context.event.StartupEvent>] exists.
  io.micronaut.context.exceptions.NoSuchBeanException: No bean of type [io.micronaut.context.event.ApplicationEventPublisher<io.micronaut.context.event.StartupEvent>] exists.
  at io.micronaut.context.DefaultBeanContext.newNoSuchBeanException(DefaultBeanContext.java:2798)
  at io.micronaut.context.DefaultApplicationContext.newNoSuchBeanException(DefaultApplicationContext.java:329)
  at io.micronaut.context.DefaultBeanContext.resolveBeanRegistration(DefaultBeanContext.java:2761)
  at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1745)
  at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:842)
  at io.micronaut.context.BeanLocator.getBean(BeanLocator.java:97)
  at io.micronaut.context.DefaultBeanContext.publishEvent(DefaultBeanContext.java:1831)
  at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:360)
  at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:216)
  at io.micronaut.runtime.Micronaut.start(Micronaut.java:75)
  at io.micronaut.runtime.Micronaut.run(Micronaut.java:334)
  at io.micronaut.runtime.Micronaut.run(Micronaut.java:320)
  at jdbc.tracing.Application.main(Application.java:8)

Steps To Reproduce

Using the example repository I'm providing, do the following.

To verify that the application without the dependency starts, comment out the dependency (implementation('io.micronaut.tracing:micronaut-tracing-opentelemetry-jdbc:6.9.0')) in build.gradle, then:

• build a docker container with ./gradlew dockerBuild
• tag the container image according to the docker repository that you will push it to, keeping in mind that the repository should be accessible from the Kubernetes

  docker tag jdbc-tracing:latest <your-docker-repository>/jdbc-tracing:latest

• push the container image to the docker repository docker push <your-docker-repository>/jdbc-tracing:latest
• update the pod.yaml manifest with the new image name
• apply the manifest to the Kubernetes cluster kubectl apply -f pod.yaml
• check the logs of the pod kubectl logs <pod-name> where you should see that the application does start

To reproduce the issue, uncomment the dependency in build.gradle, then:

• build a docker container with ./gradlew dockerBuild
• tag the container image according to the docker repository that you will push it to, keeping in mind that the repository should be accessible from the Kubernetes

  docker tag jdbc-tracing:latest <your-docker-repository>/jdbc-tracing:latest-with-bug

• push the container image to the docker repository docker push <your-docker-repository>/jdbc-tracing:latest-with-bug
• update the pod.yaml manifest with the new image name
• apply the manifest to the Kubernetes cluster kubectl apply -f pod.yaml
• check the logs of the pod kubectl logs <pod-name> where you should see that the application does not start

Environment Information

• Operating system: default docker container image produced with ./gradlew dockerBuild
• JDK version: default JDK bundled in docker container image produced with ./gradlew dockerBuild

Example Application

https://github.com/miguelaferreira/micronaut-jdbc-tracing

Version

4.6.3

[graemerocher]

sounds unrelated to this module. The error indicates the micronaut-context module is missing which is likely a packaging issue

[miguelaferreira]

Thanks for the quick reply @graemerocher.

I can confirm that the micronaut-context classes are being packaged in the docker container. I've copied all the files under /home/app in the docker container to my localhost, and then was able to list all the libs and micronaut-context is there.

The files I copied from the container

$ ls
application.jar
libs
resources

The micronaut-context lib is there

$ ls libs | grep micronaut-context
micronaut-context-4.6.6.jar
micronaut-context-propagation-4.6.6.jar

The bean that the application complains is missing is in the libs/micronaut-inject-4.6.6.jar

$ jar -tf libs/micronaut-inject-4.6.6.jar | grep ApplicationEventPublisher
io/micronaut/context/event/ApplicationEventPublisher.class
io/micronaut/context/event/ApplicationEventPublisherFactory$1.class
io/micronaut/context/event/ApplicationEventPublisherFactory$2.class
io/micronaut/context/event/ApplicationEventPublisherFactory$EventLogger.class
io/micronaut/context/event/ApplicationEventPublisherFactory.class
io/micronaut/context/event/NoOpApplicationEventPublisher.class
META-INF/micronaut/io.micronaut.inject.BeanDefinitionReference/io.micronaut.context.event.ApplicationEventPublisherFactory

I've verified that these files are present on both the application version that works (without the dependency) and in the one that does not work.

If you feel like I should move this issue somewhere else, please let me know.

[graemerocher]

probably you need to upgrade to micronaut core 4.7

[miguelaferreira]

@graemerocher I've tried to upgrade micronaut-core to 4.7.2 and that did not help.

    implementation("io.micronaut:micronaut-core:4.7.2")
    implementation("io.micronaut:micronaut-core-processor:4.7.2")
    implementation("io.micronaut:micronaut-context:4.7.2")
    implementation("io.micronaut:micronaut-context-propagation:4.7.2")
    implementation("io.micronaut:micronaut-discovery-core:4.7.2")
    implementation("io.micronaut:micronaut-http:4.7.2")
    implementation("io.micronaut:micronaut-http-client:4.7.2")
    implementation("io.micronaut:micronaut-http-client-core:4.7.2")
    implementation("io.micronaut:micronaut-http-netty:4.7.2")
    implementation("io.micronaut:micronaut-http-server:4.7.2")
    implementation("io.micronaut:micronaut-http-server-netty:4.7.2")
    implementation("io.micronaut:micronaut-inject:4.7.2")

I've pushed a commit to my example repo with that ☝️ patch.