GrpcStorageOptions with setAttemptDirectPath(true) doesn't work with native image

[BenWhitehead]

Error message

Sep 19, 2024 8:53:44 PM io.grpc.internal.ManagedChannelImpl$NameResolverListener handleErrorInSyncContext
WARNING: [Channel<3>: (google-c2p:///storage.googleapis.com)] Failed to resolve name. status=Status{code=UNAVAILABLE, description=Unable to load LDS xdstp://traffic-director-c2p.xds.googleapis.com/envoy.config.listener.v3.Listener/storage.googleapis.com. xDS server returned: CANCELLED: Failed to read message. nodeID: C2P-627175314, cause=java.lang.AssertionError: java.lang.RuntimeException: Failed to get default instance for class io.grpc.xds.shaded.io.envoyproxy.envoy.config.listener.v3.Listener
	at io.grpc.xds.client.XdsClientImpl$1.uncaughtException(XdsClientImpl.java:78)
	at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:96)
	at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:126)
	at io.grpc.xds.client.ControlPlaneClient$AdsStream.onRecvMessage(ControlPlaneClient.java:334)
	at io.grpc.xds.client.ControlPlaneClient$AdsStream.onRecvMessage(ControlPlaneClient.java:267)
	at io.grpc.xds.GrpcXdsTransportFactory$EventHandlerToCallListenerAdapter.onMessage(GrpcXdsTransportFactory.java:144)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInternal(ClientCallImpl.java:657)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:644)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base@17.0.8/java.lang.Thread.run(Thread.java:833)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203)
Caused by: java.lang.RuntimeException: Failed to get default instance for class io.grpc.xds.shaded.io.envoyproxy.envoy.config.listener.v3.Listener
	at com.google.protobuf.Internal.getDefaultInstance(Internal.java:355)
	at com.google.protobuf.Any.is(Any.java:85)
	at com.google.protobuf.Any.unpack(Any.java:109)
	at io.grpc.xds.client.XdsResourceType.unpackCompatibleType(XdsResourceType.java:214)
	at io.grpc.xds.client.XdsResourceType.parse(XdsResourceType.java:155)
	at io.grpc.xds.client.XdsClientImpl.handleResourceUpdate(XdsClientImpl.java:443)
	at io.grpc.xds.client.XdsClientImpl.handleResourceResponse(XdsClientImpl.java:138)
	at io.grpc.xds.client.ControlPlaneClient$AdsStream.handleRpcResponse(ControlPlaneClient.java:379)
	at io.grpc.xds.client.ControlPlaneClient$AdsStream$1.run(ControlPlaneClient.java:352)
	at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:94)
	... 13 more
Caused by: java.lang.NoSuchMethodException: io.grpc.xds.shaded.io.envoyproxy.envoy.config.listener.v3.Listener.getDefaultInstance()
	at java.base@17.0.8/java.lang.Class.getMethod(DynamicHub.java:2227)
	at com.google.protobuf.Internal.getDefaultInstance(Internal.java:352)
	... 22 more
}

[BenWhitehead]

Attached is the output of the graalvm tracing agent included in a java program that downloads a 128MiB object 10 times.

Collected from the following jvm

> java -version
java version "17.0.12" 2024-07-16 LTS
Java(TM) SE Runtime Environment Oracle GraalVM 17.0.12+8.1 (build 17.0.12+8-LTS-jvmci-23.0-b41)
Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 17.0.12+8.1 (build 17.0.12+8-LTS-jvmci-23.0-b41, mixed mode, sharing)

native-image-agent-config-output.tar.gz

[BenWhitehead]

Failures all appear to be related to protobuf generated classes.

Our own code generation, generates the necessary reflect-config.json https://github.com/googleapis/java-storage/blob/v2.43.0/gapic-google-cloud-storage-v2/src/main/resources/META-INF/native-image/com.google.storage.v2/reflect-config.json

Quick and dirty program to generate reflect-config entries for all of grpc:

    // uses io.github.classgraph:classgraph:4.8.176
    try (ScanResult scanResult =
        new ClassGraph()
            .enableAllInfo()
            .acceptPackages("io.grpc")
            .scan()) {
      String json = Stream.of(
              scanResult.getSubclasses(GeneratedMessageV3.class).stream(),
              scanResult.getSubclasses(AbstractMessage.Builder.class).stream(),
              scanResult.getAllEnums()
                  .filter(ci -> ci.implementsInterface(ProtocolMessageEnum.class))
                  .stream()
          )
          .flatMap(s -> s)
          .map(ClassInfo::getName)
          .sorted()
          .map(name -> String.format("""
                {
                  "name": "%s",
                  "queryAllDeclaredConstructors": true,
                  "queryAllPublicConstructors": true,
                  "queryAllDeclaredMethods": true,
                  "allPublicMethods": true,
                  "allDeclaredClasses": true,
                  "allPublicClasses": true
                }""", name))
          .collect(Collectors.joining(",\n"));
      System.out.println(json);
    }