Protos from java-library project are not extracted for downstream dependencies

[radimk]

Gradle 6.0.1, Java 8, protobuf-plugin 0.8.10

I have a protobuf a.proto defined in project :lib and second protobuf b.proto in project :app that has dependency compile project(':lib'). My second protobuf imports a.proto and relies on a.proto being extracted into app/build/extracted-include-protos/main/a.proto by extractIncludeProto task. This works :lib project uses 'java' plugin but it doesn't if it uses 'java-library' plugin.

This can be a duplicate of issue #338 but the important part about java-library being used for upstream dependency is not mentioned there.

The problem is related to how Gradle resolves compileClasspath of lib project. When it is defined like

plugins {
  id "com.google.protobuf" version "0.8.10"
  id "java"
}

repositories {
    jcenter()
}

protobuf {
  protoc {
    artifact = 'com.google.protobuf:protoc:3.8.0'
  }
}

dependencies {
  compile 'com.google.protobuf:protobuf-java:3.8.0'
}

The entry from compileClasspath will be resolved as JAR:
2019-12-05T23:41:58.416+0100 [DEBUG] [org.gradle.api.Task] Extracting protos from /home/rkubacki/tmp/protobuf_bug/lib/build/libs/lib.jar to /home/rkubacki/tmp/protobuf_bug/app/build/extracted-include-protos/main
When 'java-library' and api configuration is used:

plugins {
  id "com.google.protobuf" version "0.8.10"
  id "java-library"
}

repositories {
    jcenter()
}

protobuf {
  protoc {
    artifact = 'com.google.protobuf:protoc:3.8.0'
  }
}

dependencies {
  api 'com.google.protobuf:protobuf-java:3.8.0'
}

it will get output folder containg classes. That's why a.proto will not be found and copied and thus it is missing when generateProto runs.

2019-12-05T23:38:56.391+0100 [DEBUG] [org.gradle.api.Task] Extracting protos from /home/rkubacki/tmp/protobuf_bug/lib/build/classes/java/main to /home/rkubacki/tmp/protobuf_bug/app/build/extracted-include-protos/main

[ejona86]

After a very involved session of figuring out how the newfangled world is supposed to work, we figured out how the plugin is supposed to get the input files:

configurations {
  compileProto {
    extendsFrom implementation, compileOnly
    getAttributes().attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, getObjects().named(LibraryElements.class, Boolean.getBoolean(JavaBasePlugin.COMPILE_CLASSPATH_PACKAGING_SYSTEM_PROPERTY) ? LibraryElements.JAR : LibraryElements.RESOURCES));
  }
}

println(files(configurations["compileProto"]).getFiles())

The plugin will need to create a configuration like that and then use it in ProtobufPlugin when configuration inputs.

This is mirroring what is done with compileClasspath. We would need something similar to mirror testCompileClasspath as well.

That also allows us to use the output folders directly instead of requiring the output jar. Although we may need some other code changes to handle folders (doesn't seem like it, but maybe).

[radimk]

Awesome! This sounds really promising, thanks for that hard work to find this in the guts of Java plugin :-)

[ejona86]

@voidzcy, thinking more about the system property, I think we should be fine to drop it and just use RESOURCES unconditionally. The system property is there because of a performance problem with javac but we need folders anyway so I don't think it really does much for us to use JAR.

[voidzcy]

@radimk Are you sure the issue occurs when using compile project(':lib') in :app? I do not get the error when using compile project(':lib'), but I do reproduce the import not found error when using implementation project(':lib').

Also, this issue is likely to be the same one as #242 (people observed error for implementation dependency #242 (comment)).

[radimk]

@voidzcy you're right. compile project(':lib') builds but implementation project(':lib') doesn't (same for api). Then it seems the problem happens when the library uses new java-library plugin and both sides the new configurations instead of deprecated compile.

[voidzcy]

Please try out 0.8.11 release, we did some work regarding this issue. Please let us know if there is anything else related to this issue.

[radimk]

Works now. Thank you.