Gradle build: Upgrade protobuf Gradle plugin (for Gradle 8, Java 20)

[msgilligan]

• Upgrade protobuf-javalite to version 3.22.3
• Upgrade Gradle protobuf plugin to version 0.9.4 (if Gradle 7 or later)
• Remove generated ProtoBuf files, they're now produced in build/generated/...

[schildbach]

Can we keep the version upgrade separate from the rest?

There were good reasons against generating the protobufs on the fly, if I only could remember.

[schildbach]

Related: 002fe81 and 6e85ea3

[msgilligan]

Can we keep the version upgrade separate from the rest?

I think so. Let's merge #3058 and then I'll try breaking this one down to the minimum functionality to get Gradle 8 working.

There were good reasons against generating the protobufs on the fly, if I only could remember.

Something seemed to break when I upgraded either protobuf itself or the plugin (I started getting duplicate Proto.java files) -- so I just deleted the build.gradle line that set the directory and that seemed to resolve the issue. I can investigate further...

[msgilligan]

There were good reasons against generating the protobufs on the fly, if I only could remember.

6e85ea3 says "too much support hassle." Does that ring a bell?

[schildbach]

I picked one of the version upgrades.

[schildbach]

I've updated the plugin to 0.9.1. 0.9.2 seems to have some regression regarding generatedFilesBaseDir:

/home/aschildbach/dev/workspace/bitcoinj/core/build/generated/source/proto/main/java/org/bitcoin/protocols/payments/Protos.java:6: error: duplicate class: org.bitcoin.protocols.payments.Protos
public final class Protos {
             ^
/home/aschildbach/dev/workspace/bitcoinj/core/build/generated/source/proto/main/java/org/bitcoinj/wallet/Protos.java:6: error: duplicate class: org.bitcoinj.wallet.Protos
public final class Protos {
             ^

[msgilligan]

Can we keep the version upgrade separate from the rest?

Apparently not. It looks like the Gradle Plug-in has dropped support for specifying the (complete) output directory. (and without the courtesy of giving an error when the removed option is used)

[msgilligan]

I've updated the plugin to 0.9.1. 0.9.2 seems to have some regression regarding generatedFilesBaseDir:

/home/aschildbach/dev/workspace/bitcoinj/core/build/generated/source/proto/main/java/org/bitcoin/protocols/payments/Protos.java:6: error: duplicate class: org.bitcoin.protocols.payments.Protos

public final class Protos {

             ^

/home/aschildbach/dev/workspace/bitcoinj/core/build/generated/source/proto/main/java/org/bitcoinj/wallet/Protos.java:6: error: duplicate class: org.bitcoinj.wallet.Protos

public final class Protos {

             ^

See google/protobuf-gradle-plugin#636

[schildbach]

So to clarify: there is no way with 0.8.2 to keep the existing behaviour to generate the protobuf files within src/?

If that is true, is there a way to use a new special directory, which is not part of src/ but not within build/ either? Somewhere where it can remain under version control.

And also, what are the downsides of staying with 0.8.1 for as long as this issue is resolved by the plugin (which I assume should be resolved; our configuration used to be very common years ago)?

[msgilligan]

So to clarify: there is no way with 0.8.2 to keep the existing behaviour to generate the protobuf files within src/?

I don't know if there is no way. I have very little knowledge of protobuf and the protobuf Gradle plugin. I only tinker with it when I'm trying to upgrade Gradle. I also assume you mean 0.9.2 above, right?

[schildbach]

I also assume you mean 0.9.2 above, right?

Yes, sorry.

[msgilligan]

Rebased and force-pushed. (This still combines upgrading the plugin, removing the generated files from Git, and removing the deprecated generatedFilesBaseDir property, but now reflects that we've upgraded the protobuf library itself.)

[ejona86]

So to clarify: there is no way with 0.8.2 to keep the existing behaviour to generate the protobuf files within src/?

In 0.9.2 the plugin will always use the default directory. But if you specify generatedFilesBaseDir, then it will copy the generated files into that directory afterward. generatedFilesBaseDir is deprecated because it doesn't handle deletes properly, because it can't distinguish between "old generated code" and "your artisan source files."

[msgilligan]

...is there a way to use a new special directory, which is not part of src/ but not within build/ either? Somewhere where it can remain under version control.

I think we could make it a "source set" in Gradle and put it in core/src/protobufGenerated or something similar. The first step in doing that would be to move the generated files to a dedicated package (or packages) -- see PR #3101.

[msgilligan]

@schildbach I have rebased and force-pushed.

I think we should merge #3101 to move the generated Java classes to org.bitcoinj.protobuf (and not create a new source set) and then rebase and merge this PR.

Given that you don't remember the "good reason" for checking in the generated source, and that I recall you saying elsewhere that you "weren't opposed" to removing it, I think we should go ahead and remove it. If we find a problem we can revert the change or try a different approach (maybe a source-set)

[msgilligan]

@ejona86 I forgot to thank you for your comment until now. Thank you!

[msgilligan]

@schildbach OK, this is rebased to upgrade to the latest Protobuf plugin and to remove the deprecated generateFilesBaseDir setting.

[schildbach]

Merged.