Handle non-jar / metadata-only artifacts

[sschuberth]

In regards to com.github.ajalt.clikt:clikt, this is because we don't handle non-jar dependencies yet. This dependency is metadata only, so an extension is applied during resolution. When we create our dependency query, we are not applying the classifier and extension. This causes a failure because Gradle is looking for a jar and fails.

I did find where to extract this information in #224 but never got around to adding it into the query. It's probably very simple if you'd like to give it a shot. It should just take a little formatting magic in the query string.

Originally posted by @ben-manes in #334 (comment)

[serebit]

0.31.0 still fails to detect one dependency in one of my projects, and that's com.soywiz.korlibs.klock:klock. Resolving this issue should fix that.

[ben-manes]

@sschuberth your issue is due to JCenter which behaves oddly because it only half proxies queries. If you add Maven Central then it works fine. While it is only a metadata dependency, the declaration doesn't have an extension or classifier in the dependency declaration which is what we don't support (e.g. a:b:1.0@zip) yet.

@serebit Your issue appears to be due to needing the dependency capabilities copied over. That dependency has a platform capability (e.g. JVM) to select from. I would need a sample project to test against. The api implies that we could provide the source's getRequestedCapabilities() via the target's capabilities​(configureAction). Please provide a minimal failing test build that I can verify a fix against, and open a new issue with those details.

[sschuberth]

@sschuberth your issue is due to JCenter which behaves oddly because it only half proxies queries.

Would have have some more details on that, so we could report this to JCenter / JFrog?

[ben-manes]

The error (--info) says that the metadata cannot be fetched, which a cURL verifies.

 - com.github.ajalt.clikt:clikt
The exception that is the cause of unresolved state: org.gradle.internal.resolve.ModuleVersionNotFoundException: Could not find any matches for com.github.ajalt.clikt:clikt:+ as no versions of com.github.ajalt.clikt:clikt are available.
Searched in the following locations:
  - https://jcenter.bintray.com/com/github/ajalt/clikt/clikt/maven-metadata.xml

JCenter proxies explicit versions to Central, but not metadata queries, and can provide stale results from its cache (e.g. + resolves only on jcenter). Since that dependency is on Central only, they don't perform the full proxying. If you add Central then Gradle will search it too, and you'll see an update.

[sschuberth]

Thanks,

BTW, with version 0.32.0 I'm now getting something strange:

Failed to determine the latest version for the following dependencies (use --info for details):
 - com.github.ajalt.clikt:clikt
 - oss-review-toolkit:test-utils

Where test-utils is a Gradle project that is part of the same root project, so why is it checked for updates?

[ben-manes]

I'm not really sure, to be honest. We iterate over all projects to resolve their external dependencies by copying their configuration. It seems the copying confused it, but not sure what the expectation would be either way. The error is,

The exception that is the cause of unresolved state: org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve project :test-utils.
Required by:
    project :test-utils
Caused by: org.gradle.internal.component.model.ConfigurationNotFoundException: Project :test-utils declares a dependency from configuration 'funTestImplementationCopy' to configuration 'default' which is not declared in the descriptor for project :test-utils.

[sschuberth]

JCenter proxies explicit versions to Central, but not metadata queries, and can provide stale results from its cache (e.g. + resolves only on jcenter). Since that dependency is on Central only, they don't perform the full proxying.

I've hijacked this somewhat related ticket, let's see if there's a reaction from JFrog.

[ben-manes]

Oh, this is probably a side-effect of including the "super configuration" dependencies for #334. The funTestImplementation configuration extends testImplementation which has a dependency on that project. Some interaction that I don't understand results in that error.

[sschuberth]

Sounds a bit as if the funTestImplementationCopy configuration is not variant-aware anymore...

[ben-manes]

I bet it's because the when we resolve the latest, we filter down to the ExternalDependency so it excludes projects, etc. When we add back in the super configuration dependencies, we blindly take all, so it includes the project dependency. If we filter it out then there is a good chance this will go away.

[sschuberth]

Want me to create a new issue for that? 😉

[ben-manes]

haha, sure. I just verified it fixed your project and double checking it doesn't regress #334. 😃

[sschuberth]

Wow, thanks! If you already have the fix, I guess we can spare the issue creation, right?

[ben-manes]

either way is fine.

When checking against https://github.com/zacsweers/catchup there seems to be a regression in v32 where the super configuration causes some of its dependencies to not resolve, e.g.

Caused by: org.gradle.internal.component.NoMatchingConfigurationSelectionException: No matching variant of ru.ldralighieri.corbind:corbind-material:1.3.2 was found. The consumer was configured to find an API of a library, with the library elements 'aar', and its dependencies declared externally, as well as attribute 'com.android.build.api.attributes.BuildTypeAttr' with value 'release' but:
  - Variant 'compile' capability ru.ldralighieri.corbind:corbind-material:1.3.2 declares an API of a library:
      - Incompatible because this component declares a component, packaged as a jar and the consumer needed a component, with the library elements 'aar'
      - Other compatible attributes:
          - Doesn't say anything about com.android.build.api.attributes.BuildTypeAttr (required 'release')
          - Doesn't say anything about how its dependencies are found (required its dependencies declared externally)

So kind of stuck as it seems either we fix for #423 or revert it, unless there's more metadata (like capabilities) we need to copy over?

[ben-manes]

I think unwrangling this is going to be hard, as either approach results in some variant failure. There might be a partial fix for #423 by special casing Kotlin's stdlib and not bringing everything else across. A new ticket is probably worthwhile since its late again here.

[ben-manes]

Okay, that did it. So I'll just keep any super configuration's external dependency that belongs to the group org.jetbrains.kotlin and both of your projects resolve. Hopefully that's good enough for me to be inactive for another year.
😛

[ben-manes]

okay, v33 is being released 🤞

[serebit]

Sadly, v33 doesn't fix com.soywiz.korlibs.klock:klock. I'll see to creating a small reproduction build.

[ben-manes]

@serebit Please try 0.34, which has an addition fix for Kotlin MPP (special case copy of their legacy metadata).