New hamcrest 1.3 bundles don't resolve for CDT

[jonahgraham]

I have made a minimal(ish) test case from CDT's target platform here: https://github.com/jonahgraham/orbit-junit-issue

The head commit which uses the orbit-aggregation/nightly fails with this error:

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:3.0.5:validate-classpath (default-validate-classpath) on project org.eclipse.cdt.util: Execution default-validate-classpath of goal org.eclipse.tycho:tycho-compiler-plugin:3.0.5:validate-classpath failed: org.osgi.framework.BundleException: Bundle org.eclipse.cdt.util cannot be resolved:org.eclipse.cdt.util [100]
[ERROR] Unresolved requirement: Require-Bundle: org.junit
[ERROR] -> Bundle-SymbolicName: org.junit; bundle-version="4.13.2.v20230725-0701"
[ERROR] org.junit [21]
[ERROR] Unresolved requirement: Import-Package: org.hamcrest; version="1.3.0"

The parent commit builds fine with the only difference I can see being the version of hamcrest 1.3 I am using.

[merks]

Oh my goodness, what a tangle web we've woven. 😱

The ebr org.junit has this:

Require-Bundle                          org.hamcrest.core;bundle-version="1.3";visibility:=reexport

But we want to allow and support org.hamcrest 2.2 going forward. So clever me has taken several tries at this, with the most recent one like this:

Import-Package                          org.hamcrest.core;version="1.3"
                                        org.hamcrest;version="1.3"
Require-Bundle                          org.hamcrest.core;bundle-version="1.3.0";resolution:=optional;visibility:=reexport;x-installation:=greedy

And that worked for the Platform's build, with the version of org.hamcrest.core 1.3 at that time. But then, I wanted to add org.hamcrest.library 1.3 and noticed the split packages issue. I asked @HannesWell about that here:

#5

So then org.hamcrest.core was modified so that it exports the split package org.hamcrest package like this:

 org.hamcrest;core=split;version="1.3.0";mandatory:=core;uses:="org.hamcrest.core,org.hamcrest.internal"

I.e., with a mandatory attribute like it had in the ebr version. But now the org.junit's package import cannot be satisfied by the package exported from org.hamcrest.core 1.3. But if I add the mandatory attribute to the package import, then org.hamcrest's package export of org.hamcrest;version="[2.2,3)" would not match.

Then there are all the strange inconsistencies. PDE sets up the classpath as expected and happily compiles, but it does not happily run as an OSGi application:

Of course it cannot, but it would be good to know that earlier.

Adding this import adds org.hamcrest to the dependencies and then it runs:

But Tycho is not happy:

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:3.0.5:validate-classpath (default-validate-classpath) on project org.eclipse.cdt.util: Execution default-validate-classpath of goal org.eclipse.tycho:tycho-compiler-plugin:3.0.5:validate-classpath failed: org.osgi.framework.BundleException: Bundle org.eclipse.cdt.util cannot be resolved:org.eclipse.cdt.util [100]
[ERROR]   Unresolved requirement: Import-Package: org.hamcrest; version="1.3.0"

Tycho is happy when we change it to 2.0.0:

But in the IDE, F3 navigation from the package import to the package in the bundle selects the wrong bundle as above.

I'm really left scratching my head if it's even possible to provide versions of org.junit, org.hamcrest.core, and org.hamcrest.library that are drop-in replacements for the previous versions and support migration to org.hamcrest 2.2? Multiple package imports of org.hamcrest both of them optional? Surely we don't want a bundle require of the newer org.hamcrest bundle. In any case, the package import in the latest org.junit is not compatible with the package export of the latest org.hamcrest.core, so something needs to be fixed...

[merks]

I tried changing the org.hamcrest.core bundle so that it still has the core=split attribute, so that existing package imports that specify this attribute will still work, but with the mandatory directive is omitted:

org.hamcrest;core=split;version="1.3.0";uses:="org.hamcrest.core,org.hamcrest.internal"

The IDE is happy and Tycho compiles too, The application runs as well, as long as org.hamcrest 2.2 is not in the launch configuration, because then it gets a class not found exception.

One can also add a versionless package import:

The IDE and Tycho are still happy and the application works with either org.hamcrest.core 1.3.0 or with org.hamcrest 2.2.0.

A versioned import Import-Package: org.hamcrest;version="2.2.0" works too, but then of course the application on works with org.hamcrest 2.0.0 in the launch.

So this does look like a path forward...

[merks]

@nitind

Maybe/probably you were bitten by this?

[nitind]

@nitind

Maybe/probably you were bitten by this?

Were and still am. Does this mean I now have to import hamcrest into all of my test bundles?

[merks]

@nitind
Maybe/probably you were bitten by this?

Were and still am. Does this mean I now have to import hamcrest into all of my test bundles?

No. I need to build a new version of org.hamcrest.core. If @HannesWell or @jonahgraham don’t have a better idea or a concern then I will do that tomorrow.

[HannesWell]

Oh my goodness, what a tangle web we've woven. 😱

Indeed... I didn't consider this case too :/

I tried changing the org.hamcrest.core bundle so that it still has the core=split attribute, so that existing package imports that specify this attribute will still work, but with the mandatory directive is omitted:

That looks like the best solution to me as well, every case should work with that.👍🏽
The only drawback is that with 1.3 someone might forget the attributes, but it is better to migrate to 2.2 anyways.

Btw. there is also an empty org.hamcrest.core artifact (without an OSGi Manifest) that could be used to redirect to org.hamcrest.

[merks]

The problem should be fixed in the latest nightly build.

[jonahgraham]

Thanks Ed - my minimal test case works and I have now started PR eclipse-cdt/cdt#495 to check all of CDT is ok with the latest.

[nitind]

The problem should be fixed in the latest nightly build.

My builds are all back to normal with this.

[merks]

@nitind

That's great!

I've started to have a look at WTP's bundles and features today. Oh my goodness but there is a lot of stuff here, and I don't even understand how the target platform is specified yet...

[jonahgraham]

My builds are not back to normal - the knock on effect of some of the other changes means I haven't been able to verify fully (sorry for the delay, long weekend in Canada).

I do see errors when running in Eclipse with use constraint violations.

!ENTRY org.eclipse.cdt.autotools.core 4 0 2023-08-08 17:10:53.479
!MESSAGE FrameworkEvent ERROR
!STACK 0
org.osgi.framework.BundleException: Could not resolve module: org.eclipse.cdt.autotools.core [96]
  Bundle was not resolved because of a uses constraint violation.
  org.apache.felix.resolver.reason.ReasonException: Uses constraint violation. Unable to resolve resource org.eclipse.cdt.core.tests [osgi.identity; osgi.identity="org.eclipse.cdt.core.tests"; type="osgi.bundle"; version:Version="5.4.100.qualifier"; singleton:="true"] because it is exposed to package 'org.hamcrest' from resources org.hamcrest.core [osgi.identity; osgi.identity="org.hamcrest.core"; type="osgi.bundle"; version:Version="1.3.0.v20230806-0849"] and org.hamcrest [osgi.identity; osgi.identity="org.hamcrest"; type="osgi.bundle"; version:Version="2.2.0"] via two dependency chains.

Chain 1:
  org.eclipse.cdt.core.tests [osgi.identity; osgi.identity="org.eclipse.cdt.core.tests"; type="osgi.bundle"; version:Version="5.4.100.qualifier"; singleton:="true"]
    require: (&(osgi.wiring.bundle=org.hamcrest.core)(&(bundle-version>=1.3.0)(!(bundle-version>=2.0.0))))
     |
    provide: osgi.wiring.bundle: org.hamcrest.core
  org.hamcrest.core [osgi.identity; osgi.identity="org.hamcrest.core"; type="osgi.bundle"; version:Version="1.3.0.v20230806-0849"]

Chain 2:
  org.eclipse.cdt.core.tests [osgi.identity; osgi.identity="org.eclipse.cdt.core.tests"; type="osgi.bundle"; version:Version="5.4.100.qualifier"; singleton:="true"]
    require: (osgi.wiring.bundle=org.junit)
     |
    provide: osgi.wiring.bundle; bundle-version:Version="4.13.2.v20230725-0701"; osgi.wiring.bundle="org.junit"
  org.junit [osgi.identity; osgi.identity="org.junit"; type="osgi.bundle"; version:Version="4.13.2.v20230725-0701"]
    import: (&(osgi.wiring.package=org.hamcrest)(version>=1.3.0))
     |
    export: osgi.wiring.package: org.hamcrest
  org.hamcrest [osgi.identity; osgi.identity="org.hamcrest"; type="osgi.bundle"; version:Version="2.2.0"]

I can fix/workaround this in CDT by changing hamcrest dependency to Package-Import, but I see similar errors in my dependencies that I cannot resolve. e.g. swtbot (swtbot.go and swt.finder)

This isn't surprising as for a long time having a Require-Bundle for hamcrest was the way to get it to resolve. The swt.finder link above is a good example of this, everything is import package, except hamcrest.

[merks]

FYI, this looks like the same issue as reported here:

eclipse-platform/eclipse.platform.releng.aggregator#1245