Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incompatible woodstox-core and stax2-api dependencies #340

Closed
wilkinsona opened this issue May 21, 2019 · 18 comments
Closed

Incompatible woodstox-core and stax2-api dependencies #340

wilkinsona opened this issue May 21, 2019 · 18 comments
Milestone

Comments

@wilkinsona
Copy link

wilkinsona commented May 21, 2019

2.9.9 upgraded to woodstox-core 5.1 which depends on stax2-api 4.1 but it still depends directly on stax2-api 3.1.4. As a result, when using Maven, a dependency on jackson-dataformat-xml results in an incompatible combination of versions of woodstox-core and stax2-api and a NoSuchMethodError results when also using Hibernate:

…
Caused by: java.lang.NoSuchMethodError: org.codehaus.stax2.ri.SingletonIterator.create(Ljava/lang/Object;)Lorg/codehaus/stax2/ri/SingletonIterator;
	at com.ctc.wstx.util.DataUtil.singletonIterator(DataUtil.java:41)
	at com.ctc.wstx.evt.CompactStartElement.getAttributes(CompactStartElement.java:100)
	at org.hibernate.boot.jaxb.internal.stax.JpaOrmXmlEventReader.existingXmlAttributesIterator(JpaOrmXmlEventReader.java:112)
	at org.hibernate.boot.jaxb.internal.stax.JpaOrmXmlEventReader.mapAttributes(JpaOrmXmlEventReader.java:100)
	at org.hibernate.boot.jaxb.internal.stax.JpaOrmXmlEventReader.wrap(JpaOrmXmlEventReader.java:84)
	at org.hibernate.boot.jaxb.internal.stax.JpaOrmXmlEventReader.wrap(JpaOrmXmlEventReader.java:74)
	at org.hibernate.boot.jaxb.internal.stax.JpaOrmXmlEventReader.peek(JpaOrmXmlEventReader.java:63)
	at org.dom4j.io.STAXEventReader.readElement(STAXEventReader.java:315)
	at org.dom4j.io.STAXEventReader.readNode(STAXEventReader.java:198)
	at org.hibernate.boot.jaxb.internal.MappingBinder$1.readNode(MappingBinder.java:104)
	at org.dom4j.io.STAXEventReader.readDocument(STAXEventReader.java:275)
	at org.hibernate.boot.jaxb.internal.MappingBinder.toDom4jDocument(MappingBinder.java:108)
	at org.hibernate.boot.jaxb.internal.MappingBinder.doBind(MappingBinder.java:71)
	at org.hibernate.boot.jaxb.internal.AbstractBinder.doBind(AbstractBinder.java:102)
	at org.hibernate.boot.jaxb.internal.AbstractBinder.bind(AbstractBinder.java:57)
	at org.hibernate.boot.jaxb.internal.InputStreamXmlSource.doBind(InputStreamXmlSource.java:43)
	at org.hibernate.boot.jaxb.internal.InputStreamXmlSource.doBind(InputStreamXmlSource.java:38)
	at org.hibernate.boot.spi.XmlMappingBinderAccess.bind(XmlMappingBinderAccess.java:79)
	at org.hibernate.boot.model.process.internal.ScanningCoordinator.applyScanResultsToManagedResources(ScanningCoordinator.java:220)
	at org.hibernate.boot.model.process.internal.ScanningCoordinator.coordinateScan(ScanningCoordinator.java:82)
	at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.prepare(MetadataBuildingProcess.java:98)
	at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:234)
	at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.<init>(EntityManagerFactoryBuilderImpl.java:167)
	at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:51)
	at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:365)
	at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:390)
	at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:377)
	at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:341)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1842)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1779)
	... 72 more
@wilkinsona
Copy link
Author

wilkinsona commented May 21, 2019

The knock-on effect of a maintenance release of Jackson requiring a new minor release of woodstox-core which, in turn, requires a new major release of stax2-api is a bit nasty. From a compatibility perspective, I think it might be preferable for Jackson 2.9.x to stick with Woodstox 5.0.x.

@cowtowncoder
Copy link
Member

Ouch. Yes, this is unfortunate. Wish I had caught that before. Thank you for reporting this.

Unfortunately the way Maven releases work, I am not sure much that can be done to help here.
I can upgrade stax2-api dependency for 2.9, but that (or downgrade if that was done) would only go in 2.9.10 and I don't think that will get released in near future.

@wilkinsona
Copy link
Author

wilkinsona commented May 22, 2019

Thanks, @cowtowncoder. I discovered this in the context of Spring Boot where we have some dependency management that gives us a chance to adjust the versions without requiring a new release of Jackson.

Is downgrading Woodstox to 5.0.x a viable option for use with jackson-dataformat-xml 2.9.9? The upgrade commit suggests that it'll work but there may, of course, have been subsequent changes that require 5.1.

Alternatively, we could use our dependency management to upgrade stax2-api to 4.1, but the jump to a new major version makes me a bit nervous. There are other projects in the ecosystem, such as SolrJ and, as you know, woodstox-core-asl that currently use 3.1.4. Is stax2-api 4.1 backwards compatible with 3.1?

@dadoonet
Copy link

FYI after upgrading to 2.9.9, my project is now complaining because it's checking transitive dependencies:

[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce) @ fscrawler-framework ---
[WARNING] 
Dependency convergence error for org.codehaus.woodstox:stax2-api:3.1.4 paths to dependency are:
+-fr.pilato.elasticsearch.crawler:fscrawler-framework:2.7-SNAPSHOT
  +-com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.9.9
    +-org.codehaus.woodstox:stax2-api:3.1.4
and
+-fr.pilato.elasticsearch.crawler:fscrawler-framework:2.7-SNAPSHOT
  +-com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.9.9
    +-com.fasterxml.woodstox:woodstox-core:5.1.0
      +-org.codehaus.woodstox:stax2-api:4.1

[WARNING] Rule 1: org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:

@cowtowncoder What about using maven-enforcer-plugin with a dependencyConvergence rule which would hopefully detect this? Example here: https://github.com/dadoonet/fscrawler/blob/master/pom.xml#L305-L328

@cowtowncoder
Copy link
Member

@dadoonet I am not opposed to enforcer plugin, although I thought it was already enabled for Jackson.

Further on upgrade: I did not realize (... despite being author of Woodstox :) ) that there was such a jump for stax2-api. Would have only upgraded for 2.10.

@cowtowncoder
Copy link
Member

@wilkinsona Yes, downgrade should be fine as far as I know. Alternatively I would think that upgrade of stax2-api to 4.x ought to be ok since despite it being major version jump, there is no fundamental major refactoring or incompatibility. Rather change was to align with Woodstox 5.0, although somewhat ironically I did not (alas) upgrade Woodstox 5.0 use it (facepalm).

So. I think that as long as you align woodstox-core and stax2-api, external API used by everything outside should be stable and compatible.

@wilkinsona
Copy link
Author

Thank you, @cowtowncoder, that's very helpful.

@usulkies
Copy link

usulkies commented May 29, 2019

Temporary workaround:
use mvn dependency:tree to make sure Jackson-dataformat-xml is the only user of the stax2-api
After assuring that, add this to dependency management section in your pom.xml:

<dependency> <!-- Workaround for jackson-dataformat-xml issue #340 of forgetting to update stax2-api  -->  
  <groupId>org.codehaus.woodstox</groupId>  
  <artifactId>stax2-api</artifactId>  
  <version>4.1</version>  
</dependency>

@cowtowncoder
Copy link
Member

I wonder if this is something that actually belongs in jackson-bom for version alignment purposes.

@wilkinsona
Copy link
Author

FWIW, my opinion is that it does not. The bom is for Jackson so it should only manage the versions of Jackson's own modules and not of its dependencies.

dadoonet added a commit to dadoonet/fscrawler that referenced this issue Jun 1, 2019
Because of FasterXML/jackson-dataformat-xml#340,
we need to explicitly declare `stax2-api:4.1` dependency.

Also another non related side effect is that we need to exclude
`jackson-module-jaxb-annotations` from `jersey-media-json-jackson`.

Related to #734.
@cowtowncoder
Copy link
Member

Fair enough. It seems like bit of borderline case where it might help, but since use for XML (f.ex) is only from xml format module and JAX-RS xml provider, there isn't that much benefit even if it was added. Same goes for JAXB api.

@dpeger
Copy link

dpeger commented Jun 12, 2019

@cowtowncoder I'm currently experiencing the same problem given a dependency to jackson-jaxrs-xml-provider:2.9.9. In jackson-jaxrs-xml-provider there is a dependency to woodstox-core-asl:4.4.1 which is causing issues together with the dependency to woodstox-core:5.1.0 in jackson-dataformat-xml.

This is the resulting dependency graph (gradle):

com.fasterxml.jackson.jaxrs:jackson-jaxrs-xml-provider:2.9.9
+--- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:2.9.9 (*)
+--- com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.9.9
|    +--- com.fasterxml.jackson.core:jackson-core:2.9.9
|    +--- com.fasterxml.jackson.core:jackson-annotations:2.9.0 -> 2.9.9
|    +--- com.fasterxml.jackson.core:jackson-databind:2.9.9 (*)
|    +--- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:2.9.9 (*)
|    +--- org.codehaus.woodstox:stax2-api:3.1.4 -> 4.1
|    \--- com.fasterxml.woodstox:woodstox-core:5.1.0 (*)
+--- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:2.9.9 (*)
+--- org.codehaus.woodstox:stax2-api:3.1.4 -> 4.1
\--- org.codehaus.woodstox:woodstox-core-asl:4.4.1
     +--- javax.xml.stream:stax-api:1.0-2
     \--- org.codehaus.woodstox:stax2-api:3.1.4 -> 4.1

The dependency to woodstox-core-asl is causing a java.lang.NoSuchMethodError: org.codehaus.stax2.ri.EmptyIterator.getInstance()Lorg/codehaus/stax2/ri/EmptyIterator; during application startup:

org.springframework.beans.factory.BeanDefinitionStoreException: Unexpected exception parsing XML document from ServletContext resource [/WEB-INF/applicationContext.xml]; nested exception is java.lang.NoSuchMethodError: org.codehaus.stax2.ri.EmptyIterator.getInstance()Lorg/codehaus/stax2/ri/EmptyIterator;
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:419) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:336) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:304) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:188) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:224) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:195) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.web.context.support.XmlWebApplicationContext.loadBeanDefinitions(XmlWebApplicationContext.java:125) ~[spring-web-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.web.context.support.XmlWebApplicationContext.loadBeanDefinitions(XmlWebApplicationContext.java:94) ~[spring-web-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationContext.java:133) ~[spring-context-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:636) ~[spring-context-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:521) ~[spring-context-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:400) ~[spring-web-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:291) [spring-web-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103) [spring-web-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792) [catalina.jar:8.5.35]
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256) [catalina.jar:8.5.35]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) [catalina.jar:8.5.35]
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754) [catalina.jar:8.5.35]
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730) [catalina.jar:8.5.35]
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734) [catalina.jar:8.5.35]
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985) [catalina.jar:8.5.35]
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857) [catalina.jar:8.5.35]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: java.lang.NoSuchMethodError: org.codehaus.stax2.ri.EmptyIterator.getInstance()Lorg/codehaus/stax2/ri/EmptyIterator;
	at com.ctc.wstx.sw.OutputElementBase.getPrefixes(OutputElementBase.java:358) ~[woodstox-core-asl-4.4.1.jar:4.4.1]
	at org.apache.cxf.staxutils.StaxUtils.writeStartElement(StaxUtils.java:813) ~[cxf-core-3.3.1.jar:3.3.1]
	at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:760) ~[cxf-core-3.3.1.jar:3.3.1]
	at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:724) ~[cxf-core-3.3.1.jar:3.3.1]
	at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:674) ~[cxf-core-3.3.1.jar:3.3.1]
	at org.apache.cxf.configuration.spring.AbstractBeanDefinitionParser.mapElementToJaxbProperty(AbstractBeanDefinitionParser.java:349) ~[cxf-core-3.3.1.jar:3.3.1]
	at org.apache.cxf.configuration.spring.AbstractBeanDefinitionParser.mapElementToJaxbProperty(AbstractBeanDefinitionParser.java:301) ~[cxf-core-3.3.1.jar:3.3.1]
	at org.apache.cxf.transport.http.spring.HttpConduitBeanDefinitionParser.doParse(HttpConduitBeanDefinitionParser.java:58) ~[cxf-rt-transports-http-3.3.1.jar:3.3.1]
	at org.springframework.beans.factory.xml.AbstractSingleBeanDefinitionParser.parseInternal(AbstractSingleBeanDefinitionParser.java:88) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.AbstractBeanDefinitionParser.parse(AbstractBeanDefinitionParser.java:63) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.NamespaceHandlerSupport.parse(NamespaceHandlerSupport.java:74) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement(BeanDefinitionParserDelegate.java:1366) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement(BeanDefinitionParserDelegate.java:1352) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.parseBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:179) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.doRegisterBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:149) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.registerBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:96) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.registerBeanDefinitions(XmlBeanDefinitionReader.java:513) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	at org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:393) ~[spring-beans-5.1.7.RELEASE.jar:5.1.7.RELEASE]
	... 26 more

I don't think your commit 81bcf12 will fix this problem. As the dependency to woodstox-core-asl in jackson-jaxrs-xml-provider remains.

I think that is the same problem @maksymgendin mentioned here.

Excluding woodstox-core-asl resolves the issue during startup but not sure about other problems further down the line...

Relates to FasterXML/woodstox#46

@wilkinsona
Copy link
Author

Alternatively I would think that upgrade of stax2-api to 4.x ought to be ok since despite it being major version jump, there is no fundamental major refactoring or incompatibility

Looking at the comment above from @dpeger, there is an incompatibility. The signature of EmptyIterator.getInstance() has changed from:

public static EmptyIterator getInstance() { return sInstance; }

To:

public static <T> Iterator<T> getInstance() { return (Iterator<T>) sInstance; }

This is binary incompatible and may be source incompatible depending on how the result of getInstance() is used. Downgrading Woodstox back to 5.0.x appears to be the only way to restore compatibility when using Jackson 2.9.9. Perhaps this would be a better solution for 2.9.10 too?

@cowtowncoder
Copy link
Member

I can see that this is (alas!) binary incompatible, but I do not think it is source-incompatible for any use case (I don't see use case for caller relying on EmptyIterator type). But that does not matter greatly.

But I am not quite sure why downgrade would necessarily be preferable to upgrade in Jackson components: both combinations (if working with other frameworks) should work, and the problem is only between new-Woodstox/old-stax2 (and possibly vice versa)?

2.9.10 won't be out for months, most likely, so we have time to think this through. But I would like to understand what could be done, first: for example, maybe it is possible to make latest Woodstox patch work around the problem (to work against 3.x and 4.x). I know that will not help with versions that are out there, but would help with allowing "latest OR specific older version" case.

@cowtowncoder
Copy link
Member

Ok. So, the issue was reported for Woodstox itself:

FasterXML/woodstox#46

and so the problem is specific to Woodstox version 5.1.0, and should be resolved for 5.2.0 (and 5.2.1. Key, then, would be to avoid combination of stax2-api 4.x and woodstox-core 5.1.0.
Newer versions should work (if not please let me know).

@dpeger
Copy link

dpeger commented Jun 14, 2019

@cowtowncoder as you didn't change anything other than removing the dependency to woodstox-core-asl:4.4.1 and updating to woodstox-core:5.2.1, is it considered to be save to exclude the woodstox-core-asl:4.4.1 dependency for 2.9.9?

This would be our workaround for this issue until 2.9.10 is released.

@cowtowncoder
Copy link
Member

@dpeger Yes, I think so. I wish I had paid more attention to version alignment since there is now the problem of XML format module and JAX-RS provider.

It is possible that this problem, along with 2 CVEs, may mean that I'll do 2.9.10 instead of micro-patches; at point where 3 components (well, with JAX-RS being sub-modules, more) need new versions might as well do full release.

@baharclerode
Copy link

@dadoonet FYI, dependencyConvergence won't catch this because jackson-dataformat-xml does actually specify a version, so the versions do converge. What would've caught this is the requireUpperBounds enforcer rule, which will prevent a module from pulling in an older version than one of its dependencies (unless explicitly excluded)

@cowtowncoder cowtowncoder modified the milestones: 2.9,8, 2.9.10 Jul 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants