Corrupted domain.xml after configuration changes PAYARA-3880

[ionutursuleanu]

After changing -Xmx512m to -Xmx1024m, for example, the admin console saves to configuration entries: -Xmx512m & -Xmx1024m=. If I try to remove one of the entries the following exception is thrown

[2019-05-30T19:36:11.003+0300] [Payara 5.192] [SEVERE] [NCLS-CORE-00003] [javax.enterprise.system.core] [tid: _ThreadID=198 _ThreadName=admin-thread-pool::admin-listener(5)] [timeMillis: 1559234171003] [levelValue: 1000] [[
  Exception while running a command
java.lang.IllegalArgumentException: Escape at EOL
	at org.glassfish.common.util.admin.ParamTokenizer.scanToken(ParamTokenizer.java:152)
	at org.glassfish.common.util.admin.ParamTokenizer.nextToken(ParamTokenizer.java:97)
	at org.glassfish.common.util.admin.MapInjectionResolver.convertStringToList(MapInjectionResolver.java:589)
	at org.glassfish.common.util.admin.MapInjectionResolver.convertStringToObject(MapInjectionResolver.java:370)
	at org.glassfish.common.util.admin.MapInjectionResolver.convertListToObject(MapInjectionResolver.java:403)
	at org.glassfish.common.util.admin.MapInjectionResolver.getValue(MapInjectionResolver.java:136)
	at org.jvnet.hk2.config.InjectionManager.syncDoInject(InjectionManager.java:170)
	at org.jvnet.hk2.config.InjectionManager.inject(InjectionManager.java:74)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl.injectParameters(CommandRunnerImpl.java:386)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1242)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1300(CommandRunnerImpl.java:119)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1865)
	at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1741)
	at org.glassfish.admin.rest.utils.ResourceUtil.runCommand(ResourceUtil.java:270)
	at org.glassfish.admin.rest.utils.ResourceUtil.runCommand(ResourceUtil.java:238)
	at org.glassfish.admin.rest.utils.ResourceUtil.runCommand(ResourceUtil.java:292)
	at org.glassfish.admin.rest.resources.CollectionLeafResource.runCommand(CollectionLeafResource.java:307)
	at org.glassfish.admin.rest.resources.CollectionLeafResource.create(CollectionLeafResource.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
	at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:353)
	at org.glassfish.admin.rest.adapter.RestAdapter$2.service(RestAdapter.java:335)
	at org.glassfish.admin.rest.adapter.RestAdapter.service(RestAdapter.java:189)
	at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
	at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
	at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
	at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
	at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
	at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
	at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
	at java.lang.Thread.run(Thread.java:748)
]]

Environment

• Payara Version: 5.192
• Edition: Full
• JDK Version: 8 - Oracle
• Operating System: Windows

[smillidge]

Internal bug ID PAYARA-3880

[jbee]

#4002 will address creation of = after -Xmx1024m. This should no longer occur. The EOL exception is another issue that is not addressed by this PR. Another PR has to take care of it. Most likely problem in org.glassfish.common.util.admin.ParamTokenizer in combination with escaping. As far as I can tell the = at the end is first escaped to \= somewhere early. When later the option is split into key and value the \ remains at the end and is encountered by the ParamTokenizer which presumably correctly complains about the escape at EOL.

This could mean that there is a less visible issue when \ is kept at the end of an option that isn't at the EOL. While #4002 will strip those \ it could be that their presents could have effected start of next option following the \ in processing taken place before the map -> string conversion.

[pkumar72]

Domain,xml gets corrupted not just when editing a existing parameter, the problem occurs even when adding new parameter. I tried to add a new jvm option via admin gui in payara 5.192 and the server failed to restart.

Just to verify, i tried adding the same jvm option from command line (asadmin) in a clean new instance of 5.192 and it worked great.

[jbee]

@pkumar72 Your observation fits the exceptions. The issue is located in the processing applied when changing JVM options though the GUI. Since any change will process all options the problem also occurs for exiting options matching the requirements to trigger the issue.

[jbee]

Another note: while my comment #3992 (comment) should still be valid I did have problems reproducing the EOL issue after #4002 was merged.

[Dougnlizt]

Glad this is getting fixed... in my particular case, it was jvm option -XX:NewRatio=2. After saving changes (using the admin GUI), this option was getting changed to [9|]--add-exports:-XX:NewRatio=2. After changing it back, the EOL error went away.

[smillidge]

This was fixed on 5.193 any further issues raise a new issue.