ACC needs three seconds for InitialContext.lookup("myHome");

[mkarg]

I have a performance issue and hope that you can help me.

Inside Payara 4.1.1.154 ACC I'm trying to get remote EJBHome of a stateless session bean. ACC needs three seconds (!) for the following line:

initialContext.lookup("myHome");

(where "myHome" is either java:comp/env/FooBar or myPackage.FooBarHome -- makes no difference in speed)

Do you have an idea how I can speed up this, or how I could check for the actual cause? My customer wants getting the initial session to perform within less than one second, and I have no clue how to speed it up...

[smillidge]

I would suggest obtaining a thread dump during the 3 second pause to see what is happening at that line of code. Either use JConsole, JVisualVM or jstack to get the thread dump. It would also be useful to see the JNDI properties you are using to lookup the remote home.

[mkarg]

I do not supply any JNDI parameters, as my code is running inside of ACC. I simply do "new InitialContext();" and then invoke "initialContext.lookup(...)". BTW, the same problem happens when not directly using JNDI but let Paraya do that "under the hood" using @ejb annotation.

I took several stack traces, and all of them look differently at their top, but always the same at the bottom: They clearly show that the main thread is RUNNING at org.glassfish.gmbal.impl.MBeanTree.setRoot(MBeanTree.java:102), while the actual current code line invoked from here differes. In most cases it is somewhere inside typelib.TypeEvaluator.

(Please find a complete a sample jstack report attached).

#539-stackTrace.txt

So what can I do to improve this? Is Paraya simply too slow at the initial MBean construction so there is no actual help without chaning Paraya itself?

[smillidge]

I need to investigate more in the ORB and gmbal code as there doesn't seem to be an obvious code. This will be a one of performance hit as it is creating MBeans but not sure why it is taking so long at them moment.

[quipsy-karg]

Is there anything you like me to test meanwhile?

[smillidge]

A simple remote EJB hello World test case would be useful to see if this has exactly the same problem if you have time. Otherwise we can put that together or if you have the ability to profile your client to get an understanding of problematic methods.

[mkarg]

Thank you for your thoughts and kind help! :-)

If you have "EJB hello world" at hand I’ll be happy to try it out in my environment. Otherwise I can try to set this up next week. But actually I do not see why that should be any other than my application: Even when new InitialContext().lookup() is the very first command of my application, it still needs the same time (I already measured that). The problematic method is always the same: org.glassfish.gmbal.impl.MBeanTree.setRoot(MBeanTree.java:102).

So what exactly do you like me to profile?

[smillidge]

Hi we have reproduced this with a simple remote EJB client

    public static void main(String[] args) throws NamingException {
        Context ctx = new InitialContext();
        HelloWorldRemote remote = (HelloWorldRemote) ctx.lookup("fish.payara.examples.javaee.simpleremoteejb.HelloWorldRemote");
        System.out.println(remote.helloWorld());
    }

We are in the process of profiling this and can see it is the gmbal JMX library taking 3 seconds to initialise

[quipsy-karg]

Great! So I won't need to setup hello world on my own.

Do you see any chance to improve the startup time, or is fixing JMX out of scope?

[smillidge]

After some analysis there's no obvious quick fix so won't happen anytime soon. I think the best option is preventing JMX analysis of the ORB for clients with a parameter but that needs to be shoehorned into the ORB code.

[quipsy-karg]

It would be absolutely OK for us if there would be a working solution around March 2016. If you think this is not possible, will signing a commercial support contract make it possible? :-)

[smillidge]

A support contract really helps as this is how we pay ourselves to develop the software. Customer requests always come first :-)

[quipsy-karg]

Contract signed and sent. Thanks in advance! :-)

[jerrinot]

👍

[smillidge]

Thanks we appreciate it