Slow jupyterlab bootup

[amit-chandak-unskript]

Hi @kevin-bates , as you know, we are using jupyterlab with enterprise-gateway and we see the loading of jupyterlab being pretty slow, in the order of 30-40 secs.
We use s3 as the ContentsManager for jupyterlab.

When the same setup is used without enterprise-gateway enabled, the loading is much faster.

As you can see nbconvert taking a very long time.
Any pointers on why nbconvert is taking a lot of time? and what could be causing this delay. Is it related to using S3 contents manager or using enterprise gateway. Where should i be focusing on to debug this delay?

[kevin-bates]

Hi @amit-chandak-unskript.

EG will only come into play when the server's KernelManager and/or KernelSpecManager are asked to do work (and the server, of course, is configured to use EG for its kernels). That said, in looking into the nbconvert code, I see its ExecutionPreprocessor derives from nbclient which, I believe is what nbconvert uses to apply conversions to the notebooks. If no kernel manager is provided (which I don't see one), it looks like nbclient will look in its configuration to determine a kernel manager and so you should probably check to see if somehow, because nbclient is essentially within the Jupyter Server process at this time, its using the GatewayKernelManager that is configured when --gateway-url is provided.

Regarding S3, I suppose that could be coming into play somehow and would suggest you try not using S3 to compare (and help isolate the issue).

Since this is happening at startup, is your Lab configured to perform some sort of conversions? If this is default behavior, I have never seen this situation where my EG server is getting hit with a kernel request while Lab is booting. You should be able to do check this by monitoring the EG log file (with DEBUG enabled) while Lab is starting. One thing you should do prior to performing this experiment is to make sure to STOP all kernels in Lab (and close all notebook tabs) prior to stopping Lab. This way, when Lab starts, those kernels won't interfere in your experiment since Lab insists on warming up any previously-opened notebooks when it starts.

So, to summarize, I would determine if nbclient (on behalf of nbconvert) is using GatewayKernelManager. If its using the local AsyncKernelManager and using local kernels to perform its conversion, then you should look at this from the S3 angle.

Let us know how it goes.

[amit-chandak-unskript]

Thanks a lot @kevin-bates , let me explore this way and will update this thread.