_isReady never set to true when working with IRKernel

[parente]

@jameslmartin and I spotted this working on jupyter/dashboards_server#169

When creating a R kernel using startNewKernel, everything proceeds normally through POSTing to the server and websocket creation. The promise gets resolved and we begin making Kernel.execute calls. These calls, however, wind up getting queued indefinitely. Tracing through the code, we see that _isReady is never getting set to true.

The reason: We see the kernel_info_request message go out on the websocket and the response come back from IRKernel, but there is no other status change message indicating that the kernel is busy, idle, or anything else.

We dug through the jupyter_client docs a bit to see if there was some definitive way for a kernel to indicate its readiness for use and a client to know about that readiness, but we didn't spot anything in black and white. A simple fix here might be to set _isReady=true when the kernel info response comes back, but we don't know if that's just a bandaid or the correct approach across kernels.

/cc @minrk @takluyver @jhpedemonte

[minrk]

status idle/busy messages should be sent around all kernel request/reply handlers. This is used to indicate that IOPub is hooked up correctly, in addition to the shell channel. That's the first thing I would check. A failure to receive the status-idle message on the reply is meant to indicate that either:

1. the protocol isn't being followed correctly, or
2. the IOPub channel hasn't been connected properly

But I would expect the same behavior in the 4.x notebook as well if that's the case, because we also rely on the status-idle message on a kernel info request to set the initial state of the busy/idle kernel indicator. So maybe it's something not quite right in the implementation here.

[parente]

So maybe it's something not quite right in the implementation here.

To help with that, I'll try to craft a standalone example using the nodejs kernel gateway demo and post it back here to reproduce. (We were in the middle of testing the dashboard server, and I'm pretty sure we ruled out any problem there since Python works fine, but the KG demo will be a good external sanity check.)

[parente]

But I would expect the same behavior in the 4.x notebook as well if that's the case, because we also rely on the status-idle message on a kernel info request to set the initial state of the busy/idle kernel indicator.

Interestingly, if I start a new R notebook on tmpnb.org and watch the websocket frames, I see only the two messages there as well, the kernel_info_request and the kernel_info_reply:

No status messages are received, yet the state indicator is set properly and the notebook works. This suggests the current notebook JS is relying on something other than the iopub status to know that the kernel is ready for use.

In contrast, the Julia, Python, and Scala kernels all publish busy/idle on iopub around the reply.

[takluyver]

I think we (IRkernel) only send status messages around execute, rather than every shell request. IIRC, we wrote that code before the spec was clear about when they're sent, and I guessed wrong. It should be a fairly easy task if someone is looking to start contributing.

[parente]

I'm not an R expert, but I took a stab at it and sent a PR against IRKernel.

I think we can close this one since jupyter-js-services seems to be following the best practice and is working with other kernels.