Stop using IPython.lib.kernel 0.13.2 shim and add initial support for Jupyter

[SylvainCorlay]

• find_connection_file was being imported from IPython.lib.kernel which is a shim to IPython.kernel.connect since 1.0. Therefore, this does not change anything for IPython 1.0 to 3.2. We are just using a more direct path.
• Starting with IPython 4.0,
  • IPython.lib.kernel is a shim to ipykernel.connect
  • IPython.kernel.connect is a shim as well, which first imports everything from ipykernel.connect and then everything from jupyter_client.connect. The latter sets the default directory for connection files from ~/.ipython/profile_*/security to JUPYTER_DATA_DIR, which is the correct behavior.

[SylvainCorlay]

OK, this solves the last issue I had with Jupyter, where find_connection_file has a slightly different signature.

[minrk]

Minor correction: jupyter connection files go in JUPYTER_DATA_DIR, not ~/.jupyter, and Jupyter has no profiles, so there is no ~/.jupyter/profile_*.

[SylvainCorlay]

Thanks @minrk I edited the description.

[stonebig]

could it ship in coming Spyder 2.3.5 ?

[SylvainCorlay]

@stonebig good point, I guess we should do it if 3.0 does not ship before IPython 4.0.

[ccordoba12]

So with this PR we will be compatible with IPython/Jupyter 4.0?

[SylvainCorlay]

Yes, it works with the current development version.

[ccordoba12]

Ok, thanks @SylvainCorlay for the confirmation.

@stonebig, I'll backport these changes to release them in 2.3.5.

[stonebig]

Thanks.

[ccordoba12]

@SylvainCorlay, there are other instances where we're using profile_default:

1. https://github.com/spyder-ide/spyder/blob/master/spyderlib/plugins/externalconsole.py#L996
2. https://github.com/spyder-ide/spyder/blob/master/spyderlib/widgets/ipython.py#L680
3. https://github.com/spyder-ide/spyder/blob/master/spyderlib/widgets/externalshell/start_ipython_kernel.py#L33

What should we do with them?

[SylvainCorlay]

• IPython.config.loader is shimed to traitlets.config.loader, so the function load_pyconfig_files is found. In the case it does not find connection files, nothing happens. Hence 2 and 3 should not create any issue. I guess we want to keep this though for backward compatibility.
• Regarding 1, I wonder whether we should clean up for the connection file at all.
• Remark: We could add comments wherever we use IPython shims, so that we know where to import from when we drop support for older versions.

[ccordoba12]

I'm going to merge this to have some kind of support for Jupyter 4.0. Later we can see what we can do about 3.

@SylvainCorlay, please open a new PR with the comments you suggest. I guess there are going to be several of them :-)

[SylvainCorlay]

Thanks. I am looking into the points you raised and it is not immediate. I should come up with another PR soon.

[ccordoba12]

Ok, thanks!