Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enrich Docs with Usage #28

Merged
merged 14 commits into from
Dec 1, 2019
6 changes: 0 additions & 6 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,6 @@
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'

# Jupyter uses recommonmark's parser to convert markdown
source_parsers = {
'.md': 'recommonmark.parser.CommonMarkParser',
}


# The encoding of source files.
#source_encoding = 'utf-8-sig'

Expand Down
5 changes: 1 addition & 4 deletions docs/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,8 @@ channels:
- conda-forge
dependencies:
- pyzmq
- python==3.5
- python==3.7
- traitlets>=4.1
- jupyter_core
- sphinx>=1.3.6
- sphinx_rtd_theme
- pip:
- recommonmark
- sphinx-markdown-tables
554 changes: 440 additions & 114 deletions docs/images/kernel_mgmt_server_sequence.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions docs/kernel_providers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,8 @@ above).

# Use `manager` for lifecycle management, `client` for communication

.. _included_kernel_providers:

Included kernel providers
=========================

Expand Down
28 changes: 0 additions & 28 deletions docs/server.md

This file was deleted.

40 changes: 40 additions & 0 deletions docs/server.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
.. _server:

=======================
Use with Jupyter Server
=======================

This page describes the way `Jupyter Server <https://github.com/jupyter/jupyter_server>`_ uses the Kernel Management module.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultimately this page probably belongs in the Jupyter Server docs, but if it makes sense to put it here at the moment, I don't have a problem with that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will work next week with @Zsailer on the jupyter server doc. If this is merged before, we can reshuffle the details and also do a second pass on the kernel mgmt docs.


.. image:: ./images/kernel_mgmt_server.svg
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nitpick in this image: the connection between Jupyter server and JKM is labelled "Jupyter Protocol (ZeroMQ)". This isn't quite right - Jupyter Server is using JKM through Python APIs (classes & functions), and through that talks the Jupyter protocol to the kernels themselves.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@takluyver great catch. I have updated the diagram to reflect this. There is still room for improvements on that diagram, but I think it is good-enough for now and at least does not show wrong info anymore.


On the Jupyter Server side, each WEB handlers located in services/kernels/handlers:

- MainKernelHandler
- KernelHandler
- KernelActionHandler
- ZMQChannelHandler
- MainKernelSpecHandler

Each of these handlers have:

- A ``kernel_manager`` - the default manager is `MappingKernelManager` provided by `jupyter_server`.
- A ``kernel_finder`` - which is imported from the `jupyter_kernel_mgmt` library
echarles marked this conversation as resolved.
Show resolved Hide resolved

Jupyter Server runs with a single `ServerApp` built around a ``SessionManager``. The SessionManager uses a ``MappingKernelManager``. All instances of MappingKernelManager have a ``KernelFinder``.

The :ref:`included kernel providers <included_kernel_providers>` (:class:`KernelSpecProvider <jupyter_kernel_mgmt.discovery.KernelSpecProvider>` and :class:`IPykernelProvider <jupyter_kernel_mgmt.discovery.IPykernelProvider>`) register their entrypoints.

.. code-block:: python

entrypoints:
jupyter_kernel_mgmt.kernel_type_providers' : [
'spec = jupyter_kernel_mgmt.discovery:KernelSpecProvider',
'pyimport = jupyter_kernel_mgmt.discovery:IPykernelProvider',
]

External Providers can register their own entypoints, e.g the `kubernetes_kernel_provider <https://github.com/gateway-experiments/kubernetes_kernel_provider>`_ which extends the `remote_kernel_provider <https://github.com/gateway-experiments/remote_kernel_provider>`_.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think stating that the kubernetes provider extends remote kernel provider is that pertinent here. The RKP is more of an "abstract base provider". Instead, perhaps adding a link to Thomas' jupyter_ssh_kernels would be useful? @takluyver - are you okay with that? If not, you can reference the yarn kernel provider repo instead.

(What happens here affects the corresponding diagram as well.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated doc and text
@takluyver ping me if ok to add jupyter_ssh_kernels. Thx


The interactions sequence between Jupyter Server and the Kernel Management is sketched here.

.. image:: ./images/kernel_mgmt_server_sequence.svg
11 changes: 0 additions & 11 deletions docs/standalone.md

This file was deleted.

14 changes: 14 additions & 0 deletions docs/standalone.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
.. _standalone:

================
Standalone Usage
================
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section could use more content. Although I'm not entirely aware of how jupyter_client was used outside of Notebook, I know that its used directly. I believe JKM's goal is similar. You might take a look at some of the client test code to get an idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added paragraphs for this.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may rework this more at a later point. It is meant to be a public Python API as well as a component for Jupyter server.


The aim of the Kernel Management is to be integrated in larger application such as the :ref:`Jupyter Server <server>`.
echarles marked this conversation as resolved.
Show resolved Hide resolved

Although, it can be used a standalone module to, for example, launch a :ref:`Kernel Finder <kernel_finder>`
echarles marked this conversation as resolved.
Show resolved Hide resolved
from the command line and get a list of :ref:`Kernel Specifications <kernelspecs>`

.. code-block:: python

python -m jupyter_kernel_mgmt.discovery