No module named 'mywb.sklearn_cookbook'

[jhconning]

The tmpnb online test_cookbooks example worked flawlessly. However I'm running into errors when I try to run same notebook from the cloned repository on my windows 10 machine.

The new search functionality from the jupyter file navigation is working from the Jupyter file browser (and is awesome).
However when I load and the test_cookbooks.ipynb notebook I run into trouble. The first cell

%load_ext urth.cms 

magic runs with no problem but the cell

In [3]:  import mywb.sklearn_cookbook as skcook
         import mywb.api_cookbook as apicook

leads to

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-3-45432c89e6a6> in <module>()
----> 1 import mywb.sklearn_cookbook as skcook
      2 import mywb.api_cookbook as apicook

ImportError: No module named 'mywb.sklearn_cookbook'

As mentioned, I'm running this on an exact clone of the repo. I see and can open the sklearn_cookbook notebook. It's just that the 'fake dir' for mywb does not seem to be working.

P.S. -- This project is fantastic. It will transform how I organize code and notebooks. Thanks.

[parente]

Thanks for reporting the problem. You mentioned you're running on a clone of the repo. I'm assuming that means you both installed the extension from it, not from the stable pip install, and are using the notebooks from their location within the repo directory structure. If this is correct, a couple potential problems come to mine:

• The mywb reference refers to the root notebook directory of your notebook server. If you're trying to refer to notebooks in other folders, you have to refer to them on the module/package dotted path (e.g., mywb.some.other.path.some_notebook). There's a function that you can use instead as well, load_notebook I believe, for file names/paths that are not valid Python module/package names.
• The master branch and 0.2.x branch in the repo are currently compatible with Jupyter Notebook 4.x and up. The 0.1.x branch is compatible with IPython Notebook 3.2.x. Are you using a version that matches your install?

If neither of these relates to your problem, could you please write back with your notebook server version?

Edit: Typos while trying to type on a mobile. :/

[jtyberg]

@jhconning I agree with you about changing the way you organize notebooks; once you start importing notebooks, it's hard to stop.

I have to admit that it has been a looong time since I've used Windows. However, I did find a Windows 7 machine (I don't have Windows 10 handy, nor did I consider upgrading the Windows 7 machine I did find), and I tried to reproduce your error.

• I went with an install using Anaconda for 64-bit and Python 3.5.
• I then tried to pip install jupyter_cms, but I ran into an issue when pip tried to install the scandir module dependency. As it turns out, scandir has been added to the Python standard library as of 3.5, so I went ahead and created a PR (Use scandir from Python standard library, if present. #10) to try to import from the standard library, then fall back to the scandir module. I then created a new jupyter_cms distribution using make sdist from my branch. From my Anaconda Prompt shell on Windows, I did pip install C:\path\to\jupyter_cms-0.3.0.dev0.
• After going through all that, I ran jupyter notebook from my Anaconda Prompt.
• I then uploaded a copy of the sklearn_cookbook.ipynb using the Notebook interface. So it was in my root notebook directory.
• I created a new notebook and was able to import the sklearn_cookbook notebook.

As for your environment, the first thing I would check is the path to the sklearn_notebook. Try an import statement using the full path from the base of the notebook directory, e.g.

import mywb.path.to.sklearn_cookbook as skl

[jhconning]

Thank you all for your detailed troubleshooting help.
Thanks to @parente I did get this to work via the load_notebook() route!

However the 'import mywb ...' route still does not work for me on a Windows machine. In case it helps you track the problem here is my setup:

• I am running jupyter server 4.0.6 on python 3.4.3 (Anaconda) [MSC v.1600 64 bit (AMD64)]
• I installed via pip install jupyter_cms. I ran into a similar problem with scandir as @jtyberg but once that was fixed (as described below ) everything installs and I see `Successfully installed jupyter-cms-0.2.0'
• I start a jupyter notebook and cms 'Search' function at root directory at http://localhost:8888/tree works
• I have placed 'test_cookbooks.ipynb' and 'sklearn_cookbook.ipynb' into the root directory viewed at http://localhost:8888/tree. To leave nothing to the imagination, here are notebook cells and their output leading up to the import that ought to verify folder locations:
  
  The good news is that the load_notebook() route does work:
  

Regarding the scandir problem. I fear other Windows users may also encounter an error at pip install jupyter_cms. That seems to be because Windows pip installs that require a C/C++ compiler to build often fail for various reasons (many many references to this problem on SO like this). As I'd seen this problem before (e.g. with geopandas) I have found the quickest fix to just pip install scandir directly from its Windows wheel file found here.

Thanks again for your help and this expansion in the functionality and power of jupyter.

[parente]

The good news is that the load_notebook() route does work:

The fact that the function works tells me there's a platform specific bug in the handling of the dotted module name. I don't have Windows, but I can give the code a once over.

That seems to be because Windows pip installs that require a C/C++ compiler to build often fail ...

In case it's an option for you or anyone else that finds this issue in the future, the https://github.com/jupyter/docker-stacks project has a set of ready-to-run, Debian-based images that you could use on your Windows machine via the Docker Toolbox.

[parente]

I took a look at the code. The bug is likely in the finder classes https://github.com/jupyter-incubator/contentmanagement/blob/master/urth/cms/loader.py#L228. I'm not seeing it by reading the code. Need a couple prints for name and path in those classes, and then a a couple of attempts to import notebooks using the dotted module notation on Windows.

[Foucl]

I started to have a look. I'm on Win 10, x64, Anaconda Python 2.7.

I think I located the issue - at least for me. It seems to be in __init.py__, where the working directory of the current notebook is searched for. For me, it just picked one (seemingly random) of the 'nbserver-XXX.json' files in my jupyter_runtime_dir(), not the one of the current session. Removing all but the 'right' nbserver-XXX.json file in my runtime_dir worked around that, thus calling loader.enable with the correct work_dir. Imports then work. This might be my fault, maybe there shouldn't be old jsons lingering around in that directory. A quick google search does not reveal a better method for getting the current working directory (SO has the same way you are doing it currently). Maybe the fact that a 'clean' jupyter_runtime_dir is necessary could simply be documented?

Edit: typo

[parente]

I ran into this myself the other day for the first time. I do not know a better way of identify which notebook server is the "right" notebook server. Adding a note to the README about it sounds like the best approach for the moment.