jl.println and jl.display do not work in Jupyter notebook

[ShuhuaGao]

The following test was made in a .ipynb notebook in VS code.

We see that println has no output at all, while display leads to an error.

• Julia version 1.8.2
• Python version 3.9.12
• juliacall 0.9.7

[cjdoris]

If you do %load_ext juliacall.ipython first then it will work. I had never tried it from VSCode before, but am happy to see that it does!

https://cjdoris.github.io/PythonCall.jl/dev/compat/#IPython

(It's only in the dev version of the docs because it appears the docs have not built for the last few releases, that's a separate issue.)

[ShuhuaGao]

Thanks. I tested that %load_ext juliacall.ipython in VS Code notebook. It sometimes worked, and the behavior was weired.

• Work
  

• Not work. If I insert a jl.versioninfo() at the beginning, then println gives no output.
  

However, jl.VERSION does not have that side effect.

[cjdoris]

Turns out what I had implemented was pretty hacky. It's much better now, I've made a release, it certainly seems to be much better in VSCode now.

[ShuhuaGao]

That's great. Now the above issue of versioninfo() is fixed. See figure below.

However, if I mix up println and display (or even print in Python), the output order seems corrupted.

Note that, if the same code is written as a python script, then the output order is correct.

[cjdoris]

Yeah, that's because in Julia you can't (reliably) replace stdout in Julia, you can only redirect it, and that needs to happen asynchronously, so anything printed to stdout from Julia may appear out of order with things printed from Python. display should always appear in order though.

[ShuhuaGao]

display should always appear in order though.
Yes, it is true in my test.

Besides, flush after println may be a workaround to enforce the print in order.

[github-actions]

This issue has been marked as stale because it has been open for 30 days with no activity. If the issue is still relevant then please leave a comment, or else it will be closed in 7 days.

[github-actions]

This issue has been closed because it has been stale for 7 days. If it is still
relevant, please re-open it.

[MilesCranmer]

Can this be done automatically? i.e.,

1. Check if we are in a Jupyter notebook with hasattr(__builtins__, "__IPYTHON__").
2. Call get_ipython().run_line_magic('load_ext', 'juliacall.ipython').

[MilesCranmer]

I threw this in PySR and it seems to work (modified from tqdm):

try:
    get_ipython = sys.modules["IPython"].get_ipython

    if 'IPKernelApp' not in get_ipython().config:
        raise ImportError("console")

    print("Detected Jupyter notebook. Loading juliacall extension.")

    get_ipython().run_line_magic("load_ext", "juliacall")
except Exception:
    pass

[cjdoris]

That seems reasonable - if we also provide a mechanism (env var) to turn this behaviour off.

[MilesCranmer]

Great minds think alike! 😄 Here's the code I ended up releasing in PySR:

# Next, automatically load the juliacall extension if we're in a Jupyter notebook
autoload_extensions = os.environ.get("PYSR_AUTOLOAD_EXTENSIONS", "yes")
if autoload_extensions in {"yes", ""} and jl_version >= (1, 9, 0):
    try:
        get_ipython = sys.modules["IPython"].get_ipython

        if "IPKernelApp" not in get_ipython().config:
            raise ImportError("console")

        print(
            "Detected Jupyter notebook. Loading juliacall extension. Set `PYSR_AUTOLOAD_EXTENSIONS=no` to disable."
        )

        # TODO: Turn this off if juliacall does this automatically
        get_ipython().run_line_magic("load_ext", "juliacall")
    except Exception:
        pass
elif autoload_extensions not in {"no", "yes", ""}:
    warnings.warn(
        "PYSR_AUTOLOAD_EXTENSIONS environment variable is set to something other than 'yes' or 'no' or ''."
    )

Want me to PR to juliacall? I think it is nice to have printing work automatically.

[cjdoris]

Yeah go for it. Please add a setting so this feature can be disabled.