-
-
Notifications
You must be signed in to change notification settings - Fork 375
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ipykernel 6.x breaks redirecting stdout #795
Comments
While we work this out, you can disable the behavior in your # ipython_config.py
c.IPKernelApp.capture_fd_output = False |
Thanks @minrk. It looks like presently there's no API to configure this programmatically? This creates a challenge for developing a package that's cluttering the notebook with log output which previously was all divertible to the terminal window. |
@minrk a little more troubleshooting detail. This issue occurs on all of my Linux machines. But, this issue does not occur on my Windows machine: all output can be diverted to the terminal window. However, if I run a notebook in a Linux Docker container on my windows machine this issue occurs. So it seems there is cross-platform inconsistency. On Windows I'm still able to control where stdout gets directed, like always, but on Linux that ability has been broken. |
There's not an API to turn it on and off, but looking at the code, this snippet will disable it permanently at runtime: for std in (sys.stdout, sys.stderr):
if hasattr(std, "_should_watch"):
std._should_watch = False # signal that fd-watching should stop via private API Turning it on and off via a public API might be tricky, but should be doable. wurlitzer can do this.
I believe the pipe-based low-level capture only occurs on posix systems, which is why Windows would not be affected. |
Unfortunately using that private import sys
from contextlib import redirect_stdout
terminal = sys.__stdout__
print('test1') # this prints to the notebook
if hasattr(sys.stdout, "_should_watch"):
sys.stdout._should_watch = False
with redirect_stdout(terminal):
print("test2", file=terminal, flush=True) # this doesn't print anywhere
print('test3') # this prints to the notebook |
My mistake, that only stops reading the pipe, it doesn't restore the original FD. This should do it: import os
import sys
for std, __std__ in [
(sys.stdout, sys.__stdout__),
(sys.stderr, sys.__stderr__),
]:
if getattr(std, "_original_stdstream_copy", None) is not None:
# redirect captured pipe back to original FD
os.dup2(std._original_stdstream_copy, __std__.fileno())
std._original_stdstream_copy = None |
Thanks, that seems to do it. |
This issue is related to several other issues that have been reported recently, including #789, #786, #735, #737 and psf/black#2516 but they seem to have gone unanswered over the past few months in this issue tracker. They all revolve around the fact that upgrading from ipykernel 5.x to 6.x breaks users' ability to redirect output via stdout.
As a simple reproducible example:
With ipykernel 5.5.5 this code redirects
print
output to the computer’s terminal window from which JupyterLab was launched, rather than outputting it to the notebook in which the code was running. However, in ipykernel 6.x, this no longer works. Now the code above directs the output to both the terminal and the notebook. It seems new versions of ipykernel hijack stdout via the ipykernel.iostream.OutStream object and force output to the notebook, when you used to (and presumably should, since it's very useful) be able to output directly and only to the stdout of your choice, such as the terminal.If I downgrade ipykernel to 5.5.5 (from say 6.4.1), the old behavior is back and I am able to control where stdout goes. However, staying on v5.x is obviously not a sustainable solution. Is there a different technique in 6.x for users who need to control where stdout goes while working in an ipython Jupyter notebook? Or is this a bug that should be fixed in future ipykernel releases?
See also this topic on Jupyter discourse and the related issues linked above from @uldz @MarcoGorelli @ItamarShDev.
The text was updated successfully, but these errors were encountered: