proof of principle example using napari with full 3D interaction in jupyter notebook

[tlambert03]

Description

I won't have time to follow up on this example, but I thought some here might be interested to see it... If anyone wants to follow up, feel free to take this code and clean it up, otherwise fine to close it.

This draft shows the minimal changes that would need to be made to napari in order to be compatible with @almarklein's clever https://github.com/vispy/jupyter_rfb ... allowing fully interactive 3D reprs in a notebook (using a remote frame buffer).

Mostly, it means not enforcing that vispy uses at 'qt' app (allowing the end-user to use `use_app('jupyter_rfb'), and then cleaning up the various places where the vispy canvas backend abstraction has leaked into napari's code.

example notebook in the examples folder (note, it requires pip install jupyter_rfb 'ipywidgets<8' until vispy/jupyter_rfb#58 is solved)

Untitled.mov

[codecov]

Codecov Report

Merging #5090 (7823394) into main (beaa98e) will decrease coverage by 0.01%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##             main    #5090      +/-   ##
==========================================
- Coverage   87.63%   87.61%   -0.02%     
==========================================
  Files         587      587              
  Lines       50015    50037      +22     
==========================================
+ Hits        43830    43841      +11     
- Misses       6185     6196      +11     

Impacted Files	Coverage Δ
napari/viewer.py	81.81% <20.00%> (-6.19%)	⬇️
napari/_vispy/__init__.py	88.88% <60.00%> (-11.12%)	⬇️
napari/_vispy/canvas.py	89.47% <80.00%> (-2.37%)	⬇️
napari/_qt/qt_main_window.py	75.46% <100.00%> (+0.04%)	⬆️
napari/_qt/widgets/qt_welcome.py	86.81% <100.00%> (+0.14%)	⬆️
napari/_vispy/overlays/axes.py	78.48% <100.00%> (+0.27%)	⬆️
napari/_vispy/overlays/scale_bar.py	97.16% <100.00%> (+0.04%)	⬆️
napari/components/experimental/chunk/_pool.py	85.71% <0.00%> (-7.94%)	⬇️
napari/utils/theme.py	91.15% <0.00%> (+0.68%)	⬆️
napari/utils/info.py	83.33% <0.00%> (+1.11%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[github-actions]

Click here to download the docs artifacts
docs
(zip file)

[jni]

Wow. That's super super cool. ❤️ Makes me think we probably want to prioritise building a declarative GUI with both Qt and ipywidgets front-end...

[Czaki]

Did I good understand that vispy set backend as a global state, and it is impossible to have jupyter widget and napari window in one interpreter session?

[brisvag]

Wow, this is amazing! I tried going down this road before, but I got lost quickly and thought this was gonna be a lot more work. I can't believe it works with so few changes!

@Czaki yeah I think so, I remember that being one of the disappointing things about it. I' m not sure if we can change vispy to allow swapping backend.

[andy-sweet]

Very impressive! I think this has been open long enough to get some exposure/traction and there are some conflicts with main, so I'm going to close this to reduce the number of open PRs. Feel free to reopen if the work continues.