Use pyfair inside Jupyter notebooks

[priamai]

Hi there,
fantastic project but would like to see how one could embed the single chart components inside a jupyter notebook instead of producing the HTML page.

[theonaunheim]

Thanks for your interest. Tentative yes.

I'll take a look this weekend and see if I can cobble together a sustainable solution.

[priamai]

Thanks a lot, this is working in Jupyter, I will post a notebook example as a starting point:

from pyfair import FairModel, FairSimpleReport

# Create our model
model = FairModel(name='Basic Model', n_simulations=10_000)

# Add normally distributed data
model.input_data('Loss Event Frequency', mean=.3, stdev=.1)

# Add constant data
model.input_data('Loss Magnitude', constant=5_000_000)

# We could hypothetically do BetaPert data
# model.input_data('Loss Magnitude', low=0, mode=10, high=100, gamma=90)

# Run our simulations
model.calculate_all()

# Export results (if desired)
results = model.export_results()

# Create a report and write it to an output.
fsr = FairSimpleReport([model])

from pyfair.report.distribution import FairDistributionCurve
dc = FairDistributionCurve(model) 
fig,ax = dc.generate_icon(model_name='Basic Model',target='Loss Event Frequency')

fig,ax = dc.generate_image()

[theonaunheim]

Perfect, thank you.

I think this is a common enough use case that I'm leaning towards creating a new class. Haven't thought about all the implications yet, but I think this probably would make usage less painful.

shim = FairJupyterShim([model])
fig1, ax1 = shim.get_tree()
fig2, ax2 = shim.get_violins()

[priamai]

I like that yes!

[priamai]

I would say also add a few parameters such as figsize, legends could be helfpul.

[theonaunheim]

@priamai ... thanks again for the idea. This is definitely do-able and will be included in the next release.

Two questions:

1. What do you want to do with the legend?
2. Are you ever going to use the axes from the (fig, ax) return types? I'm trying to figure out how to structure the output without making it to complicated.

scratch.pdf

[priamai]

Yes sorry ignore my previous suggestion, having the figs and axes will be perfect as I will be able to change title and sizes for example.
Your example code is perfect indeed.

[theonaunheim]

@priamai , at your convenience would you check the newest Docker Hub image and see if the shim works as intended?

[priamai]

okay testing now!

[priamai]

@theonaunheim from inside the jupyter environment I tried this:

Let me know how should I use the new classes.

[theonaunheim]

Thanks for your help. I forgot to mention that I changed "get_tree()" to "get_trees()" so all the method names reflected that they took iterables as inputs.

The method names are now:
get_trees()
get_distributions()
get_exceedence_curves()
get_violins()

[priamai]

Hi @theonaunheim tested it again

Do I have to install ipympl?

[priamai]

I also tried:

import matplotlib.pyplot as plt
%matplotlib ipympl
shim = FairJupyterShim([model])
trees = shim.get_trees()
(fig,ax)=trees['Basic Model']

then I get same error.

[theonaunheim]

Hmm ... I will try debugging this tomorrow. I am not entirely sure what the problem is, but I should be able to sort it out by Monday and provide feedback.

[theonaunheim]

Hey, @priamai . Apologies for the delay--last weekend was a total wash.

I went ahead and included pip ipympl and the notebook plugin build in the docker image. This means that you can use the 'widget', 'inline', and 'ipympl'.

'notebook' doesn't work for me for reasons I do not fully understand. I suspect that it has something to do with this in the base jupyter-scipy-notebook image:
RUN MPLBACKEND=Agg python -c "import matplotlib.pyplot" && \ fix-permissions "/home/${NB_USER}

scratch (5).pdf

[priamai]

Hi @theonaunheim,
yes this is confusing, is it worth to open an issue on their github maybe?
Cheers.

[theonaunheim]

Edit: TL;DR: this is an upstream WONTFIX problem. You can use "%matplotlib notebook" by getting rid of "-e JUPYTER_ENABLE_LAB=yes" from the Docker run command, which will start Jupyter Notebook instead of Jupyter lab. Alternatively, you can get similar functionality in Jupyter Lab using "%matplotlib widget". Also, I've learned that you always want to import pyfair AFTER you've set the backend.

---

Hey, @priamai ... in the process of putting together a bug report I found the problem. Spoiler alert: I was the problem.

A couple of intersecting causes:

1. My run instructions specifically include a "JUPYTER_ENABLE_LAB=yes" environment variable that forces you to run Jupyter Lab.
2. Jupyter Lab does not support "%matplotlib notebook" as this uses a backend (vanilla 'backend_nbagg') that is not present/compatible.
3. Pyplot should always be imported AFTER the backend has been set. Because pyfair imports pyplot, pyfair also needs to be imported after the backend is set. You're probably safest restarting your kernel if you need to change backends.

So, if you want to use "%matplotlib notebook" you will need to remove the "-e JUPYTER_ENABLE_LAB=yes" from the run instructions and you can use "%matplotlib notebook" as usual. Take note to only import pyfair AFTER you set the backend. If you want to use interactive images in Jupyter Lab, you will need to use "%matplotlib widget" (ipympl backend).

The "MPLBACKEND=Agg" environment variable was a canard ... we're overriding this at runtime anyway. I think the problem is with "%matplotlib notebook" in Jupyter Lab is we send to a backend that that doesn't have a defined interface.

Working examples below:

• Juptyer Notebook (%matplotlib notebook)
• Jupyter Notebook (%matplotlib inline)
• Juptyer Lab (%matplotlib widget)
• Jupyter Lab (%matplotlib inline)

lab_matplotlib_inline.pdf
lab_matplotlib_widget.pdf
notebook_matplotlib_inline.pdf
notebook_matplotlib_notebook.pdf