Add always-visible titles to the example gallery.

[rpgoldman]

Previously the only titles were in tooltips, which made it near impossible to scan the page and find the example you wanted.

Note that this is a WIP if only because there's an Easter Egg in one of the titles that needs fixing!

[ColCarroll]

I found the easter egg!

Is there a reason to not just use ex_title as the title? We could even start throwing exceptions or something if it wasn't found. I don't feel strongly either way.

[ColCarroll]

watch your language!

[rpgoldman]

I was hoping you knew what it was overlayed on...

[rpgoldman]

...and whether it's "overlayed" or "overlaid"!

[canyon289]

overlaid i believe

[OriolAbril]

I would write something like Plot LOO-PIT KDE overlaid on uniform distributions KDEs, but I find it quite hard to summarize that more that what is said in the Examples section of its API docs without loosing so much meaning it can only be understood if you already know what it means

[avehtari]

Can we assume someone using this would know what are CDF and inverse-CDF? Maybe Wikipedia's PIT article helps https://en.wikipedia.org/wiki/Probability_integral_transform ?

[rpgoldman]

@avehtari Probably we can make that assumption, in which case we can give this a title that explains it sufficiently for such a person.

Plot LOO overlaid on uniform distributions to assess convergence.

Would that be good?

This does beg the question -- if a person does not know the purpose of this plot, where should they look? API docs? Examples? Or do we need a tutorial of some form?

[OriolAbril]

I would say that ideally they would look at the API docs (each example already has a link to the relevant function) if necessary they can go to other resources from there, reference papers, it's page in the educational materials repo (which does not exist yet) and other relevant links that may be present.

[rpgoldman]

I put in the above suggested title -- "Plot LOO overlaid on uniform distributions to assess convergence." Marking this as resolved.

[avehtari]

1. It's missing a word: Plot LOO "what" overlaid on...
   LOO can mean many things, and the confusion could that some would think it's LOO elpds. If you think that probability integral transformation is not well known, how about "LOO predictive ECDF" ?
2. It's not about convergence, but about model checking. So the full sentence could be
   "Plot LOO predictive ECDF overlaid on ECDFs of uniform samples to assess predictive calibration"

[rpgoldman]

I found the easter egg!

Is there a reason to not just use ex_title as the title? We could even start throwing exceptions or something if it wasn't found. I don't feel strongly either way.

I put changes in ex_title for a couple of reasons:

1. It lets us put something different on the thumbnail for people who are searching for a particular plot.
2. It also lets us have longer titles, if we want, that wouldn't fit on the examples page.

[codecov]

Codecov Report

Merging #1484 (7dec9fd) into master (ae8a613) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1484   +/-   ##
=======================================
  Coverage   91.92%   91.92%           
=======================================
  Files         105      105           
  Lines       11252    11252           
=======================================
  Hits        10343    10343           
  Misses        909      909           

Impacted Files	Coverage Δ
arviz/plots/khatplot.py	94.87% <ø> (ø)
arviz/plots/separationplot.py	73.68% <ø> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ae8a613...7dec9fd. Read the comment docs.

[rpgoldman]

One thing that would be nice to fix before merging: separation plot.

1. the only description of the separation plot cited is from an article hidden behind a paywall.
2. the thumbnail is just a wall of blue.

[rpgoldman]

I just pushed a new version with a better-centered matplotlib separation plot.

[rpgoldman]

Calling this good enough and clearing the WIP flag.

[rpgoldman]

The only one I'm not sure about is the change to mpl_plot_ess_local -- the reason I added the phrase that you took out was that the term "local" was not clear. What makes an expected sample size local?

[rpgoldman]

Other than that one point, these all seem like clear improvements.

[rpgoldman]

These test failures aren't due to this MR.

[rpgoldman]

Last thing, if we have the energy, would be to apply these changes to the Bokeh figures, as well.

[OriolAbril]

What makes an expected sample size local?

The quantile ess is calculated with P(theta < theta_quantile) (see equation 16 in the reference) whereas the local ess is calculated with P(theta_quantile < theta < theta_quantile+delta) (eq 19). From the paper:

(...) This gives us a local efficiency measure which is more localized than efficiency measure for quantiles and can be used to build intuition about what types of posterior functionals can be computed as illustrated in the examples. While the expectation of a function that only depends on intermediate values can be usually estimated with relative ease, expectations of tail probabilities or other posterior functionals that depend critically on the tail of the distribution will be usually more difficult to estimate. In addition, small probability intervals can be used in practical equivalence testing

reference: https://arxiv.org/pdf/1903.08008.pdf

if we have the energy, would be to apply these changes to the Bokeh figures

I would wait for that, I want to try simplifying the example gallery using tabs like https://sphinx-panels.readthedocs.io/en/latest/#tabbed-content. After all, the title is the same, the api reference is the same, and it would allow having a single thumbnail and avoid generating bokeh thumbnails which is the main complication when generating the docs and requires weird dependencies.

[rpgoldman]

In that case, I think we are ready to merge.