Suggestions for DE analysis

[grst]

1. use decoupler

I think a few sections of the DE section could be cleaned up by using decoupler functions instead of custom ones:

• decoupler already has a function to compute pseudobulk: get_pseudobulk
• decoupler also has a function to make volcano plots: plot_volcano.
  

2. additional visualization ideas

I have used some additional approaches for visualizing DE results in the past. Let me know if you are interested:

• paired or unpaired scatter+boxplots are nice ways of visualizing individual genes (each dot is a pseudobulk-sample)

• barcharts with scatterplot for paired observations (bar = mean fold change, dots = fold changes for all paired observations) is a nice way of visualzing top N results.
  

[Zethson]

Hey @grst

1. we've also started using more decoupler in other chapters and I fully agree that it'd make sense if we also adopted it here.
2. Those are some pretty visualizations. I'm always happy if we don't have to add a lot of complex plotting code so I'm wondering how complex the implementation of those (in particular the gene specific ones) are?

Cheers

[grst]

Those are some pretty visualizations. I'm always happy if we don't have to add a lot of complex plotting code so I'm wondering how complex the implementation of those (in particular the gene specific ones) are?

yeah, well, they are not exactly compact: https://github.com/icbi-lab/nmp-liver/blob/d3f0823dfd014a1d64b860a54320682405c4b032/lib/scanpy_helper/scanpy_helpers/pairwise.py

The pairwise function could probably be reduced to mostly these seaborn calls, but even that is quite a few lines.

Should probably think of putting those into some package at some point. Maybe in decoupler? Maybe there's even room for a dedicated package for DE visualizations 🤔. Also relevant: scverse/scanpy#1832.

[Zethson]

I'll CC @PauBadiaM here

[PauBadiaM]

Interesting discussion. In decoupler I had to add some plotting functions like the volcano one due to a lack of better alternatives, happy to see its useful! I feel like scverse is lacking a decoupler like framework (a package including many different tools) but for differential testing, then the plotting functions related to DEA could be there. Since the main DE testing frameworks are quite stable nowadays (limma, deseq2, edgeR, etc) it should be relatively feasible to port them into python. Although it would be a huge task, some people are starting to work on this, for example with deseq2 (which sparked some controversy).

[Zethson]

Maybe there's even room for a dedicated package for DE visualizations thinking

It could go into https://github.com/theislab/pertpy/ but I'm not sure whether I'd like adding plotting functions without having a corresponding tool?

What do you guys think?

[PauBadiaM]

What about https://github.com/theislab/diffxpy?

[Zethson]

It's currently being rewritten, but it's not something that I consider stable or maintained enough to be used for this purpose.

[grst]

We are in touch with the pyDESeq2 developers: owkin/PyDESeq2#15

Ideally, in the future that package will "just work" with scverse data structures, rather than having a meta package that implements a wrapper. That still leaves the question what's the place for such plots.

RE pertpy: I'm not quite sure I understood the scope of that package. If DE methods are not the scope of this package, it indeed feels a bit weird if DE plotting functions are.

I could also start something like scviz and put some functions there... If there appears a better place for them later they can still be moved. But @Zethson already has something like that (https://github.com/schillerlab/sc-toolbox)...

[Zethson]

RE pertpy: I'm not quite sure I understood the scope of that package. If DE methods are not the scope of this package, it indeed feels a bit weird if DE plotting functions are.

All things perturbation analysis. DE is of course the simplest approach and the foundation of perturbation analysis. It is in scope of pertpy technically. However, we don't want to reimplement DE methods because doing that well is a project itself (see diffxpy or the recent pydeseq2 attempts).
Another pro argument for adding them to pertpy (especially in the context of the book) is that pertpy is (will be shortly) used all over the place in the book.

I could also start something like scviz and put some functions there... If there appears a better place for them later they can still be moved. But @Zethson already has something like that (https://github.com/schillerlab/sc-toolbox)...

sc-toolbox is dead and unmaintained. It was basically just a quick way for me to make a couple of methods available. I don't think that starting scviz with a bunch of random helper visualizations would be the way to go (for now). It would be yet another project to maintain and I don't have the impression that scverse is ready yet to design a complete scviz package for all plotting functions.

At the moment adding them to pertpy sounds best to me although it might not be the perfect solution.

WDYT?

[PauBadiaM]

Sounds good! It can always then be moved to a more DE specialized library.

[grst]

However, we don't want to reimplement DE methods because doing that well is a project itself (see diffxpy or the recent pydeseq2 attempts).

Ok, but maybe there's room for wrappers for DE methods as @PauBadiaM suggested (e.g. consistent interface to PyDESeq2, diffxpy, scanpy wilcoxon test) in the future?

In any case, if you are comfortable with adding the viz functions there, that's fine for me!

[Zethson]

However, we don't want to reimplement DE methods because doing that well is a project itself (see diffxpy or the recent pydeseq2 attempts).

Ok, but maybe there's room for wrappers for DE methods as @PauBadiaM suggested (e.g. consistent interface to PyDESeq2, diffxpy, scanpy wilcoxon test) in the future?

Yes! For sure. I think that we could draw some inspiration from your helper methods for this? I'll create an issue at pertpy soon and will ping you so that we could outline this. I could likely organize the implementation then.

[grst]

yeah, let's discuss this elsewhere. this is not the issue for it.

[Zethson]

So take away from this issue is:

More decoupler - yes.
More/better visualizations - yes, but we'll implement them externally first and then add them to the DE notebook.

Thank you for the feedback again @grst !