Merge main into branch (#705)

* Set legend anchor as parameter (#660) * Fix missing space * Remove explicit anndata in dependencies (#666) * Incorporate use case tutorials (#665) * Fixed DEG layer retrieval * Use-case tutorial icons * Restructure tutorial page * Subgroup tutorials * Improve KNN label_transfer in PerturbationSpace (#658) * Add uncertainty score in KNN label_transfer in PerturbationSpace Certainty is quantified as the fraction of nearest neighbors belonging to the classified (i.e. the most abundant) label compared to the total number of nearest neighbors. * Update pre-commit-config.yaml Replaces yanked dependency of mypy "types-pkg-resources" with "types-setuptools" as recommended: https://pypi.org/project/types-pkg-resources/ * Improve label imputation in PerturbationSpace class Key changes: - Now uses KNN graph in adata: saves cost and increases consistency - Vectorized operations instead of expensive for loop - Distance weighting for KNN imputation - Quantifies uncertainty as local KNN label entropy * Fixed plotting for mixscape.plot_barplot and sccoda.plot_effects_barplot (#667) * Augur scsim warnings (#670) * Augur scsim warnings Signed-off-by: zethson <lukas.heumos@posteo.net> * Submodules Signed-off-by: zethson <lukas.heumos@posteo.net> --------- Signed-off-by: zethson <lukas.heumos@posteo.net> * Add PerturbationDataValidator (#672) * Augur scsim warnings Signed-off-by: zethson <lukas.heumos@posteo.net> * Submodules Signed-off-by: zethson <lukas.heumos@posteo.net> * Add super draft of pertpy validator Signed-off-by: zethson <lukas.heumos@posteo.net> * Polish Signed-off-by: zethson <lukas.heumos@posteo.net> * Polish Signed-off-by: zethson <lukas.heumos@posteo.net> * Nested try Signed-off-by: zethson <lukas.heumos@posteo.net> * validator in test Signed-off-by: zethson <lukas.heumos@posteo.net> * try uv for rtd Signed-off-by: zethson <lukas.heumos@posteo.net> * rtd uv Signed-off-by: zethson <lukas.heumos@posteo.net> * rtd uv Signed-off-by: zethson <lukas.heumos@posteo.net> * rtd uv fix Signed-off-by: zethson <lukas.heumos@posteo.net> * mb sphinx fix for validator Signed-off-by: zethson <lukas.heumos@posteo.net> * docs Signed-off-by: zethson <lukas.heumos@posteo.net> * remove PerturbationValidator from docs Signed-off-by: zethson <lukas.heumos@posteo.net> * remove PerturbationValidator from docs Signed-off-by: zethson <lukas.heumos@posteo.net> --------- Signed-off-by: zethson <lukas.heumos@posteo.net> * Latest OS for RTD * Remove curator again Signed-off-by: zethson <lukas.heumos@posteo.net> * Fix jax random array (#686) * Fix jax random array Signed-off-by: zethson <lukas.heumos@posteo.net> * Fix further jax warnings Signed-off-by: zethson <lukas.heumos@posteo.net> * Fix edger Signed-off-by: zethson <lukas.heumos@posteo.net> * Fix choice Signed-off-by: zethson <lukas.heumos@posteo.net> --------- Signed-off-by: zethson <lukas.heumos@posteo.net> * Switch to formulaic-contrasts (#682) * Switch to formulaic-contrasts * Cleanup * removing design matrix workaround (#691) Co-authored-by: Emma Dann <emmadann@comino.stanford.edu> * Fix PyDESeq2 * Update tests * fix typo in gitignore * Remove contrast dataclass, which isnt used anywhere * Fix edgeR rpy2 tests (#692) * fix broken rpy2 edger tests * updated edger tests * Fix tests (scipy) Signed-off-by: zethson <lukas.heumos@posteo.net> * submodule Signed-off-by: zethson <lukas.heumos@posteo.net> * Remove unused code Signed-off-by: zethson <lukas.heumos@posteo.net> * type hints Signed-off-by: zethson <lukas.heumos@posteo.net> --------- Signed-off-by: zethson <lukas.heumos@posteo.net> Co-authored-by: Emma Dann <32264060+emdann@users.noreply.github.com> Co-authored-by: Emma Dann <emmadann@comino.stanford.edu> Co-authored-by: zethson <lukas.heumos@posteo.net> * Release 0.9.5 Signed-off-by: zethson <lukas.heumos@posteo.net> * Prepare 0.10.0 Signed-off-by: zethson <lukas.heumos@posteo.net> * Added Mixscape seeds and test (#683) Co-authored-by: Lukas Heumos <lukas.heumos@posteo.net> * Fix probability data type (#696) Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * Optimize MeanVarDistributionDistance (#697) * Fix probability data type Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * Optimize mean_var distance Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> --------- Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * Optimize test speed (#699) * Try buildjet Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * Try buildjet large Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * speed up predict_differential_prioritization Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * speed up tests Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> --------- Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * Lower bound for scikit-learn (#701) Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * Fix type annotation Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> * Fix empty figure returns when show=True in plotting functions (#703) * Removed show parameter * Adapt plotting API for Augur, Coda, Dialogue * Adapted plotting API for Milo, Mixscape, scgen * Add joblib * Remove joblib --------- Co-authored-by: Lukas Heumos <lukas.heumos@posteo.net> * Fix scikit-learn intendation Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> --------- Signed-off-by: zethson <lukas.heumos@posteo.net> Signed-off-by: Lukas Heumos <lukas.heumos@posteo.net> Co-authored-by: Lilly May <93096564+Lilly-May@users.noreply.github.com> Co-authored-by: Lukas Heumos <lukas.heumos@posteo.net> Co-authored-by: Gregor Sturm <mail@gregor-sturm.de> Co-authored-by: Emma Dann <32264060+emdann@users.noreply.github.com> Co-authored-by: Emma Dann <emmadann@comino.stanford.edu>
scverse · Feb 7, 2025 · 5593c0f · 5593c0f
1 parent 1ff7ce3
commit 5593c0f
Show file tree

Hide file tree

Showing 50 changed files with 585 additions and 926 deletions.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -4,9 +4,9 @@
 
 <!-- Please fill in the appropriate checklist below (delete whatever is not relevant). These are the most common things requested on pull requests (PRs). -->
 
--   [ ] Referenced issue is linked
--   [ ] If you've fixed a bug or added code that should be tested, add tests!
--   [ ] Documentation in `docs` is updated
+- [ ] Referenced issue is linked
+- [ ] If you've fixed a bug or added code that should be tested, add tests!
+- [ ] Documentation in `docs` is updated
 
 **Description of changes**
 

diff --git a/.github/release-drafter.yml b/.github/release-drafter.yml
@@ -1,5 +1,5 @@
-name-template: "0.9.4 🌈"
-tag-template: 0.9.4
+name-template: "0.10.0 🌈"
+tag-template: 0.10.0
 exclude-labels:
     - "skip-changelog"
 

diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -16,10 +16,10 @@ jobs:
         steps:
             - uses: actions/checkout@v4
 
-            - name: Set up Python 3.11
+            - name: Set up Python
               uses: actions/setup-python@v5
               with:
-                  python-version: "3.11"
+                  python-version: "3.12"
                   cache: "pip"
                   cache-dependency-path: "**/pyproject.toml"
 

diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -13,10 +13,10 @@ jobs:
             - name: Checkout code
               uses: actions/checkout@v4
 
-            - name: Set up Python 3.11
+            - name: Set up Python 3.12
               uses: actions/setup-python@v5
               with:
-                  python-version: "3.11"
+                  python-version: "3.12"
 
             - name: Install hatch
               run: pip install hatch

diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -22,14 +22,11 @@ jobs:
             matrix:
                 include:
                     - os: ubuntu-latest
-                      python: "3.10"
+                      python: "3.12"
                       run_mode: "slow"
                     - os: ubuntu-latest
                       python: "3.12"
-                      run_mode: "slow"
-                    # - os: ubuntu-latest
-                    #   python: "3.12"
-                    #   run_mode: "fast"
+                      run_mode: "fast"
                     # - os: ubuntu-latest
                     #   python: "3.12"
                     #   run_mode: slow

diff --git a/.gitignore b/.gitignore
@@ -137,6 +137,9 @@ dmypy.json
 # Jetbrains IDE
 .idea/
 
+# VSCode
+.vscode
+
 # Coala
 *.orig
 
@@ -155,3 +158,11 @@ lightning_logs/*
 */lightning_logs/*
 
 node_modules
+
+# lamindb
+test.ipynb
+test-perturbation
+test-bug
+
+# uv
+uv.lock
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -2,16 +2,16 @@ fail_fast: false
 default_language_version:
     python: python3
 default_stages:
-    - commit
-    - push
+    - pre-commit
+    - pre-push
 minimum_pre_commit_version: 2.16.0
 repos:
     - repo: https://github.com/pre-commit/mirrors-prettier
       rev: v4.0.0-alpha.8
       hooks:
           - id: prettier
     - repo: https://github.com/astral-sh/ruff-pre-commit
-      rev: v0.4.7
+      rev: v0.8.6
       hooks:
           - id: ruff
             args: [--fix, --exit-non-zero-on-fix, --unsafe-fixes]
@@ -27,7 +27,7 @@ repos:
           - id: trailing-whitespace
           - id: check-case-conflict
     - repo: https://github.com/pre-commit/mirrors-mypy
-      rev: v1.10.0
+      rev: v1.14.1
       hooks:
           - id: mypy
             args: [--no-strict-optional, --ignore-missing-imports]

diff --git a/.readthedocs.yml b/.readthedocs.yml
@@ -1,23 +1,18 @@
 version: 2
 build:
-    os: ubuntu-22.04
+    os: ubuntu-24.04
     tools:
-        python: "3.11"
-    jobs:
-        pre_build:
-            - python -c "import pertpy"
-            - pip freeze
+        python: "3.12"
+    commands:
+        - asdf plugin add uv
+        - asdf install uv latest
+        - asdf global uv latest
+        - uv venv
+        - uv pip install .[doc,coda,de]
+        - .venv/bin/python -m sphinx -T -b html -d docs/_build/doctrees -D language=en docs $READTHEDOCS_OUTPUT/html
 sphinx:
     configuration: docs/conf.py
     fail_on_warning: false
-python:
-    install:
-        - method: pip
-          path: .
-          extra_requirements:
-              - doc
-              - coda
-              - de
 
 submodules:
     include: all
diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md
@@ -14,23 +14,23 @@ religion, or sexual identity and orientation.
 Examples of behavior that contributes to creating a positive environment
 include:
 
--   Using welcoming and inclusive language
--   Being respectful of differing viewpoints and experiences
--   Gracefully accepting constructive criticism
--   Focusing on what is best for the community
--   Showing empathy towards other community members
+- Using welcoming and inclusive language
+- Being respectful of differing viewpoints and experiences
+- Gracefully accepting constructive criticism
+- Focusing on what is best for the community
+- Showing empathy towards other community members
 
 Examples of unacceptable behavior by participants include:
 
--   The use of sexualized language or imagery and unwelcome sexual
-    attention or advances
--   Trolling, insulting/derogatory comments, and personal or political
-    attacks
--   Public or private harassment
--   Publishing others’ private information, such as a physical or
-    electronic address, without explicit permission
--   Other conduct which could reasonably be considered inappropriate in a
-    professional setting
+- The use of sexualized language or imagery and unwelcome sexual
+  attention or advances
+- Trolling, insulting/derogatory comments, and personal or political
+  attacks
+- Public or private harassment
+- Publishing others’ private information, such as a physical or
+  electronic address, without explicit permission
+- Other conduct which could reasonably be considered inappropriate in a
+  professional setting
 
 ## Our Responsibilities
 

diff --git a/docs/_static/tutorials/mcfarland.png b/docs/_static/tutorials/mcfarland.png
diff --git a/docs/_static/tutorials/norman.png b/docs/_static/tutorials/norman.png
diff --git a/docs/_static/tutorials/zhang.png b/docs/_static/tutorials/zhang.png
diff --git a/docs/conf.py b/docs/conf.py
@@ -195,4 +195,7 @@
     "tutorials/notebooks/differential_gene_expression": "_static/tutorials/dge.png",
     "tutorials/notebooks/metadata_annotation": "_static/tutorials/metadata.png",
     "tutorials/notebooks/ontology_mapping": "_static/tutorials/ontology.png",
+    "tutorials/notebooks/norman_use_case": "_static/tutorials/norman.png",
+    "tutorials/notebooks/mcfarland_use_case": "_static/tutorials/mcfarland.png",
+    "tutorials/notebooks/zhang_use_case": "_static/tutorials/zhang.png",
 }
diff --git a/docs/contributing.md b/docs/contributing.md
@@ -132,11 +132,11 @@ in the cookiecutter-scverse template.
 
 Please write documentation for new or changed features and use-cases. This project uses [sphinx][] with the following features:
 
--   the [myst][] extension allows to write documentation in markdown/Markedly Structured Text
--   Google-style docstrings
--   Jupyter notebooks as tutorials through [myst-nb][] (See [Tutorials with myst-nb](#tutorials-with-myst-nb-and-jupyter-notebooks))
--   [Sphinx autodoc typehints][], to automatically reference annotated input and output types
--   Citations (like {cite:p}`Virshup_2023`) can be included with [sphinxcontrib-bibtex](https://sphinxcontrib-bibtex.readthedocs.io/)
+- the [myst][] extension allows to write documentation in markdown/Markedly Structured Text
+- Google-style docstrings
+- Jupyter notebooks as tutorials through [myst-nb][] (See [Tutorials with myst-nb](#tutorials-with-myst-nb-and-jupyter-notebooks))
+- [Sphinx autodoc typehints][], to automatically reference annotated input and output types
+- Citations (like {cite:p}`Virshup_2023`) can be included with [sphinxcontrib-bibtex](https://sphinxcontrib-bibtex.readthedocs.io/)
 
 See the [scanpy developer docs](https://scanpy.readthedocs.io/en/latest/dev/documentation.html) for more information
 on how to write documentation.
@@ -150,10 +150,10 @@ These notebooks come from [pert-tutorials](https://github.com/scverse/pertpy-tut
 
 #### Hints
 
--   If you refer to objects from other packages, please add an entry to `intersphinx_mapping` in `docs/conf.py`. Only
-    if you do so can sphinx automatically create a link to the external documentation.
--   If building the documentation fails because of a missing link that is outside your control, you can add an entry to
-    the `nitpick_ignore` list in `docs/conf.py`
+- If you refer to objects from other packages, please add an entry to `intersphinx_mapping` in `docs/conf.py`. Only
+  if you do so can sphinx automatically create a link to the external documentation.
+- If building the documentation fails because of a missing link that is outside your control, you can add an entry to
+  the `nitpick_ignore` list in `docs/conf.py`
 
 #### Building the docs locally
 

diff --git a/docs/index.md b/docs/index.md
@@ -54,13 +54,13 @@ Discussions <https://github.com/scverse/pertpy/discussions>
 references
 ```
 
--   Consider citing [scanpy Genome Biology (2018)] along with original {doc}`references <references>`.
--   A paper for pertpy is in the works.
+- Consider citing [scanpy Genome Biology (2018)] along with original {doc}`references <references>`.
+- A paper for pertpy is in the works.
 
 # Indices and tables
 
--   {ref}`genindex`
--   {ref}`modindex`
--   {ref}`search`
+- {ref}`genindex`
+- {ref}`modindex`
+- {ref}`search`
 
 [scanpy genome biology (2018)]: https://doi.org/10.1186/s13059-017-1382-0
diff --git a/docs/tutorials/index.md b/docs/tutorials/index.md
@@ -11,13 +11,25 @@ Many are also designed to work seamlessly in Google colab.
 For questions about the usage of pertpy use the [scverse discourse](https://discourse.scverse.org/).
 :::
 
-## Quick start
+## Quick start: Tool specific tutorials
+
+### Data transformation
 
 ```{eval-rst}
 .. nbgallery::
 
    notebooks/guide_rna_assignment
    notebooks/mixscape
+   notebooks/perturbation_space
+   notebooks/metadata_annotation
+   notebooks/ontology_mapping
+```
+
+### Knowledge inference
+
+```{eval-rst}
+.. nbgallery::
+
    notebooks/augur
    notebooks/sccoda
    notebooks/sccoda_extended
@@ -29,13 +41,24 @@ For questions about the usage of pertpy use the [scverse discourse](https://disc
    notebooks/distance_tests
    notebooks/cinemaot
    notebooks/scgen_perturbation_prediction
-   notebooks/perturbation_space
    notebooks/differential_gene_expression
-   notebooks/metadata_annotation
-   notebooks/ontology_mapping
 ```
 
-### Glossary
+## Use cases
+
+Our use cases showcase a variety of pertpy tools applied to one dataset.
+They are designed to give you a sense of how to use pertpy in a real-world scenario.
+The use cases featured here are those we present in the pertpy [preprint](https://www.biorxiv.org/content/10.1101/2024.08.04.606516v1).
+
+```{eval-rst}
+.. nbgallery::
+
+   notebooks/norman_use_case
+   notebooks/mcfarland_use_case
+   notebooks/zhang_use_case
+```
+
+## Glossary
 
 ```{eval-rst}
 .. tab-set::

diff --git a/docs/tutorials/notebooks b/docs/tutorials/notebooks
diff --git a/docs/usage/usage.md b/docs/usage/usage.md
@@ -563,33 +563,33 @@ including cell line annotation, bulk RNA and protein expression data.
 
 Available databases for cell line metadata:
 
--   [The Cancer Dependency Map Project at Broad](https://depmap.org/portal/)
--   [The Cancer Dependency Map Project at Sanger](https://depmap.sanger.ac.uk/)
--   [Genomics of Drug Sensitivity in Cancer (GDSC)](https://www.cancerrxgene.org/)
+- [The Cancer Dependency Map Project at Broad](https://depmap.org/portal/)
+- [The Cancer Dependency Map Project at Sanger](https://depmap.sanger.ac.uk/)
+- [Genomics of Drug Sensitivity in Cancer (GDSC)](https://www.cancerrxgene.org/)
 
 ### Compound
 
 The Compound module enables the retrieval of various types of information related to compounds of interest, including the most common synonym, pubchemID and canonical SMILES.
 
 Available databases for compound metadata:
 
--   [PubChem](https://pubchem.ncbi.nlm.nih.gov/)
+- [PubChem](https://pubchem.ncbi.nlm.nih.gov/)
 
 ### Mechanism of Action
 
 This module aims to retrieve metadata of mechanism of action studies related to perturbagens of interest, depending on the molecular targets.
 
 Available databases for mechanism of action metadata:
 
--   [CLUE](https://clue.io/)
+- [CLUE](https://clue.io/)
 
 ### Drug
 
 This module allows for the retrieval of Drug target information.
 
 Available databases for drug metadata:
 
--   [chembl](https://www.ebi.ac.uk/chembl/)
+- [chembl](https://www.ebi.ac.uk/chembl/)
 
 ```{eval-rst}
 .. autosummary::

diff --git a/pertpy/__init__.py b/pertpy/__init__.py
@@ -2,7 +2,7 @@
 
 __author__ = "Lukas Heumos"
 __email__ = "lukas.heumos@posteo.net"
-__version__ = "0.9.4"
+__version__ = "0.10.0"
 
 import warnings
 

diff --git a/pertpy/_doc.py b/pertpy/_doc.py
@@ -15,6 +15,5 @@ def dec(obj):
 
 
 doc_common_plot_args = """\
-show: if `True`, shows the plot.
-            return_fig: if `True`, returns figure of the plot.\
+return_fig: if `True`, returns figure of the plot, that can be used for saving.\
 """
diff --git a/pertpy/data/_datasets.py b/pertpy/data/_datasets.py
@@ -66,7 +66,7 @@ def sc_sim_augur() -> AnnData:  # pragma: no cover
     output_file_path = settings.datasetdir / output_file_name
     if not Path(output_file_path).exists():
         _download(
-            url="https://figshare.com/ndownloader/files/31645886",
+            url="https://figshare.com/ndownloader/files/49828902",
             output_file_name=output_file_name,
             output_path=settings.datasetdir,
             is_zip=False,
+1 −2		.pre-commit-config.yaml
+1 −1		augur.ipynb
+1,190 −226		differential_gene_expression.ipynb
+21 −62		guide_rna_assignment.ipynb
+1,178 −0		mcfarland_use_case.ipynb
+951 −0		norman_use_case.ipynb
+1,040 −0		zhang_use_case.ipynb