Merge pull request #39 from rajewsky-lab/docs_enhance

docs: improve example data tutorials & fix typos

docs/computational/pairwise_alignment.md

@@ -22,9 +22,9 @@ single [h5ad] file containing all the [barcoded tiles](preprocessing_sequencing.
 (see [Preprocessing of sequencing](preprocessing_sequencing.md)). 
 
 !!! warning
-    Remember that spacemake generates one file per [barcoded tile], 
-    and it will be necessary to perform [stitching of tiles] to
-    obtain a single file with all tiles for a sample.
+    Remember that spacemake generates one file per [barcoded tile](preprocessing_sequencing.md#flow-cell-related-terms), 
+    and it will be necessary to perform [stitching of tiles](preprocessing_sequencing.md) to
+    obtain a single file with all tiles for a sample. This is done with `openst spatial_stitch`.
 
 ### Image modality
 The expected input for the image modality is a single tiff file of the whole tile-scan.
@@ -100,6 +100,20 @@ alignment [yourself](#manual-workflow)), you can specify the argument `--only-co
 
     These allow to increase the number of matches and possibly the number of RANSAC inliers. There are other RANSAC-specific parameters that can be changed, such as `--ransac-coarse-residual-threshold`, `--ransac-coarse-max-trials` and `--ransac-fine-min-samples`. For more parameters, check `openst pairwise_aligner --help`.
 
+!!! warning
+    In some environments, the following error might happen:
+
+    ```sh
+    qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/user/miniconda3/envs/openst/lib/python3.11/site-packages/cv2/qt/plugins" even though it was found.
+
+    This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
+
+    Available platform plugins are: xcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl.
+    ```
+
+    This can be solved by removing the files inside `/home/user/miniconda3/envs/openst/lib/python3.11/site-packages/cv2/qt/plugins`, e.g. see this [StackOverflow thread](https://stackoverflow.com/questions/71029300/python-pyqt-xcb-plugin-with-thread-error)
+
+
 ### Visual assessment with report
 You can generate an HTML report that contains a qualitative summary of the alignment (images, parameters...)
 

docs/examples/adult_mouse/pairwise_alignment.md

@@ -21,17 +21,30 @@ in an environment different to the one used for `spacemake`).
 1. You are in the `/home/user/openst_adult_demo` folder (or similar, depending on your system, username...)
 
 ```sh
+mkdir alignment
+
+cp projects/openst_demo/processed_data/openst_demo_adult_mouse/illumina/complete_data/dge/dge.all.polyA_adapter_trimmed.mm_included.spatial_beads_puck_collection.h5ad alignment/spatial_beads_adult_mouse.h5ad
+
+# download the image data
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/adult_mouse_hippocampus.tif"
+
+openst merge_modalities \
+    --h5-in alignment/spatial_beads_adult_mouse.h5ad \
+    --image-in adult_mouse_hippocampus.tif
+
 openst pairwise_aligner \
-    --image-in alignment/adult_mouse_hippocampus.tif \
-    --h5-in projects/openst_demo/processed_data/openst_demo_adult_mouse/illumina/complete_data/dge/dge.all.polyA_adapter_trimmed.mm_included.spatial_beads_puck_collection.h5ad \
-    --h5-out alignment/openst_demo_adult_mouse_spatial_beads_puck_collection_aligned.h5ad \
-    --save-image-in-h5 \
+    --h5-in alignment/spatial_beads_adult_mouse.h5ad \
     --only-coarse \
     --device cuda
 ```
 
-This will create the file `alignment/openst_demo_adult_mouse_spatial_beads_puck_collection_aligned.h5ad`, that hopefully contains a
-proper coarse alignment of the two modalities (at low resolution).
+This will create the file `alignment/spatial_beads_adult_mouse.h5ad`, that hopefully contains a proper coarse alignment of the two modalities (at low resolution). 
+
+!!! note
+    For this image data, and the spatial coordinates after alignment, the conversion factor to physical distance is **1 pixel = 0.345 µm**.
+
+!!! tip
+    From `openst>=0.0.7`, there is `openst from_spacemake` to populate the data input/output arguments automatically, and keep the processing inside the `spacemake` folder structure. You can read more in the [tutorial](../../computational/from_spacemake.md).
 
 ## Fine alignment (manual)
 
@@ -44,7 +57,7 @@ openst manual_pairwise_aligner
 
 Then, follow these instructions on the GUI, by pressing the *buttons*:
 
-1. Click on *Load h5ad* and browse to the location where the `openst_demo_adult_mouse_spatial_beads_puck_collection_aligned.h5ad` file
+1. Click on *Load h5ad* and browse to the location where the `spatial_beads_adult_mouse.h5ad` file
 is located. Select it and click *Open*. 
 2. Go to *Data properties*, then click under *Image data*, and select from the tree `uns>spatial_pairwise_aligned>staining_image_transformed`. Click *Ok*.
 3. Click under *Spatial coordinates*, and select from the tree `obsm/spatial_pairwise_aligned_coarse`. Click *Ok*.
@@ -67,11 +80,11 @@ coarse alignment, and the keypoints file, to perform the fine alignment:
 ```sh
 openst apply_transform \
     --keypoints-in alignment/openst_adult_demo_fine_keypoints.json \
-    --h5-in alignment/openst_demo_adult_mouse_spatial_beads_puck_collection_aligned.h5ad \
+    --h5-in alignment/spatial_beads_adult_mouse.h5ad \
     --per-tile
 ```
 
 After this, no file will be created nor removed; the coordinates of the fine alignment will be added to the existing
-`openst_demo_adult_mouse_spatial_beads_puck_collection_aligned.h5ad` file.
+`spatial_beads_adult_mouse.h5ad` file.
 
 That's it! Now you're ready to go to the next step.

docs/examples/adult_mouse/preprocessing_sequencing.md

@@ -75,20 +75,30 @@ we are going to omit `(spacemake) user@computer:/home/user/openst_adult_demo/spa
 spacemake config add_species \
    --name mouse \
    --reference genome \
-   --sequence <path_to_genome.fa> \
-   --annotation <path_to_genome_annotation.gtf>
+   --sequence GRCm39vM30.genome.fa \
+   --annotation gencodevM30.annotation.gtf
 
 spacemake config add_species \
    --name mouse \
    --reference rRNA \
-   --sequence <path_to_rRNA_sequences.fa>
+   --sequence mouse.rRNA.fa
 
 spacemake config add_species \
    --name mouse \
    --reference phiX \
-   --sequence <path_to_phiX_genome.fa>
+   --sequence phiX.fa
 ```
 
+!!! note
+    The `.fa` and `.gtf` files for mouse are available for http download under the [example datasets](../datasets.md) page.
+    For instance, you can run:
+
+    ```sh
+    # for the mouse genome sequence
+    wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/genomes/GRCm39vM30.genome.fa"
+    # etc...
+    ```
+
 ### Adding sample
 
 Now you need to add the sample data and metadata to `spacemake`. For this, you will also need the puck (tile) barcode files, which [can be
@@ -101,19 +111,27 @@ When downloading the tile barcode files, create a folder under `openst_adult_dem
 into this folder. Also, move the coordinate file to the `puck_data` folder in the `spacemake` directory.
 
 Remember! You need to be in the `/home/user/openst_adult_demo/spacemake` directory (or similar, depending on what you created);
-then run the following command:
+then run the following commands:
 
 ```sh
+# downloading R1 and R2 sequences
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/adult_mouse_hippocampus_R1_001.fastq.gz"
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/adult_mouse_hippocampus_R2_001.fastq.gz"
+
+# downloading the spatial barcode sequences
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/adult_hippocampus_tiles.tar.xz"
+tar -xvf adult_hippocampus_tiles.tar.xz
+
 spacemake projects add_sample \
     --project_id openst_demo \
     --sample_id openst_demo_adult_mouse \
-    --R1 <path_to_R1.fastq.gz> \
-    --R2 <path_to_R2.fastq.gz> \
+    --R1 adult_mouse_hippocampus_R1_001.fastq.gz \
+    --R2 adult_mouse_hippocampus_R2_001.fastq.gz \
     --species mouse \
     --puck openst \
     --run_mode openst \
     --barcode_flavor openst \
-    --puck_barcode_file tiles/*.txt.gz \
+    --puck_barcode_file adult_hippocampus_tiles/*.txt.gz \
     --map_strategy "bowtie2:phiX->bowtie2:rRNA->STAR:genome:final"
 ```
 
@@ -122,7 +140,7 @@ after you run the `spacemake init` command (see above). Modify the following lin
 
 ```yaml
 openst:
-    coordinate_system: puck_id/openst_coordinates.csv
+    coordinate_system: puck_data/openst_coordinates.csv
     spot_diameter_um: 0.6
     width_um: 1200
 ```
@@ -131,11 +149,19 @@ into this:
 
 ```yaml
 openst:
-    coordinate_system: puck_id/openst_demo_adult_brain_coordinate_system.csv
+    coordinate_system: puck_data/fc_2_coordinate_system.csv
     spot_diameter_um: 0.6
     width_um: 1200
 ```
 
+You can download the coordinate system file from the Open-ST website, for example:
+
+```sh
+# download the coordinate system
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/fc_2_coordinate_system.csv"
+cp fc_2_coordinate_system.csv puck_data/.
+```
+
 ### Running `spacemake`
 That's all you need to configure! Now, you can run spacemake with the following:
 

docs/examples/e13_mouse/pairwise_alignment.md

@@ -21,17 +21,30 @@ in an environment different to the one used for `spacemake`).
 1. You are in the `/home/user/openst_e13_demo` folder (or similar, depending on your system, username...)
 
 ```sh
+mkdir alignment
+
+cp projects/openst_demo/processed_data/openst_demo_e13_mouse_head/illumina/complete_data/dge/dge.all.polyA_adapter_trimmed.mm_included.spatial_beads_puck_collection.h5ad alignment/spatial_beads_e13_mouse_head.h5ad
+
+# download the image data
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/e13_mouse_head.tif"
+
+openst merge_modalities \
+    --h5-in alignment/spatial_beads_e13_mouse_head.h5ad \
+    --image-in e13_mouse_head.tif
+
 openst pairwise_aligner \
-    --image-in alignment/e13_mouse_head.tif \
-    --h5-in projects/openst_demo/processed_data/openst_demo_e13_mouse_head/illumina/complete_data/dge/dge.all.polyA_adapter_trimmed.mm_included.spatial_beads_puck_collection.h5ad \
-    --h5-out alignment/openst_demo_e13_mouse_head_spatial_beads_puck_collection_aligned.h5ad \
-    --save-image-in-h5 \
+    --h5-in alignment/spatial_beads_e13_mouse_head.h5ad \
     --only-coarse \
     --device cuda
 ```
 
-This will create the file `alignment/openst_demo_e13_mouse_head_spatial_beads_puck_collection_aligned.h5ad`, that hopefully contains a
-proper coarse alignment of the two modalities (at low resolution).
+This will create the file `alignment/spatial_beads_e13_mouse_head.h5ad`, that hopefully contains a proper coarse alignment of the two modalities (at low resolution).
+
+!!! note
+    For this image data, and the spatial coordinates after alignment, the conversion factor to physical distance is **1 pixel = 0.345 µm**.
+
+!!! tip
+    From `openst>=0.0.7`, there is `openst from_spacemake` to populate the data input/output arguments automatically, and keep the processing inside the `spacemake` folder structure. You can read more in the [tutorial](../../computational/from_spacemake.md).
 
 ## Fine alignment (manual)
 
@@ -44,7 +57,7 @@ openst manual_pairwise_aligner
 
 Then, follow these instructions on the GUI, by pressing the *buttons*:
 
-1. Click on *Load h5ad* and browse to the location where the `openst_demo_e13_mouse_head_spatial_beads_puck_collection_aligned.h5ad` file
+1. Click on *Load h5ad* and browse to the location where the `spatial_beads_e13_mouse_head.h5ad` file
 is located. Select it and click *Open*. 
 2. Go to *Data properties*, then click under *Image data*, and select from the tree `uns>spatial_pairwise_aligned>staining_image_transformed`. Click *Ok*.
 3. Click under *Spatial coordinates*, and select from the tree `obsm/spatial_pairwise_aligned_coarse`. Click *Ok*.
@@ -67,11 +80,11 @@ coarse alignment, and the keypoints file, to perform the fine alignment:
 ```sh
 openst apply_transform \
     --keypoints-in alignment/openst_e13_demo_fine_keypoints.json \
-    --h5-in alignment/openst_demo_e13_mouse_head_spatial_beads_puck_collection_aligned.h5ad \
+    --h5-in alignment/spatial_beads_e13_mouse_head.h5ad \
     --per-tile
 ```
 
 After this, no file will be created nor removed; the coordinates of the fine alignment will be added to the existing
-`openst_demo_e13_mouse_head_spatial_beads_puck_collection_aligned.h5ad` file.
+`spatial_beads_e13_mouse_head.h5ad` file.
 
 That's it! Now you're ready to go to the next step.

docs/examples/e13_mouse/preprocessing_sequencing.md

@@ -75,20 +75,30 @@ we are going to omit `(spacemake) user@computer:/home/user/openst_e13_mouse_head
 spacemake config add_species \
    --name mouse \
    --reference genome \
-   --sequence <path_to_genome.fa> \
-   --annotation <path_to_genome_annotation.gtf>
+   --sequence GRCm39vM30.genome.fa \
+   --annotation gencodevM30.annotation.gtf
 
 spacemake config add_species \
    --name mouse \
    --reference rRNA \
-   --sequence <path_to_rRNA_sequences.fa>
+   --sequence mouse.rRNA.fa
 
 spacemake config add_species \
    --name mouse \
    --reference phiX \
-   --sequence <path_to_phiX_genome.fa>
+   --sequence phiX.fa
 ```
 
+!!! note
+    The `.fa` and `.gtf` files for mouse are available for http download under the [example datasets](../datasets.md) page.
+    For instance, you can run:
+
+    ```sh
+    # for the mouse genome sequence
+    wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/genomes/GRCm39vM30.genome.fa"
+    # etc...
+    ```
+
 ### Adding sample
 
 Now you need to add the sample data and metadata to `spacemake`. For this, you will also need the puck (tile) barcode files, which [can be
@@ -104,16 +114,28 @@ Remember! You need to be in the `/home/user/openst_e13_mouse_head_demo/spacemake
 then run the following command:
 
 ```sh
+# downloading R1 and R2 sequences
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/e13_mouse_head_R1_001.fastq.gz"
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/e13_mouse_head_R2_001.fastq.gz"
+
+# downloading R1 and R2 (reseq) sequences
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/e13_mouse_head_reseq_R1_001.fastq.gz"
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/e13_mouse_head_reseq_R2_001.fastq.gz"
+
+# downloading the spatial barcode sequences
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/e13_mouse_head_tiles.tar.xz"
+tar -xvf e13_mouse_head_tiles.tar.xz
+
 spacemake projects add_sample \
     --project_id openst_demo \
     --sample_id openst_demo_e13_mouse_head_mouse \
-    --R1 <path_to_R1.fastq.gz> \
-    --R2 <path_to_R2.fastq.gz> \
+    --R1 e13_mouse_head_R1_001.fastq.gz e13_mouse_head_reseq_R1_001.fastq.gz \
+    --R2 e13_mouse_head_R2_001.fastq.gz e13_mouse_head_reseq_R2_001.fastq.gz \
     --species mouse \
     --puck openst \
     --run_mode openst \
     --barcode_flavor openst \
-    --puck_barcode_file tiles/*.txt.gz \
+    --puck_barcode_file e13_mouse_head_tiles/*.txt.gz \
     --map_strategy "bowtie2:phiX->bowtie2:rRNA->STAR:genome:final"
 ```
 
@@ -131,11 +153,19 @@ into this:
 
 ```yaml
 openst:
-    coordinate_system: puck_id/openst_demo_e13_mouse_head_brain_coordinate_system.csv
+    coordinate_system: puck_id/fc_1_coordinate_system.csv
     spot_diameter_um: 0.6
     width_um: 1200
 ```
 
+You can download the coordinate system file from the Open-ST website, for example:
+
+```sh
+# download the coordinate system
+wget "http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/fc_1_coordinate_system.csv"
+cp fc_1_coordinate_system.csv puck_data/.
+```
+
 ### Running `spacemake`
 That's all you need to configure! Now, you can run spacemake with the following:
 

docs/faq.md

@@ -129,7 +129,7 @@ Shallow sequencing can always be performed first to assess general library quali
 
 
 ## Pairwise alignment
-[__The fiducial marks cannot be detected/are not visible)__](#fiducial-alignment){ #fiducial-alignment }
+[__The fiducial marks cannot be detected/are not visible__](#fiducial-alignment){ #fiducial-alignment }
 
 Sometimes, fiducial marks might not be visible when imaging thick tissue (we have noticed this with > 10 µm thickness) or under areas with high cellular density. Thus, automatic coarse alignment will work, but the fine alignment might fail, as the model cannot find these markers in the image. 
 
@@ -139,12 +139,12 @@ In the latter case, we cannot ensure that the alignment accuracy will lead to su
 
 [__In manual alignment mode, how many fiducials/features should I select per tile?__](#n-features-fiducial){ #n-features-fiducial }
 
-Given that a rgiid transformation model is estimated from the selected pairs of keypoints, we recommend at least 2 points. The more corresponding points are selected, the better.
+Given that a rigid transformation model is estimated from the selected pairs of keypoints, we recommend at least 2 points. The more corresponding points are selected, the better.
 
 ## Image segmentation
 
 [__The segmentation did not perform well__](#segmentation-model){ #segmentation-model }
 
-We provide an interface to the default, pre-trained cellpose models, as well as our fine-tuned openst_he model. We have tested this on a wide diversity of tissues, but it is possible that different microscopy setups and imaged tissues deliver different segmentation performance. 
+We provide an interface to the default, pre-trained cellpose models, as well as our fine-tuned [HE_cellpose_rajewsky](http://bimsbstatic.mdc-berlin.de/rajewsky/openst-public-data/models/HE_cellpose_rajewsky) model. We have tested this on a wide diversity of tissues, but it is possible that different microscopy setups and imaged tissues deliver different segmentation performance. 
 
 Especially, tissues with higher cellular densities and lower contrast between background/nuclei (or cells) might perform worse. Thus, we recommend referring to the [cellpose tutorial](https://cellpose.readthedocs.io/en/latest/gui.html#training-your-own-cellpose-model) on how to train your own model.