From 34f12adfc14db14248237d387bf9fa0ea961786c Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Wed, 31 Jan 2024 16:59:50 -0500 Subject: [PATCH 01/17] Update README.md --- README.md | 88 +++---------------------------------------------------- 1 file changed, 4 insertions(+), 84 deletions(-) diff --git a/README.md b/README.md index 31e54759..45f7b0d8 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,9 @@ Developed by [CatalystNeuro](http://catalystneuro.com/). `pip install roiextractors` ## Usage: + +See [documentation](https://roiextractors.readthedocs.io/en/latest/) for details. + ### Supported file types: #### Imaging 1. HDF5 @@ -31,74 +34,8 @@ Developed by [CatalystNeuro](http://catalystneuro.com/). 45. Numpy (a data format for manual input of optical physiology data as various numpy datasets) #### Functionality: -Interconversion amongst the various data formats as well as conversion to the NWB format and back. - -#### Features: -1. SegmentationExtractor object: - * `seg_obj.get_channel_names()` : - List of optical channel names - * `seg_obj.get_num_channels()` : - Number of channels - * `seg_obj.get_movie_framesize()`: - (height, width) of raw movie - * `seg_obj.get_movie_location()`: - Location of storage of movie/tiff images - * `seg_obj.get_image_masks(self, roi_ids=None)`: - Image masks as (ht, wd, num_rois) with each value as the weight given during segmentation operation. - * `seg_obj.get_pixel_masks(roi_ids=None)`: - Get pixel masks as (total_pixels(ht*wid), no_rois) - * `seg_obj.get_traces(self, roi_ids=None, start_frame=None, end_frame=None)`: - df/F trace as (num_rois, num_frames) - * `seg_obj.get_sampling_frequency()`: - Sampling frequency of movie/df/F trace. - * `seg_obj.get_roi_locations()`: - Centroid pixel location of the ROI (Regions Of Interest) as (x,y). - * `seg_obj.get_num_rois()`: - Total number of ROIs after segmentation operation. - * `seg_obj.get_roi_ids()`: - Any integer tags associated with an ROI, defaults to `0:num_of_rois` - -#### SegmentationExtractor object creation: -```python -import roiextractors -import numpy as np +This package provides a common API for various optical imaging and segmentation formats streamline conversion and data analysis. -seg_obj_cnmfe = roiextractors.CnmfeSegmentationExtractor('cnmfe_filename.mat') # cnmfe -seg_obj_extract = roiextractors.ExtractSegmentationExtractor('extract_filename.mat') # extract -seg_obj_sima = roiextractors.SimaSegmentationExtractor('sima_filename.sima') # SIMA -seg_obj_numpy = roiextractors.NumpySegmentationExtractor( - filepath = 'path-to-file', - masks=np.random.rand(movie_size[0],movie_size[1],no_rois), - signal=np.random.randn(num_rois,num_frames), - roi_idx=np.random.randint(no_rois,size=[1,no_rois]), - no_of_channels=None, - summary_image=None, - channel_names=['Blue'], -) # Numpy object -seg_obj_nwb = roiextractors.NwbSegmentationExtractor( - filepath_of_nwb, optical_channel_name=None, # optical channel to extract and store info from - imaging_plane_name=None, image_series_name=None, # imaging plane to extract and store data from - processing_module_name=None, - neuron_roi_response_series_name=None, # roi_response_series name to extract and store data from - background_roi_response_series_name=None) # nwb object -``` -#### Data format conversion: SegmentationExtractor to NWB: -```python -roiextractors.NwbSegmentationExtractor.write_segmentation(seg_obj, saveloc, - propertydict=[{'name': 'ROI feature 1, - 'description': 'additional attribute of each ROI', - 'data': np.random.rand(1,no_rois), - 'id': seg_obj.get_roi_ids()}, - {'name': 'ROI feature 2, - 'description': 'additional attribute of each ROI', - 'data': np.random.rand(1,no_rois), - 'id': seg_obj.get_roi_ids()}], - nwbfile_kwargs={'session_description': 'nwbfiledesc', - 'experimenter': 'experimenter name', - 'lab': 'test lab', - 'session_id': 'test sess id'}, - emission_lambda=400.0, excitation_lambda=500.0) -``` ## Example Datasets: Example datasets are maintained at https://gin.g-node.org/CatalystNeuro/ophys_testing_data. @@ -116,23 +53,6 @@ gin get-content To update data later, `cd` into the test directory and run `gin get-content` - -## Class descriptions: - -* **SegmentationExtractor:** An abstract class that contains all the meta-data and output data from the ROI segmentation operation when applied to the pre-processed data. It also contains methods to read from and write to various data formats ouput from the processing pipelines like SIMA, CaImAn, Suite2p, CNNM-E. - -* **NumpySegmentationExtractor:** Contains all data coming from a file format for which there is currently no support. To construct this, all data must be entered manually as arguments. - -* **CnmfeSegmentationExtractor:** This class inherits from the SegmentationExtractor class, having all its funtionality specifically applied to the dataset output from the 'CNMF-E' ROI segmentation method. - -* **ExtractSegmentationExtractor:** This class inherits from the SegmentationExtractor class, having all its funtionality specifically applied to the dataset output from the 'EXTRACT' ROI segmentation method. - -* **SimaSegmentationExtractor:** This class inherits from the SegmentationExtractor class, having all its funtionality specifically applied to the dataset output from the 'SIMA' ROI segmentation method. - -* **NwbSegmentationExtractor:** Extracts data from the NWB data format. Also implements a static method to write any format specific object to NWB. - -* **Suite2PSegmentationExtractor:** Extracts data from suite2p format. - ## Troubleshooting ##### Installing SIMA with python>=3.7: Will need a manual installation for package dependency **SIMA** since it does not currently support python 3.7: From 5a276dd5dd056a3e4efec8958ae3aec172f82d79 Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Wed, 31 Jan 2024 17:06:20 -0500 Subject: [PATCH 02/17] Update support list --- README.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 45f7b0d8..a488ef4e 100644 --- a/README.md +++ b/README.md @@ -21,20 +21,23 @@ See [documentation](https://roiextractors.readthedocs.io/en/latest/) for details ### Supported file types: #### Imaging 1. HDF5 -2. TIFF -3. STK -4. FLI -5. SBX +2. Miniscope +3. NWB +4. SBX +5. Micromanager +6. Bruker +7. ScanImage #### Segmentation 1. [calciumImagingAnalysis](https://github.com/bahanonu/calciumImagingAnalysis) (CNMF-E, EXTRACT) 2. [SIMA](http://www.losonczylab.org/sima/1.3.2/) 3. [NWB](https://pynwb.readthedocs.io/en/stable/) 4. [suite2p](https://github.com/MouseLand/suite2p) -45. Numpy (a data format for manual input of optical physiology data as various numpy datasets) +5. Numpy (a data format for manual input of optical physiology data as various numpy datasets) +6. [CaImAn](https://github.com/flatironinstitute/CaImAn) #### Functionality: -This package provides a common API for various optical imaging and segmentation formats streamline conversion and data analysis. +This package provides a common API for various optical imaging and segmentation formats to streamline conversion and data analysis. ## Example Datasets: Example datasets are maintained at https://gin.g-node.org/CatalystNeuro/ophys_testing_data. From bdca04c78ff771ed9e179d12060c14efb74409da Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Thu, 1 Feb 2024 16:29:20 -0500 Subject: [PATCH 03/17] Update README.md to mirror NeuroConv's format --- README.md | 83 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 54 insertions(+), 29 deletions(-) diff --git a/README.md b/README.md index a488ef4e..01943ebc 100644 --- a/README.md +++ b/README.md @@ -5,41 +5,62 @@ [![documentation](https://readthedocs.org/projects/roiextractors/badge/?version=latest)](https://roiextractors.readthedocs.io/en/latest/) [![License](https://img.shields.io/pypi/l/pynwb.svg)](https://github.com/catalystneuro/roiextractors/license.txt) -# ROI Extractors -Python-based module for extracting from, converting between, and handling recorded and optical imaging data from several file formats. Inspired by [SpikeExtractors](https://github.com/SpikeInterface/spikeextractors). +# ROIExtractors +

+

Automatically read optical imaging/segmentation data into a common API

+

+

+ Explore our documentation » +

-Developed by [CatalystNeuro](http://catalystneuro.com/). + -## Getting Started: -#### Installation: -`pip install roiextractors` +## Table of Contents -## Usage: +- [About](#about) +- [Installation](#installation) +- [Documentation](#documentation) +- [Funding](#funding) -See [documentation](https://roiextractors.readthedocs.io/en/latest/) for details. +## About -### Supported file types: -#### Imaging -1. HDF5 -2. Miniscope -3. NWB -4. SBX -5. Micromanager -6. Bruker -7. ScanImage +ROIExtractors provides a common API for various optical imaging and segmentation formats to streamline conversion and data analysis. ROI stands for Region Of Interest, which is the region in a set of acquired fluorescence images which the segmentation software has determined as a neuron. -#### Segmentation -1. [calciumImagingAnalysis](https://github.com/bahanonu/calciumImagingAnalysis) (CNMF-E, EXTRACT) -2. [SIMA](http://www.losonczylab.org/sima/1.3.2/) -3. [NWB](https://pynwb.readthedocs.io/en/stable/) -4. [suite2p](https://github.com/MouseLand/suite2p) -5. Numpy (a data format for manual input of optical physiology data as various numpy datasets) -6. [CaImAn](https://github.com/flatironinstitute/CaImAn) +Features: -#### Functionality: -This package provides a common API for various optical imaging and segmentation formats to streamline conversion and data analysis. +* Reads data from 10+ popular optical imaging and segmentation data formats into a common API. +* Extracts relevant metadata from each format. +* Provides a handy set of methods to analyze data such as `get_roi_locations()` -## Example Datasets: +## Installation + +To install the latest stable release of **roiextractors** though PyPI, type: +```shell +pip install roiextractors +``` + +For more flexibility we recommend installing the latest version directly from GitHub. The following commands create an environment with all the required dependencies and the latest updates: + +```shell +git clone https://github.com/catalystneuro/roiextractors +cd roiextractors +conda env create roiextractors_env +conda activate roiextractors_env +pip install -e . +``` +Note that this will install the package in [editable mode](https://pip.pypa.io/en/stable/cli/pip_install/#editable-installs). + +Finally, if you prefer to avoid `conda` altogether, the following commands provide a clean installation within the current environment: +```shell +pip install git+https://github.com/catalystneuro/roiextractors.git@main +``` + +## Documentation +See our [ReadTheDocs page](https://roiextractors.readthedocs.io/en/latest/) for full documentation, including a gallery of all supported formats. + +## Developer Guide + +### Example Datasets Example datasets are maintained at https://gin.g-node.org/CatalystNeuro/ophys_testing_data. To download test data on your machine, @@ -56,13 +77,17 @@ gin get-content To update data later, `cd` into the test directory and run `gin get-content` -## Troubleshooting +### Troubleshooting ##### Installing SIMA with python>=3.7: Will need a manual installation for package dependency **SIMA** since it does not currently support python 3.7: 1. Download SIMA wheels distribution [here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#sima). 2. `pip install ` 3. `pip install roiextractors` -### Funded by +## Funding +ROIExtractors is funded by * Stanford University as part of the Ripple U19 project (U19NS104590). * LBNL as part of the NWB U24 (U24NS120057). + +## License +ROIExtractors is distributed under the BSD3 License. See [LICENSE](https://github.com/catalystneuro/roiextractors/blob/main/LICENSE.txt) for more information. From bf5885ebde351475060340646b40cc8781da57d9 Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Thu, 1 Feb 2024 16:29:56 -0500 Subject: [PATCH 04/17] add license to toc --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 01943ebc..ffdc872a 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ - [Installation](#installation) - [Documentation](#documentation) - [Funding](#funding) +- [License](#license) ## About From 18206b7a8cb073b558f38ca091e063c4ed67f671 Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Thu, 1 Feb 2024 18:40:03 -0500 Subject: [PATCH 05/17] removed developer guide --- README.md | 26 -------------------------- 1 file changed, 26 deletions(-) diff --git a/README.md b/README.md index ffdc872a..d1f7b201 100644 --- a/README.md +++ b/README.md @@ -59,32 +59,6 @@ pip install git+https://github.com/catalystneuro/roiextractors.git@main ## Documentation See our [ReadTheDocs page](https://roiextractors.readthedocs.io/en/latest/) for full documentation, including a gallery of all supported formats. -## Developer Guide - -### Example Datasets -Example datasets are maintained at https://gin.g-node.org/CatalystNeuro/ophys_testing_data. - -To download test data on your machine, - -1. Install the gin client (instructions [here](https://gin.g-node.org/G-Node/Info/wiki/GIN+CLI+Setup#linux)) -2. Use gin to download data. -```shell -gin get CatalystNeuro/ophys_testing_data -cd ophys_testing_data -gin get-content -``` - -3. Change the file at `roiextractors/tests/gin_test_config.json` to point to the path of this test data - -To update data later, `cd` into the test directory and run `gin get-content` - -### Troubleshooting -##### Installing SIMA with python>=3.7: -Will need a manual installation for package dependency **SIMA** since it does not currently support python 3.7: -1. Download SIMA wheels distribution [here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#sima). -2. `pip install ` -3. `pip install roiextractors` - ## Funding ROIExtractors is funded by * Stanford University as part of the Ripple U19 project (U19NS104590). From dd2ccc5e3ae1d48a531e96cdde5fe9d80db3ddbe Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Thu, 1 Feb 2024 19:03:07 -0500 Subject: [PATCH 06/17] Update contribute.rst with developer guide info from the readme --- docs/source/contribute.rst | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/docs/source/contribute.rst b/docs/source/contribute.rst index edcd1122..22adaa6d 100644 --- a/docs/source/contribute.rst +++ b/docs/source/contribute.rst @@ -1,7 +1,38 @@ Contribute: =========== -To contribute to Roiextractors to help us develop an api to handle a new data format for optophysiology: +Example Datasets +---------------- + +Example datasets are maintained at https://gin.g-node.org/CatalystNeuro/ophys_testing_data. + +To download test data on your machine, + +1. Install the gin client (instructions `here `_) +2. Use gin to download data: + + .. code-block:: bash + + gin get CatalystNeuro/ophys_testing_data + cd ophys_testing_data + gin get-content + +3. Change the file at ``roiextractors/tests/gin_test_config.json`` to point to the path of this test data + +To update data later, ``cd`` into the test directory and run ``gin get-content`` + +Troubleshooting +--------------- + +Installing SIMA with python>=3.7: +________________________________ +Will need a manual installation for package dependency **SIMA** since it does not currently support python 3.7: + +1. Download SIMA wheels distribution `here `_. +2. `pip install ` +3. `pip install roiextractors` + +More details on how to construct individual extractors can be found here: .. toctree:: :maxdepth: 1 From 2aff36de922427045475f8cd4319309fba34aca9 Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Thu, 1 Feb 2024 19:14:13 -0500 Subject: [PATCH 07/17] Update supported formats to live table --- docs/source/compatible.rst | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-) diff --git a/docs/source/compatible.rst b/docs/source/compatible.rst index 80663ea3..ef436d3e 100644 --- a/docs/source/compatible.rst +++ b/docs/source/compatible.rst @@ -1,25 +1,8 @@ Supported formats ================= -Roiextractors supports various image formats obtained from different 2photon acquisition systems. -It also supports well known post processing cell extraction/segmentation pipelines. If you use any other format and would like to see it being supported we would love to know! (:doc:`Contact `) +The following is a live record of all the formats supported by ROIExtractors. -Imaging -------- -1. HDF5 -2. TIFF -3. STK -4. FLI +.. raw:: html -Segmentation ------------- -#. `calciumImagingAnalysis `_ (CNMF-E, EXTRACT) -#. `Caiman `_ -#. `SIMA `_ -#. `NWB `_ -#. `suite2p `_ -#. Numpy (for any currently un-supported format) - -Example Datasets ----------------- -Example datasets for each fo the file formats can be downloaded `here `_ + From faa5cb39d9b6c2409ef898dcce21d00c96973f92 Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Thu, 1 Feb 2024 19:16:50 -0500 Subject: [PATCH 08/17] specify ophys tab in support table link --- docs/source/compatible.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/compatible.rst b/docs/source/compatible.rst index ef436d3e..225664fd 100644 --- a/docs/source/compatible.rst +++ b/docs/source/compatible.rst @@ -1,8 +1,8 @@ Supported formats ================= -The following is a live record of all the formats supported by ROIExtractors. +The following is a live record of all the formats supported by ROIExtractors as well as other relevant ecosystem support. .. raw:: html - + From 607b9a44b243fae7a0dcfb72ab071c5989526e12 Mon Sep 17 00:00:00 2001 From: Paul Adkisson Date: Mon, 5 Feb 2024 13:21:12 -0500 Subject: [PATCH 09/17] rollback README.md --- README.md | 167 +++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 122 insertions(+), 45 deletions(-) diff --git a/README.md b/README.md index d1f7b201..31e54759 100644 --- a/README.md +++ b/README.md @@ -5,64 +5,141 @@ [![documentation](https://readthedocs.org/projects/roiextractors/badge/?version=latest)](https://roiextractors.readthedocs.io/en/latest/) [![License](https://img.shields.io/pypi/l/pynwb.svg)](https://github.com/catalystneuro/roiextractors/license.txt) -# ROIExtractors -

-

Automatically read optical imaging/segmentation data into a common API

-

-

- Explore our documentation » -

+# ROI Extractors +Python-based module for extracting from, converting between, and handling recorded and optical imaging data from several file formats. Inspired by [SpikeExtractors](https://github.com/SpikeInterface/spikeextractors). + +Developed by [CatalystNeuro](http://catalystneuro.com/). + +## Getting Started: +#### Installation: +`pip install roiextractors` + +## Usage: +### Supported file types: +#### Imaging +1. HDF5 +2. TIFF +3. STK +4. FLI +5. SBX + +#### Segmentation +1. [calciumImagingAnalysis](https://github.com/bahanonu/calciumImagingAnalysis) (CNMF-E, EXTRACT) +2. [SIMA](http://www.losonczylab.org/sima/1.3.2/) +3. [NWB](https://pynwb.readthedocs.io/en/stable/) +4. [suite2p](https://github.com/MouseLand/suite2p) +45. Numpy (a data format for manual input of optical physiology data as various numpy datasets) + +#### Functionality: +Interconversion amongst the various data formats as well as conversion to the NWB format and back. + +#### Features: +1. SegmentationExtractor object: + * `seg_obj.get_channel_names()` : + List of optical channel names + * `seg_obj.get_num_channels()` : + Number of channels + * `seg_obj.get_movie_framesize()`: + (height, width) of raw movie + * `seg_obj.get_movie_location()`: + Location of storage of movie/tiff images + * `seg_obj.get_image_masks(self, roi_ids=None)`: + Image masks as (ht, wd, num_rois) with each value as the weight given during segmentation operation. + * `seg_obj.get_pixel_masks(roi_ids=None)`: + Get pixel masks as (total_pixels(ht*wid), no_rois) + * `seg_obj.get_traces(self, roi_ids=None, start_frame=None, end_frame=None)`: + df/F trace as (num_rois, num_frames) + * `seg_obj.get_sampling_frequency()`: + Sampling frequency of movie/df/F trace. + * `seg_obj.get_roi_locations()`: + Centroid pixel location of the ROI (Regions Of Interest) as (x,y). + * `seg_obj.get_num_rois()`: + Total number of ROIs after segmentation operation. + * `seg_obj.get_roi_ids()`: + Any integer tags associated with an ROI, defaults to `0:num_of_rois` + +#### SegmentationExtractor object creation: +```python +import roiextractors +import numpy as np + +seg_obj_cnmfe = roiextractors.CnmfeSegmentationExtractor('cnmfe_filename.mat') # cnmfe +seg_obj_extract = roiextractors.ExtractSegmentationExtractor('extract_filename.mat') # extract +seg_obj_sima = roiextractors.SimaSegmentationExtractor('sima_filename.sima') # SIMA +seg_obj_numpy = roiextractors.NumpySegmentationExtractor( + filepath = 'path-to-file', + masks=np.random.rand(movie_size[0],movie_size[1],no_rois), + signal=np.random.randn(num_rois,num_frames), + roi_idx=np.random.randint(no_rois,size=[1,no_rois]), + no_of_channels=None, + summary_image=None, + channel_names=['Blue'], +) # Numpy object +seg_obj_nwb = roiextractors.NwbSegmentationExtractor( + filepath_of_nwb, optical_channel_name=None, # optical channel to extract and store info from + imaging_plane_name=None, image_series_name=None, # imaging plane to extract and store data from + processing_module_name=None, + neuron_roi_response_series_name=None, # roi_response_series name to extract and store data from + background_roi_response_series_name=None) # nwb object +``` +#### Data format conversion: SegmentationExtractor to NWB: +```python +roiextractors.NwbSegmentationExtractor.write_segmentation(seg_obj, saveloc, + propertydict=[{'name': 'ROI feature 1, + 'description': 'additional attribute of each ROI', + 'data': np.random.rand(1,no_rois), + 'id': seg_obj.get_roi_ids()}, + {'name': 'ROI feature 2, + 'description': 'additional attribute of each ROI', + 'data': np.random.rand(1,no_rois), + 'id': seg_obj.get_roi_ids()}], + nwbfile_kwargs={'session_description': 'nwbfiledesc', + 'experimenter': 'experimenter name', + 'lab': 'test lab', + 'session_id': 'test sess id'}, + emission_lambda=400.0, excitation_lambda=500.0) +``` +## Example Datasets: +Example datasets are maintained at https://gin.g-node.org/CatalystNeuro/ophys_testing_data. - +To download test data on your machine, -## Table of Contents +1. Install the gin client (instructions [here](https://gin.g-node.org/G-Node/Info/wiki/GIN+CLI+Setup#linux)) +2. Use gin to download data. +```shell +gin get CatalystNeuro/ophys_testing_data +cd ophys_testing_data +gin get-content +``` -- [About](#about) -- [Installation](#installation) -- [Documentation](#documentation) -- [Funding](#funding) -- [License](#license) +3. Change the file at `roiextractors/tests/gin_test_config.json` to point to the path of this test data -## About +To update data later, `cd` into the test directory and run `gin get-content` -ROIExtractors provides a common API for various optical imaging and segmentation formats to streamline conversion and data analysis. ROI stands for Region Of Interest, which is the region in a set of acquired fluorescence images which the segmentation software has determined as a neuron. -Features: +## Class descriptions: -* Reads data from 10+ popular optical imaging and segmentation data formats into a common API. -* Extracts relevant metadata from each format. -* Provides a handy set of methods to analyze data such as `get_roi_locations()` +* **SegmentationExtractor:** An abstract class that contains all the meta-data and output data from the ROI segmentation operation when applied to the pre-processed data. It also contains methods to read from and write to various data formats ouput from the processing pipelines like SIMA, CaImAn, Suite2p, CNNM-E. -## Installation +* **NumpySegmentationExtractor:** Contains all data coming from a file format for which there is currently no support. To construct this, all data must be entered manually as arguments. -To install the latest stable release of **roiextractors** though PyPI, type: -```shell -pip install roiextractors -``` +* **CnmfeSegmentationExtractor:** This class inherits from the SegmentationExtractor class, having all its funtionality specifically applied to the dataset output from the 'CNMF-E' ROI segmentation method. -For more flexibility we recommend installing the latest version directly from GitHub. The following commands create an environment with all the required dependencies and the latest updates: +* **ExtractSegmentationExtractor:** This class inherits from the SegmentationExtractor class, having all its funtionality specifically applied to the dataset output from the 'EXTRACT' ROI segmentation method. -```shell -git clone https://github.com/catalystneuro/roiextractors -cd roiextractors -conda env create roiextractors_env -conda activate roiextractors_env -pip install -e . -``` -Note that this will install the package in [editable mode](https://pip.pypa.io/en/stable/cli/pip_install/#editable-installs). +* **SimaSegmentationExtractor:** This class inherits from the SegmentationExtractor class, having all its funtionality specifically applied to the dataset output from the 'SIMA' ROI segmentation method. -Finally, if you prefer to avoid `conda` altogether, the following commands provide a clean installation within the current environment: -```shell -pip install git+https://github.com/catalystneuro/roiextractors.git@main -``` +* **NwbSegmentationExtractor:** Extracts data from the NWB data format. Also implements a static method to write any format specific object to NWB. + +* **Suite2PSegmentationExtractor:** Extracts data from suite2p format. -## Documentation -See our [ReadTheDocs page](https://roiextractors.readthedocs.io/en/latest/) for full documentation, including a gallery of all supported formats. +## Troubleshooting +##### Installing SIMA with python>=3.7: +Will need a manual installation for package dependency **SIMA** since it does not currently support python 3.7: +1. Download SIMA wheels distribution [here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#sima). +2. `pip install ` +3. `pip install roiextractors` -## Funding -ROIExtractors is funded by +### Funded by * Stanford University as part of the Ripple U19 project (U19NS104590). * LBNL as part of the NWB U24 (U24NS120057). - -## License -ROIExtractors is distributed under the BSD3 License. See [LICENSE](https://github.com/catalystneuro/roiextractors/blob/main/LICENSE.txt) for more information. From 4b974d4b164095c5b5644ccb8117076eba455a32 Mon Sep 17 00:00:00 2001 From: pauladkisson Date: Mon, 5 Feb 2024 13:31:19 -0800 Subject: [PATCH 10/17] updated conf to mirror neuroconvs --- docs/source/conf.py | 117 +++++++++++++++++++++++--------------------- 1 file changed, 62 insertions(+), 55 deletions(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index 36119d95..a26ddd1e 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -1,65 +1,72 @@ -# Configuration file for the Sphinx documentation builder. -# -# This file only contains a selection of the most common options. For a full -# list see the documentation: -# https://www.sphinx-doc.org/en/master/usage/configuration.html - -# -- Path setup -------------------------------------------------------------- - -# If extensions (or modules to document with autodoc) are in another directory, -# add these directories to sys.path here. If the directory is relative to the -# documentation root, use os.path.abspath to make it absolute, like shown here. -# -import os import sys +from pathlib import Path -sys.path.insert(0, os.path.abspath(".")) - - -# -- Project information ----------------------------------------------------- +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) project = "RoiExtractors" copyright = "2021, Saksham Sharda" author = "Saksham Sharda" -# The full version, including alpha/beta/rc tags -release = "0.2.1" - - -# -- General configuration --------------------------------------------------- - -# Add any Sphinx extension module names here, as strings. They can be -# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom -# ones. -extensions = [] - -# Add any paths that contain templates here, relative to this directory. -templates_path = ["_templates"] - -# List of patterns, relative to source directory, that match files and -# directories to ignore when looking for source files. -# This pattern also affects html_static_path and html_extra_path. -exclude_patterns = [] - -nbsphinx_execute_arguments = [ - "--InlineBackend.figure_formats={'svg', 'pdf'}", - "--InlineBackend.rc={'figure.dpi': 96}", +extensions = [ + "sphinx.ext.napoleon", # Support for NumPy and Google style docstrings + "sphinx.ext.autodoc", # Includes documentation from docstrings in docs/api + "sphinx.ext.autosummary", # Not clear. Please add if you know + "sphinx.ext.intersphinx", # Allows links to other sphinx project documentation sites + "sphinx_search.extension", # Allows for auto search function the documentation + "sphinx.ext.viewcode", # Shows source code in the documentation + "sphinx.ext.extlinks", # Allows to use shorter external links defined in the extlinks variable. ] -# -- Options for HTML output ------------------------------------------------- - -# The theme to use for HTML and HTML Help pages. See the documentation for -# a list of builtin themes. -# -try: - import sphinx_rtd_theme - html_theme = "sphinx_rtd_theme" - html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] -except ImportError: - print("RTD theme not installed, using default") - html_theme = "alabaster" - -# Add any paths that contain custom static files (such as style sheets) here, -# relative to this directory. They are copied after the builtin static files, -# so a file named "default.css" will overwrite the builtin "default.css". +templates_path = ["_templates"] +master_doc = "index" +exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"] +html_theme = "pydata_sphinx_theme" html_static_path = ["_static"] + +html_context = { + # "github_url": "https://github.com", # or your GitHub Enterprise site + "github_user": "catalystneuro", + "github_repo": "roiextractors", + "github_version": "main", + "doc_path": "docs", +} + +linkcheck_anchors = False + +# -------------------------------------------------- +# Extension configuration +# -------------------------------------------------- + +# Napoleon +napoleon_google_docstring = False +napoleon_numpy_docstring = True +napoleon_use_param = False +napoleon_use_ivar = True +napoleon_include_init_with_doc = False +napoleon_include_private_with_doc = True +napoleon_include_special_with_doc = True + +# Autodoc +autoclass_content = "both" # Concatenates docstring of the class with that of its __init__ +autodoc_member_order = "bysource" # Displays classes and methods by their order in source code +autodata_content = "both" +autodoc_default_options = { + "members": True, + "member-order": "bysource", + "private-members": True, + "show-inheritance": False, + "toctree": True, +} +add_module_names = False + +# Intersphinx +intersphinx_mapping = { + "hdmf": ("https://hdmf.readthedocs.io/en/stable/", None), + "pynwb": ("https://pynwb.readthedocs.io/en/stable/", None), + "spikeinterface": ("https://spikeinterface.readthedocs.io/en/latest/", None), +} + +# To shorten external links +extlinks = { + "nwbinspector": ("https://nwbinspector.readthedocs.io/en/dev/%s", ""), +} From ce828689cacc2b871c57372f2afa43ec56f22a9d Mon Sep 17 00:00:00 2001 From: pauladkisson Date: Mon, 5 Feb 2024 13:33:19 -0800 Subject: [PATCH 11/17] updated conf to mirror neuroconvs --- requirements-rtd.txt | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 requirements-rtd.txt diff --git a/requirements-rtd.txt b/requirements-rtd.txt new file mode 100644 index 00000000..9fdd2959 --- /dev/null +++ b/requirements-rtd.txt @@ -0,0 +1,8 @@ +Jinja2 +Sphinx +sphinx_rtd_theme +readthedocs-sphinx-search +sphinx-toggleprompt +sphinx-copybutton +roiextractors +pydata_sphinx_theme From 0dbc753ac4b9a0b7b6db4431263dfea3a7444d95 Mon Sep 17 00:00:00 2001 From: pauladkisson Date: Mon, 5 Feb 2024 13:35:51 -0800 Subject: [PATCH 12/17] removed old doc requirements --- docs/requirements_doc.txt | 16 ---------------- 1 file changed, 16 deletions(-) delete mode 100644 docs/requirements_doc.txt diff --git a/docs/requirements_doc.txt b/docs/requirements_doc.txt deleted file mode 100644 index 135cd2c2..00000000 --- a/docs/requirements_doc.txt +++ /dev/null @@ -1,16 +0,0 @@ -spikeextractors==0.9.0 -tqdm==4.48.2 -lazy_ops==0.2.0 -dill==0.3.2 -scipy==1.10.0 -PyYAML -h5py==2.9.0 -pynwb -nbsphinx -jupyter-sphinx -numpydoc -matplotlib -numpy -notebook -tifffile -git+https://github.com/catalystneuro/roiextractors From 9d2769c039cea68ee1792da5d1b244fee6d657c5 Mon Sep 17 00:00:00 2001 From: pauladkisson Date: Mon, 5 Feb 2024 13:39:06 -0800 Subject: [PATCH 13/17] updated readthedocs yaml to match modern example file --- .readthedocs.yml | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/.readthedocs.yml b/.readthedocs.yml index 619e29d1..50688d3a 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -1,12 +1,13 @@ +version: "2" + build: - image: latest + os: "ubuntu-22.04" + tools: + python: "3.10" -version: 2 +python: + install: + - requirements: requirements-rtd.txt sphinx: configuration: docs/source/conf.py - -python: - version: 3.8 - install: - - requirements: docs/requirements_doc.txt From c422fffd09d4b2e94b580c9308a3c78273892c60 Mon Sep 17 00:00:00 2001 From: pauladkisson Date: Mon, 5 Feb 2024 14:12:32 -0800 Subject: [PATCH 14/17] added API docs to toc --- docs/source/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/source/index.rst b/docs/source/index.rst index dc4bd2e0..8964b03e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -31,6 +31,7 @@ With this package, a user can: contribute licence contact + api Indices and tables ================== From bd229a56d73a6c98660b7b244f57a0727db7f484 Mon Sep 17 00:00:00 2001 From: pauladkisson Date: Mon, 5 Feb 2024 16:14:01 -0800 Subject: [PATCH 15/17] rollback API docs --- docs/source/index.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 8964b03e..dc4bd2e0 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -31,7 +31,6 @@ With this package, a user can: contribute licence contact - api Indices and tables ================== From 98fc9b5ae89a9178b6c61eb5d67ac244b768af51 Mon Sep 17 00:00:00 2001 From: pauladkisson Date: Thu, 8 Feb 2024 14:49:11 -0800 Subject: [PATCH 16/17] updated definition of roi --- docs/source/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index dc4bd2e0..e9025b36 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -10,7 +10,7 @@ Welcome to RoiExtractors's documentation! Roiextractors is a library that helps in analyzing, visualizing and interacting with optical physiology data acquired from various acquisition systems. **ROI** - Stands for Region Of Interest, which is the region in a set of acquired fluorescence images which the segmentation software has determined as a neuron. + Stands for Region Of Interest, which is the set of pixels from acquired fluorescence images which the segmentation software has determined to follow a particular cellular structure. With this package, a user can: From 39900880d104494707bafd92ff07dbcf40ed9d1e Mon Sep 17 00:00:00 2001 From: pauladkisson Date: Thu, 8 Feb 2024 15:13:36 -0800 Subject: [PATCH 17/17] removed unneccessary indices and tables from home page --- docs/source/index.rst | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index e9025b36..82f72a0b 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -20,9 +20,7 @@ With this package, a user can: .. toctree:: :maxdepth: 2 - :caption: Contents: - :glob: - :numbered: + :caption: Contents gettingstarted compatible @@ -31,10 +29,3 @@ With this package, a user can: contribute licence contact - -Indices and tables -================== - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search`