From 61a1dc57759ced6344657308bbe522fd29c3798d Mon Sep 17 00:00:00 2001 From: Biplov Bhandari Date: Mon, 29 Apr 2024 13:03:42 -0400 Subject: [PATCH] update paper add notebook figures (WIP) update docs --- docs/installation.md | 48 +++++++++++++++++- docs/usage.md | 9 ++-- .../Book_Chapter_1.png} | Bin .../Book_Cover.png | Bin .../{notebookfigures => images}/U_Net.png | Bin .../sampling_approach.png | Bin paper/paper.bib | 26 ++++------ paper/paper.md | 10 ++-- 8 files changed, 67 insertions(+), 26 deletions(-) rename notebook/{notebookfigures/Book_Chap1.png => images/Book_Chapter_1.png} (100%) rename notebook/{notebookfigures => images}/Book_Cover.png (100%) rename notebook/{notebookfigures => images}/U_Net.png (100%) rename notebook/{notebookfigures => images}/sampling_approach.png (100%) diff --git a/docs/installation.md b/docs/installation.md index a454881..493111d 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1 +1,47 @@ -## Coming soon \ No newline at end of file +# Installation + +## Earth Engine Account + +Google Earth Engine (GEE) is used for data processing to get it ready in a usable TF Format. You can [sign up](https://earthengine.google.com/signup/) for a [Google Earth Engine](https://earthengine.google.com/) account. + +![signup](https://i.imgur.com/ng0FzUT.png) + +## Install from PyPI + +**servir-aces** is available on [PyPI](https://pypi.org/project/servir-aces/). To install **servir-aces**, run this command in your terminal: + +```bash +pip install servir-aces +``` + + +The optional dependencies can be installed using one of the following: + +- `pip install "servir-aces[extra]"`: installing all optional dependencies. + +## Install from GitHub + +To install the development version from GitHub using [Git](https://git-scm.com/), run the following command in your terminal: + +```bash +pip install git+https://github.com/SERVIR/servir-aces +``` diff --git a/docs/usage.md b/docs/usage.md index 5b800bc..3e841c8 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -1,13 +1,16 @@ # Usage -To use servir-aces in a project: +To use servir-aces in a project. -``` +```python from aces.config import Config from aces.model_trainer import ModelTrainer if __name__ == "__main__": - config = Config() + config_file = "config.env" + config = Config(config_file) trainer = ModelTrainer(config) trainer.train_model() ``` + +Several notebooks are also available on the `notebook` folder implementing DL methods. diff --git a/notebook/notebookfigures/Book_Chap1.png b/notebook/images/Book_Chapter_1.png similarity index 100% rename from notebook/notebookfigures/Book_Chap1.png rename to notebook/images/Book_Chapter_1.png diff --git a/notebook/notebookfigures/Book_Cover.png b/notebook/images/Book_Cover.png similarity index 100% rename from notebook/notebookfigures/Book_Cover.png rename to notebook/images/Book_Cover.png diff --git a/notebook/notebookfigures/U_Net.png b/notebook/images/U_Net.png similarity index 100% rename from notebook/notebookfigures/U_Net.png rename to notebook/images/U_Net.png diff --git a/notebook/notebookfigures/sampling_approach.png b/notebook/images/sampling_approach.png similarity index 100% rename from notebook/notebookfigures/sampling_approach.png rename to notebook/images/sampling_approach.png diff --git a/paper/paper.bib b/paper/paper.bib index 9e9f323..8bbfb44 100644 --- a/paper/paper.bib +++ b/paper/paper.bib @@ -1,19 +1,11 @@ @article{Gorelick2017, - title = "{Google Earth Engine: Planetary-scale geospatial analysis for - everyone}", - author = "Gorelick, Noel and Hancher, Matt and Dixon, Mike and - Ilyushchenko, Simon and Thau, David and Moore, Rebecca", - journal = "Remote Sensing of Environment", - publisher = "Elsevier", - volume = 202, - pages = "18--27", - month = dec, - year = 2017, - url = "http://www.sciencedirect.com/science/article/pii/S0034425717302900", - keywords = "Cloud computing; Big data; Analysis; Platform; Data - democratization; Earth Engine", - issn = "0034-4257", - doi = "10.1016/j.rse.2017.06.031" + title={Google Earth Engine: Planetary-scale geospatial analysis for everyone}, + author={Gorelick, Noel and Hancher, Matt and Dixon, Mike and Ilyushchenko, Simon and Thau, David and Moore, Rebecca}, + journal={Remote sensing of Environment}, + volume={202}, + pages={18--27}, + year={2017}, + publisher={Elsevier} } @article{Mayer2021, @@ -29,7 +21,7 @@ @article{Mayer2021 @article{Mayer2023, title={Employing the agricultural classification and estimation service (ACES) for mapping smallholder rice farms in Bhutan}, author={Mayer, Timothy and Bhandari, Biplov and Mart{\'\i}nez, Filoteo G{\'o}mez and Walker, Kaitlin and Jim{\'e}nez, Stephanie A and Kruskopf, Meryl and Maganini, Micky and Phalke, Aparna and Wangchen, Tshering and Phuntsho, Loday and others}, - journal={ISPRS Open Journal of Photogrammetry and Remote Sensing}, + journal={Frontiers in Environmental Science}, volume={11}, pages={1137835}, year={2023}, @@ -39,7 +31,7 @@ @article{Mayer2023 @article{Bhandari2024, title={Comparing Deep Learning Models for Rice Mapping in Bhutan using High Resolution Satellite Imagery}, author={Bhandari, Biplov and Mayer, Timothy}, - journal={Frontiers in Environmental Science}, + journal={ISPRS Open Journal of Photogrammetry and Remote Sensing}, volume={to appear}, pages={to appear}, year={2024}, diff --git a/paper/paper.md b/paper/paper.md index fb60ddf..7a23426 100644 --- a/paper/paper.md +++ b/paper/paper.md @@ -27,9 +27,9 @@ bibliography: paper.bib # Summary -**`servir-aces`** Agricultural Classification and Estimation Service (ACES) is a Python package for generating training data using highly parallelized [apache-beam](https://beam.apache.org/) and [Google Earth Engine (GEE)](https://earthengine.google.com/) [@Gorelick2017] workflows as well as for training various Machine Learning (ML) and Deep Learning (DL) models for Remote Sensing Applications [@Mayer2023], [@Bhandari2024]. With petabytes of data available via GEE, and integration of the TensorFlow (TF) platfrom, models trained in TF can be easily loaded into GEE. This package provides functionalities for 1) data processing; 2) data loading from GEE; 3) feature extraction, and 4) model training. The use of combining TF and GEE for has enabled several large scale ML and DL Remote Sensing applications. Some of them include Wetland Area Mapping [@Bakkestuen2023], Crop Type Mapping [@Poortinga2021], Surface Water Mapping [@Mayer2021], and Urban Mapping [@Parekh2021]. +**`servir-aces`** Agricultural Classification and Estimation Service (ACES) is a Python package for generating training data using highly parallelized [apache-beam](https://beam.apache.org/) and [Google Earth Engine (GEE)](https://earthengine.google.com/) [@Gorelick2017] workflows as well as for training various Machine Learning (ML) and Deep Learning (DL) models for Remote Sensing Applications [@Mayer2023], [@Bhandari2024]. With petabytes of data available via GEE, and integration of the TensorFlow (TF) platfrom, models trained in TF can be easily loaded into GEE. This package provides functionalities for 1) data processing; 2) data loading from GEE; 3) feature extraction, 4) model training, and 5) model inference. The combination of TF and GEE has enabled several large scale ML and DL Remote Sensing applications. Some of them include Wetland Area Mapping [@Bakkestuen2023], Crop Type Mapping [@Poortinga2021], Surface Water Mapping [@Mayer2021], and Urban Mapping [@Parekh2021]. -Despite robust platforms, specialized technical knowledge is required to setup and run various ML/DL models. Leading many practitioners, scientists, and domain experts to find it difficult to implement DL frameworks. The **`servir-aces`** Python package is created to fill this gap. **`servir-aces`** siginicantly lowers the barrier for users to export training data and both train and run DL models using cloud-based technology with their GEE workflows. Several examples are provided to make it easier for scientists utilize this emerging field of DL. +Despite robust platforms, specialized technical knowledge is required to setup and run various ML/DL models, leading many practitioners, scientists, and domain experts to find it difficult to implement them. The **`servir-aces`** Python package is created to fill this gap. **`servir-aces`** siginicantly lowers the barrier for users to export training data and both train and run DL models using cloud-based technology with their GEE workflows. Several examples are provided via a runnable notebook to make it easier for scientists utilize this emerging field of DL. # **`servir-aces`** Audience @@ -43,12 +43,13 @@ The major high-level functionality of the **servir-aces** packages are: - Data loading and processing from GEE. - Generation of training data for various ML and DL models. - Training and evaluation of ML/DL Models. +- Inferences of the trained ML/DL models. - Support for remote sensing feature extraction. - Integration with Apache Beam for data processing and parallelization. The key functionality of **`servir-aces`** is organized into several modules: -- [data_processor](https://servir.github.io/servir-aces/data_processor/): this module provides functions for data input/output and preprocessing for the image segmentation project. +- [data_processor](https://servir.github.io/servir-aces/data_processor/): this module provides functionality for data input/output and preprocessing for the image segmentation project. - [model_builder](https://servir.github.io/servir-aces/model_builder/): this module provides functionality for creating and compiling various Neural Network Models, including DNN, CNN, U-Net. @@ -64,7 +65,6 @@ The key functionality of **`servir-aces`** is organized into several modules: This research was funded through the US Agency for International Development (USAID) and NASA initiative Cooperative Agreement Number: AID486-A-14-00002. Individuals affiliated with the University of Alabama in Huntsville (UAH) are funded through the NASA Applied Sciences Capacity Building Program, NASA Cooperative Agreement: 80MSFC22M004. # **`servir-aces`** Acknowledgement -The authors would like to thank NASA’s Applied Sciences Program and Capacity Building Program, specifically Dr. Nancy Searby. We also want to thank the [SERVIR](https://servirglobal.net/) program Dan Irwin, Dr. Ashutosh Limaye, Eric Anderson in particular. Additionally, we would like to thank the the United States Agency for International Development (USAID) especially Dr. Pete Epanchin. We would also like to thank the University of Alabama in Huntsville specifically Dr. Rob Griffin and the [Lab for Applied Science (LAS)]([https://www.uah.edu/essc/laboratory-for-applied-science](https://www.uah.edu/essc/laboratory-for-applied-science)). As well as [SERVIR's Geospatial Artificial Intelleigence Working Group (Geo-AI WG)](https://sites.google.com/uah.edu/2020-tensorflow-technical-exch/home) for their support and collaboration over the years. Finally, Nick Clinton of the Google Earth Outreach Team for the support. - +The authors would like to thank NASA’s Applied Sciences Program and Capacity Building Program, specifically Dr. Nancy Searby. We also want to thank the [SERVIR](https://servirglobal.net/) program especially Dan Irwin, Dr. Ashutosh Limaye, and Eric Anderson. Additionally, we would like to thank the USAID especially Dr. Pete Epanchin. We would also like to thank UAH specifically Dr. Rob Griffin and the [Lab for Applied Science (LAS)](https://www.uah.edu/essc/laboratory-for-applied-science) as well as [SERVIR's Geospatial Artificial Intelleigence Working Group (Geo-AI WG)](https://tinyurl.com/servir-geo-ai-wg) for their support and collaboration over the years. Finally, wse are indebted to Dr Nick Clinton from the Google Earth Outreach Team for the support. # References