source/notebooks/sagemaker_predictive_maintenance.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import zipfile\n",
    "\n",
    "!bash setup.sh\n",
    "\n",
    "import sagemaker\n",
    "from sagemaker_predictive_maintenance import config, preprocess, utils\n",
    "\n",
    "role = config.role"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation\n",
    "Data gathering, storing, denoising, alignment, curating and querying is THE most complicated aspect of predictive maintenance. Fault reports can for example be buried in pdf scans, fault types being misdiagnosed, fault only discovered weeks after they actually occured, or reported in a different timezone without the timezone information. These are a few of a thousands of pitfalls that may\n",
    "await you when trying to collate such real-world dataset at scale. In this scenario we are using a hypothetical dataset where the data is cleaned and aligned.\n",
    "\n",
    "\n",
    "### Background\n",
    "NASA’s Prognostic Center of Excellence established a repository with datasets to be used for benchmarking prognostics and predictive maintenance related algorithms. Among these datasets involves data from a turbofan engine simulation model C-MAPPS (or Commercial Modular Aero Propulsion System Simulation). The references section contains details about the over 100 publications using this dataset. C-MAPPS is a tool used to generate health, control and engine parameters from a simulated turbofan engine. A custom code wrapper was used to inject synthetic faults and continuous degradation trends into a time series of sensor data. Some high level characteristics of this dataset are as follows:\n",
    "The data obtained is from a high fidelity simulation of a turbofan engine, but closely models the sensor values of an actual engine. Synthetic noise was added to the dataset to replicate real-world scenarios. The effects of faults are masked due to operational conditions, which is a common trait of most real world systems."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "source_data_location = 's3://sagemaker-solutions-us-west-2/Predictive-maintenance-using-machine-learning/data'\n",
    "\n",
    "#local data folder\n",
    "data_folder = 'data'\n",
    "\n",
    "!aws s3 cp --recursive $source_data_location $data_folder\n",
    "\n",
    "\n",
    "with zipfile.ZipFile(os.path.join(data_folder, 'CMAPSSData.zip'), \"r\") as zip_ref:\n",
    "    zip_ref.extractall(data_folder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Process  Data\n",
    "\n",
    "There are 4 different training dataset, corresponding to 4 different engines, and we will train our models on each individually. But first we must preprocess the dataset. Since the example dataset we're using is small we can run the preprocessing code located in `preprocess.py` on this notebook instance. The preprocessing script computes the RUL (Remaining Useful Life) column for each engine data in the training dataset. We also apply min-max scaling to normalize our sensor readings to between 0 and 1 for each column. We read and preprocess the test data as well. It consists of sensor readings and the actual RUL. See the code in `preprocess.py` for more. To adapt this to your own dataset, you would need to implement your custom preprocessing for your dataset. We plot a few columns from the training data for visualization purposes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_df, test_df, columns = preprocess.preprocess_data(data_folder)\n",
    "\n",
    "o = train_df[0][columns[2:10]][train_df[0]['id'] == 3].plot(subplots=True, sharex=True, figsize=(20,10), title=\"Train: 8 sensors of Engine 1 before failure\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SageMaker MXNet Estimator\n",
    "\n",
    "Now we will go over the steps needed to define the MXNet model and train with SageMaker."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload processed data to S3 for training\n",
    "\n",
    "We have to upload the processed data to a location in S3 so that the SageMaker training instance can access the data from that location. We will also, at the same time, upload the test data to the S3 bucket so that we can use that as an input to the trained model for scheduled inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.s3 import S3Uploader\n",
    "\n",
    "bucket = config.solution_bucket\n",
    "prefix = config.s3_prefix\n",
    "\n",
    "s3_data_prefix = \"s3://{}/{}/{}\".format(bucket, prefix, 'data')\n",
    "\n",
    "# Make local folder for processed data\n",
    "! mkdir -p processed-data\n",
    "\n",
    "# Upload processed test data for inference\n",
    "for i in range(len(test_df)):\n",
    "    local_test_file = 'processed-data/test-{}.csv'.format(i)\n",
    "    test_df[i].to_csv(local_test_file)\n",
    "    S3Uploader.upload(local_test_file, s3_data_prefix)\n",
    "\n",
    "# Upload processed data for training\n",
    "for i in range(len(train_df)):\n",
    "    local_train_file = 'processed-data/train-{}.csv'.format(i)\n",
    "    train_df[i].to_csv(local_train_file)\n",
    "    S3Uploader.upload(local_train_file, s3_data_prefix)\n",
    "\n",
    "\n",
    "print('uploaded training data location: {}'.format(s3_data_prefix))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set model output location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_location = 's3://{}/{}/output'.format(bucket, prefix)\n",
    "print('training artifacts will be uploaded to: {}'.format(output_location))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MXNet Model Training script\n",
    "\n",
    "Training MXNet models using MXNet Estimators is a two-step process. First, you prepare your training script, then second, you run this on SageMaker via an MXNet Estimator. The training script we have prepared for the model is located in the `sagemaker_predictive_maintenance_entry_point` folder. \n",
    "\n",
    "The training script contains functions to create the model for training and for inference. We also have functions to convert our dataframes into a Gluon Dataset so that it can be efficiently prefetched, transformed into numerical features used by the network and padded so that we can learn from multiple samples in batches.\n",
    "\n",
    "For more information on how to setup a training script for SageMaker using the MXNet estimator see: https://sagemaker.readthedocs.io/en/stable/using_mxnet.html#preparing-the-mxnet-training-script"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pygmentize sagemaker_predictive_maintenance/sagemaker_predictive_maintenance_entry_point/sagemaker_predictive_maintenance_entry_point.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train MXNet Estimator\n",
    "\n",
    "Now, we can start the SageMaker training job by creating an MXNet estimator. We pass in the required arguments such as the `entry_point`, `role`, `train_instance_type`, and `train_instance_count` into the MXNet Estimator constructor.\n",
    "\n",
    "Then we start the training script by calling `fit` on the MXNet Estimator. `fit` takes both required and optional arguments. The required argument here is the S3 location of the training data passed in as a dictionary. We are also adding an optional argument for the job name. This is important because when the training job is complete and SageMaker needs to create a SageMaker Model for real-time inference or batch transformation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from sagemaker.mxnet import MXNet\n",
    "from time import strftime, gmtime\n",
    "\n",
    "training_job_name = \"{}-{}\".format(config.model_name, strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime()))\n",
    "train_instance_type = 'ml.p3.2xlarge'\n",
    "\n",
    "m = MXNet(entry_point='sagemaker_predictive_maintenance_entry_point.py',\n",
    "          source_dir='sagemaker_predictive_maintenance/sagemaker_predictive_maintenance_entry_point',\n",
    "          py_version='py3',\n",
    "          role=role, \n",
    "          train_instance_count=1, \n",
    "          train_instance_type=train_instance_type,\n",
    "          output_path=output_location,\n",
    "          hyperparameters={'num-datasets' : len(train_df),\n",
    "                           'num-gpus': 1,\n",
    "                           'epochs': 200,\n",
    "                           'optimizer': 'adam',\n",
    "                           'batch-size':1,\n",
    "                           'log-interval': 100},\n",
    "         input_mode='File',\n",
    "         train_max_run=7200,\n",
    "         framework_version='1.6.0')\n",
    "\n",
    "m.fit({'train': s3_data_prefix}, job_name=training_job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Transformer Model\n",
    "\n",
    "We can now call the `transformer` function to create a SageMaker Model from the estimator. The SageMaker Model will have the same name as the base name for the training job that just completed. We check first that a previous model doesn't exist with the same name. If it does, we delete the stale model and update the model name to refer to the model artifacts from the newly completed training job. This will ensure that SageMaker stores a reference to the current trained model which can be used for predictions later on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "utils.delete_model_if_exists(config.model_name)\n",
    "batch_output = 's3://{}/{}/{}'.format(bucket, prefix, 'batch-inference')\n",
    "transformer = m.transformer(instance_count=1, instance_type='ml.m4.xlarge', output_path=batch_output,\n",
    "                            model_name=config.model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Transform test data using the transformer model\n",
    "\n",
    "Using the `transformer` SageMaker Model, we can run a [SageMaker Batch Transform](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html) job to get some predictions on test dataset for the model. Here we have a function that takes some test data in S3 and copies it to a new location where it's used as the input to the `transform` function of the Batch Transformer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_test_key = \"{}/data/test-0.csv\".format(prefix)\n",
    "s3_transform_input = \"{}/batch-transform-input\".format(prefix)\n",
    "\n",
    "job_name, input_key = utils.get_transform_input(bucket, config.solution_prefix, s3_test_key, s3_transform_input)\n",
    "transformer.transform(input_key, wait=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View model prediction results\n",
    "\n",
    "Once the transform job terminates, we can see the models predictions for the fractional remaining useful life left for the sensor readings in the `data/test-0.csv`. The predictions are a fraction of `MAX_RUL` which is `130.0`, therefore the Remaining Useful Life predictions can be obtained by multiplying the output with 130"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "utils.get_transform_output(bucket, prefix, job_name)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}