09 - Create a Real-time Inferencing Service.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create a real-time inferencing service\n",
    "\n",
    "After training a predictive model, you can deploy it as a real-time service that clients can use to get predictions from new data.\n",
    "\n",
    "## Install the Azure Machine Learning SDK\n",
    "\n",
    "The Azure Machine Learning SDK is updated frequently. Run the following cell to upgrade to the latest release, along with the additional package to support notebook widgets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade azureml-sdk azureml-widgets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to your workspace\n",
    "\n",
    "With the latest version of the SDK installed, now you're ready to connect to your workspace.\n",
    "\n",
    "> **Note**: If you haven't already established an authenticated session with your Azure subscription, you'll be prompted to authenticate by clicking a link, entering an authentication code, and signing into Azure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace\n",
    "\n",
    "# Load the workspace from the saved config file\n",
    "ws = Workspace.from_config()\n",
    "print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train and register a model\n",
    "\n",
    "Now let's train and register a model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Experiment\n",
    "from azureml.core import Model\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "\n",
    "# Create an Azure ML experiment in your workspace\n",
    "experiment = Experiment(workspace=ws, name=\"mslearn-train-diabetes\")\n",
    "run = experiment.start_logging()\n",
    "print(\"Starting experiment:\", experiment.name)\n",
    "\n",
    "# load the diabetes dataset\n",
    "print(\"Loading Data...\")\n",
    "diabetes = pd.read_csv('data/diabetes.csv')\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Train a decision tree model\n",
    "print('Training a decision tree model')\n",
    "model = DecisionTreeClassifier().fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "# Save the trained model\n",
    "model_file = 'diabetes_model.pkl'\n",
    "joblib.dump(value=model, filename=model_file)\n",
    "run.upload_file(name = 'outputs/' + model_file, path_or_stream = './' + model_file)\n",
    "\n",
    "# Complete the run\n",
    "run.complete()\n",
    "\n",
    "# Register the model\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'Inline Training'},\n",
    "                   properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "print('Model trained and registered.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy the model as a web service\n",
    "\n",
    "You have trained and registered a machine learning model that classifies patients based on the likelihood of them having diabetes. This model could be used in a production environment such as a doctor's surgery where only patients deemed to be at risk need to be subjected to a clinical test for diabetes. To support this scenario, you will deploy the model as a web service.\n",
    "\n",
    "First, let's determine what models you have registered in the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Right, now let's get the model that we want to deploy. By default, if we specify a model name, the latest version will be returned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = ws.models['diabetes_model']\n",
    "print(model.name, 'version', model.version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're going to create a web service to host this model, and this will require some code and configuration files; so let's create a folder for those."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "folder_name = 'diabetes_service'\n",
    "\n",
    "# Create a folder for the web service files\n",
    "experiment_folder = './' + folder_name\n",
    "os.makedirs(experiment_folder, exist_ok=True)\n",
    "\n",
    "print(folder_name, 'folder created.')\n",
    "\n",
    "# Set path for scoring script\n",
    "script_file = os.path.join(experiment_folder,\"score_diabetes.py\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The web service where we deploy the model will need some Python code to load the input data, get the model from the workspace, and generate and return predictions. We'll save this code in an *entry script* (often called a *scoring script*) that will be deployed to the web service:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $script_file\n",
    "import json\n",
    "import joblib\n",
    "import numpy as np\n",
    "from azureml.core.model import Model\n",
    "\n",
    "# Called when the service is loaded\n",
    "def init():\n",
    "    global model\n",
    "    # Get the path to the deployed model file and load it\n",
    "    model_path = Model.get_model_path('diabetes_model')\n",
    "    model = joblib.load(model_path)\n",
    "\n",
    "# Called when a request is received\n",
    "def run(raw_data):\n",
    "    # Get the input data as a numpy array\n",
    "    data = np.array(json.loads(raw_data)['data'])\n",
    "    # Get a prediction from the model\n",
    "    predictions = model.predict(data)\n",
    "    # Get the corresponding classname for each prediction (0 or 1)\n",
    "    classnames = ['not-diabetic', 'diabetic']\n",
    "    predicted_classes = []\n",
    "    for prediction in predictions:\n",
    "        predicted_classes.append(classnames[prediction])\n",
    "    # Return the predictions as JSON\n",
    "    return json.dumps(predicted_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The web service will be hosted in a container, and the container will need to install any required Python dependencies when it gets initialized. In this case, our scoring code requires **scikit-learn**, so we'll create a .yml file that tells the container host to install this into the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from azureml.core.conda_dependencies import CondaDependencies \n",
    "\n",
    "# Add the dependencies for our model (AzureML defaults is already included)\n",
    "myenv = CondaDependencies()\n",
    "myenv.add_conda_package('scikit-learn')\n",
    "\n",
    "# Save the environment config as a .yml file\n",
    "env_file = os.path.join(experiment_folder,\"diabetes_env.yml\")\n",
    "with open(env_file,\"w\") as f:\n",
    "    f.write(myenv.serialize_to_string())\n",
    "print(\"Saved dependency info in\", env_file)\n",
    "\n",
    "# Print the .yml file\n",
    "with open(env_file,\"r\") as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you're ready to deploy. We'll deploy the container a service named **diabetes-service**. The deployment process includes the following steps:\n",
    "\n",
    "1. Define an inference configuration, which includes the scoring and environment files required to load and use the model.\n",
    "2. Define a deployment configuration that defines the execution environment in which the service will be hosted. In this case, an Azure Container Instance.\n",
    "3. Deploy the model as a web service.\n",
    "4. Verify the status of the deployed service.\n",
    "\n",
    "> **More Information**: For more details about model deployment, and options for target execution environments, see the [documentation](https://docs.microsoft.com/azure/machine-learning/how-to-deploy-and-where).\n",
    "\n",
    "Deployment will take some time as it first runs a process to create a container image, and then runs a process to create a web service based on the image. When deployment has completed successfully, you'll see a status of **Healthy**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.webservice import AciWebservice\n",
    "from azureml.core.model import InferenceConfig\n",
    "\n",
    "# Configure the scoring environment\n",
    "inference_config = InferenceConfig(runtime= \"python\",\n",
    "                                   entry_script=script_file,\n",
    "                                   conda_file=env_file)\n",
    "\n",
    "deployment_config = AciWebservice.deploy_configuration(cpu_cores = 1, memory_gb = 1)\n",
    "\n",
    "service_name = \"diabetes-service\"\n",
    "\n",
    "service = Model.deploy(ws, service_name, [model], inference_config, deployment_config)\n",
    "\n",
    "service.wait_for_deployment(True)\n",
    "print(service.state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hopefully, the deployment has been successful and you can see a status of **Healthy**. If not, you can use the following code to get the service logs to help you troubleshoot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(service.get_logs())\n",
    "\n",
    "# If you need to make a change and redeploy, you may need to delete unhealthy service using the following code:\n",
    "#service.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take a look at your workspace in [Azure Machine Learning Studio](https://ml.azure.com) and view the **Endpoints** page, which shows the deployed services in your workspace.\n",
    "\n",
    "You can also retrieve the names of web services in your workspace by running the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for webservice_name in ws.webservices:\n",
    "    print(webservice_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use the web service\n",
    "\n",
    "With the service deployed, now you can consume it from a client application."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "x_new = [[2,180,74,24,21,23.9091702,1.488172308,22]]\n",
    "print ('Patient: {}'.format(x_new[0]))\n",
    "\n",
    "# Convert the array to a serializable list in a JSON document\n",
    "input_json = json.dumps({\"data\": x_new})\n",
    "\n",
    "# Call the web service, passing the input data (the web service will also accept the data in binary format)\n",
    "predictions = service.run(input_data = input_json)\n",
    "\n",
    "# Get the predicted class - it'll be the first (and only) one.\n",
    "predicted_classes = json.loads(predictions)\n",
    "print(predicted_classes[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also send multiple patient observations to the service, and get back a prediction for each one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "# This time our input is an array of two feature arrays\n",
    "x_new = [[2,180,74,24,21,23.9091702,1.488172308,22],\n",
    "         [0,148,58,11,179,39.19207553,0.160829008,45]]\n",
    "\n",
    "# Convert the array or arrays to a serializable list in a JSON document\n",
    "input_json = json.dumps({\"data\": x_new})\n",
    "\n",
    "# Call the web service, passing the input data\n",
    "predictions = service.run(input_data = input_json)\n",
    "\n",
    "# Get the predicted classes.\n",
    "predicted_classes = json.loads(predictions)\n",
    "   \n",
    "for i in range(len(x_new)):\n",
    "    print (\"Patient {}\".format(x_new[i]), predicted_classes[i] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code above uses the Azure Machine Learning SDK to connect to the containerized web service and use it to generate predictions from your diabetes classification model. In production, a model is likely to be consumed by business applications that do not use the Azure Machine Learning SDK, but simply make HTTP requests to the web service.\n",
    "\n",
    "Let's determine the URL to which these applications must submit their requests:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint = service.scoring_uri\n",
    "print(endpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you know the endpoint URI, an application can simply make an HTTP request, sending the patient data in JSON format, and receive back the predicted class(es)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "\n",
    "x_new = [[2,180,74,24,21,23.9091702,1.488172308,22],\n",
    "         [0,148,58,11,179,39.19207553,0.160829008,45]]\n",
    "\n",
    "# Convert the array to a serializable list in a JSON document\n",
    "input_json = json.dumps({\"data\": x_new})\n",
    "\n",
    "# Set the content type\n",
    "headers = { 'Content-Type':'application/json' }\n",
    "\n",
    "predictions = requests.post(endpoint, input_json, headers = headers)\n",
    "predicted_classes = json.loads(predictions.json())\n",
    "\n",
    "for i in range(len(x_new)):\n",
    "    print (\"Patient {}\".format(x_new[i]), predicted_classes[i] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You've deployed your web service as an Azure Container Instance (ACI) service that requires no authentication. This is fine for development and testing, but for production you should consider deploying to an Azure Kubernetes Service (AKS) cluster and enabling token-based authentication. This would require REST requests to include an **Authorization** header.\n",
    "\n",
    "## Delete the service\n",
    "\n",
    "When you no longer need your service, you should delete it to avoid incurring unecessary charges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "service.delete()\n",
    "print ('Service deleted.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more information about publishing a model as a service, see the [documentation](https://docs.microsoft.com/azure/machine-learning/how-to-deploy-and-where)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}