5. Premium Business Users Requests Implementation

Model Training Process

Current Phase Included on Deep Image Hub

Step 1. Verify User Request before training the Compute Vision Model

Inputs: user_id, label_list

1. Get user_info from user_id

2. Verify given labels

   1. Check the images counts for each given label is there enough images for training - Threshold: 100

3. If there are enough images:

   1. Download the images to the /tmp folder with proper folder structure for the requested labels
   2. Save the meta data into database for this training request and return an model_id from database

If there is not enough images for training, store the shorted labels into a watch list and notify that there is not enough training data for the listing labels at the moment.

To start a model training request please run the following script:

returned_model_id=$(python ~/Deep_Images_Hub/src/training/requester.py  --label_List Guitar Piano Microphone  --user_id 2 2>&1)

# a model id will be return once requester process is completed and we will carry on this variable for the rest of the example 
echo $returned_model_id  

Step 2. The actual model training process.

I modified the code from this out of box Keras and Convolutional Neural Networks (CNNs) Tutorial from PyImageSearch

The training process can be run as the following

python ~/Deep_Images_Hub/src/training/cnn-keras/train.py --dataset /tmp/Deep_image_hub_Model_Training/dataset --model /tmp/Deep_image_hub_Model_Training/model/sample_model.model --plot /tmp/Deep_image_hub_Model_Training/model/plot.png --labelbin lb.pickle --training_request_number $returned_model_id

Step 3. Post training processes.

Post training processes will take care the images clean up on the tmp folder, save the trained model and the training progress plot to S3 Bucket, and upload the meta data to database.

python ~/Deep_Images_Hub/src/training/postTraining.py --training_request_number $returned_model_id --user_id 2

The trained model will be listed on S3

The trained model will also list on Deep Image Hub Web Site

The detailed summary about the model will be display. And user can download the trained model from the web site.

Airflow CLI

Currently the above steps can be trigger with this airflow dag.

airflow trigger_dag local_model_training --conf '{"Labels":"Guitar Piano Microphone " , "User_id":"2" }'

---

Tactical and Strategic Solutions