7. References & Resources

The following are references and resources I came across while I was working on this project. They gave me lights of how to solve some of the technically challenges.

Deep Learning at Scale:

• HANDS ON DEEP LEARNING WITH KERAS, TENSORFLOW, AND APACHE SPARK™
• Databrick distributed deep Learning Overview
• Apache Mxnet and Apache Spark on Amazon EMR
• TensorflowOnSpark create AMI
• TensorflowOnSpark on EC2
• Remember to include the package for Sparkdl

Computer Vision:

• Keras and Convolutional Neural Networks (CNNs)
• Multi-label classification with Keras

Spark:

• Apache Spark Installation Guide on Ubuntu 16.04 LTS
• psycopg2 slowness issue on pyspark
• Spark Tutorial
• Spark Dataframe and Operation
• Image support in Spark with ImageSchema

AWS:

• AWS Step Function
• Amazon EMR custom Amazon Machine Image (AMI)
• Using AWS Lambda with Amazon S3
• Amazon EC2 Spot Instances Pricing
• SageMaker Example
• Amazon Aurora vs. RDS
• Amazon S3 on Hadoop

SQL:

• PostgreSQL Arrays

• psycopg2-multiple-insert

• PostgreSQL get last inserted Id

• Style Guide

• Run on Shell

Unix:

• Check ports in use
• Pop a line of a file from bash

Airflow:

• Get started developing workflows with Apache Airflow
• Airflow Tutorial for Data Pipelines

Flask

The Flask Mega-Tutorial

Open Images:

• Suggested to use OIDv4_ToolKit for downloading source image dataset
• Pretrained Model Resnet 101 trained with Open Images V2

Mobile:

• Transfer Files and Data Using TransferUtility and Amazon S3
• Swift S3TransferUtility Sample
• Swift Amazon S3 Uploading Tutorial

Performance Optimizations: