6.Checkpoints_ML_Data_Preparation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center> <img style=\"float: center;\" src=\"images/CI_horizontal.png\" width=\"450\">\n",
    "<center>\n",
    "    <span style=\"font-size: 1.5em;\">\n",
    "        <a href='https://www.coleridgeinitiative.org'>Website</a>\n",
    "    </span> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center>**Checkpoints</center>**\n",
    "## **<center>Part 1: Machine Learning Data Preparation</center>**\n",
    "\n",
    "<a href=\"https://doi.org/10.5281/zenodo.6369190\"><img src=\"https://zenodo.org/badge/DOI/10.5281/zenodo.6369190.svg\" alt=\"DOI\"></a>\n",
    "\n",
    "<center>Tian Lou</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we provide you with checkpoints to practice on. The structure of this notebook is the same as the first ML notebook's. We removed the texts between code cells so that you can focus on the code. **For detailed explanations about the analysis steps and code, refer back to the [machine learning data preparation notebook](./4.Machine_Learning_Data_Preparation.ipynb).**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **1. Load the Data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Database interaction imports\n",
    "library(odbc)\n",
    "\n",
    "# For data manipulation/visualization\n",
    "library(tidyverse)\n",
    "\n",
    "# For faster date conversions\n",
    "library(lubridate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Connect to the database\n",
    "con <- DBI::dbConnect(odbc::odbc(),\n",
    "                     Driver = \"SQL Server\",\n",
    "                     Server = \"msssql01.c7bdq4o2yhxo.us-gov-west-1.rds.amazonaws.com\",\n",
    "                     Database = \"tr_dol_eta\",\n",
    "                     Trusted_Connection = \"True\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **IL PROMIS Certified Claims File**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select PROMIS certified claimant records from the database to a dataframe\n",
    "\n",
    "# Store SQL query to a character variable\n",
    "query <- \"\n",
    "SELECT ssn_id,\n",
    "    week_end_date,\n",
    "    byr_start_week,\n",
    "    birth_date,\n",
    "    gender,\n",
    "    race,\n",
    "    ethnicity,\n",
    "    disability,\n",
    "    education,\n",
    "    county_fips_code,\n",
    "    naics_code,\n",
    "    occupation_code,\n",
    "    total_pay,\n",
    "    wages_2019\n",
    "FROM tr_dol_eta.dbo.il_des_promis_1pct\n",
    "WHERE sub_program_type = 1\n",
    "AND program_type = 1\n",
    "AND claim_type = 1\n",
    "AND byr_start_week in ('2020-03-28','2020-04-04','2020-06-27','2020-07-04');\n",
    "\"\n",
    "\n",
    "# Execute query\n",
    "df_claimants <-dbGetQuery(con,query)\n",
    "\n",
    "# R interprets dates as character when pulling from the database, must convert with ymd()\n",
    "df_claimants <- df_claimants %>%\n",
    "    mutate(week_end_date=ymd(week_end_date),\n",
    "          byr_start_week=ymd(byr_start_week),\n",
    "          birth_date=ymd(birth_date))\n",
    "\n",
    "# See top records in the dataframe\n",
    "head(df_claimants)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Add the cohort indicator to df_claimants\n",
    "df_claimants <- df_claimants %>%\n",
    "    mutate(cohort = case_when(byr_start_week == '2020-03-28'|byr_start_week == '2020-04-04' ~ 'cohort1',\n",
    "                              byr_start_week == '2020-06-27'|byr_start_week == '2020-07-04' ~ 'cohort2',\n",
    "                              TRUE ~ 'other'))\n",
    "\n",
    "#See top records of the cohort indicator we just generated\n",
    "head(df_claimants %>% select(ssn_id, byr_start_week, cohort))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Opportunity Insights Economic Tracker: Consumer Spending Data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Select IL Consumer spending data from the database to a dataframe\n",
    "\n",
    "# Store SQL query to a character variable\n",
    "# In this query, we use CONVERT() and CAST() statements to combine the year, month, and day columns into a date variable 'spend_date'\n",
    "query <- \"\n",
    "SELECT CONVERT(date, CAST([year] AS varchar(4)) + '-' +  \n",
    "                     CAST([month] AS varchar(2)) + '-' + \n",
    "                     CAST([day] AS varchar(2))) AS spend_date,\n",
    "    county_fips,\n",
    "    spend_all\n",
    "FROM tr_dol_eta.dbo.affinity_county_daily;\n",
    "\"\n",
    "\n",
    "# Execute query\n",
    "df_spend <-dbGetQuery(con,query)\n",
    "\n",
    "# R interprets dates as character when pulling from the database, must convert with ymd()\n",
    "df_spend <- df_spend %>%\n",
    "    mutate(spend_date=ymd(spend_date))\n",
    "\n",
    "# See top records in the dataframe\n",
    "head(df_spend)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Clean up FIPS code and limit the spending data to IL counties that show in the data consistently over time\n",
    "df_spend <- df_spend %>%\n",
    "    mutate(county_fips = str_pad(county_fips, 5, side = c(\"left\"), pad=\"0\")) %>% # Add a leading zero to county fips code\n",
    "    mutate(state_fips = substr(county_fips, 1, 2), county_fips_code = substr(county_fips, 3, 5)) %>% # Generate two-digit state FIPS code and three-digit county FIPS code\n",
    "    filter(state_fips == 'REDACTED' & county_fips_code!= 'REDACTED') %>% # Only keep IL data and remove a county that is not consistenly available during the time period we look at\n",
    "    select(spend_date, spend_all, county_fips_code) # Only keep the columns we need\n",
    "\n",
    "#Check county_fips_code and each FIPS code's frequency\n",
    "table(df_spend$county_fips_code)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate weekly average spending\n",
    "df_weekly_spend <- df_spend %>% mutate(spend_week = epiweek(spend_date)) %>% # Create a week indicator. For example, any date between 12-29-2019 and 01-04-2020 will be labelled as 1.\n",
    "    group_by(spend_week, county_fips_code) %>% \n",
    "    summarize(avg_spend = mean(spend_all)) # Calculate average weekly spending index for each county\n",
    "\n",
    "#See the top records\n",
    "head(df_weekly_spend)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate the two cohorts' week indicators\n",
    "cohort1_week <- epiweek('2020-03-28')\n",
    "cohort2_week <- epiweek('2020-06-27')\n",
    "\n",
    "# Average weekly spending indexes one week, four weeks, and eight weeks prior to Cohort 1's entry\n",
    "df_cohort1_avg_spend <- df_weekly_spend %>%\n",
    "    filter(spend_week %in% c(cohort1_week-1, cohort1_week-4, cohort1_week-8)) %>% # Subset the data to 1, 4, 8 weeks prior program entry week\n",
    "    pivot_wider(names_from = spend_week, values_from = avg_spend) %>% # Reshape the data to wide format so that each week's spending index is a column\n",
    "    `colnames<-`(c('county_fips_code', 'avg_spend_8', 'avg_spend_4', 'avg_spend_1')) %>% # Rename columns\n",
    "    mutate(cohort = 'cohort1') # Generate the cohort indicator\n",
    "\n",
    "# Average weekly spending indexes one week, four weeks, and eight weeks prior to Cohort 1's entry\n",
    "df_cohort2_avg_spend <- df_weekly_spend %>%\n",
    "    filter(spend_week %in% c(cohort2_week-1, cohort2_week-4, cohort2_week-8)) %>% # Subset the data to 1, 4, 8 weeks prior program entry week\n",
    "    pivot_wider(names_from = spend_week, values_from = avg_spend) %>% # Reshape the data to wide format so that each week's spending index is a column\n",
    "    `colnames<-`(c('county_fips_code', 'avg_spend_8', 'avg_spend_4', 'avg_spend_1')) %>% # Rename columns\n",
    "    mutate(cohort = 'cohort2') # Generate the cohort indicator\n",
    "\n",
    "# Append(Combine) the two DataFrames\n",
    "df_avg_spend <- rbind(df_cohort1_avg_spend, df_cohort2_avg_spend)\n",
    "\n",
    "# See the top records of the DataFrame\n",
    "head(df_avg_spend)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Combine Datasets**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Inner join the claims data with the average spending data on county FIPS code and the cohort indicator\n",
    "#Inner join keeps the rows that are in both df_claimants and df_avg_spend\n",
    "df_claimants <- df_claimants %>%\n",
    "    inner_join(df_avg_spend, by = c('county_fips_code','cohort'))\n",
    "\n",
    "#See top records of the DataFrame\n",
    "head(df_claimants)           "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Checkpoint 1: Create a regional sample**\n",
    "\n",
    "Limit the DataFrame we created in this section, `df_claimants`, to your region(s) of interest.<font color=red> Some regions may not have enough observations, for example, Central, North-Central, and Southern. We suggest you to either choose a relatively big region, such as Cook County or Northeast, or choose more than one region.</font>\n",
    "\n",
    "> Values of `region`: Central, Cook County, North-Central, Northeast, Southern\n",
    "\n",
    "> Note that we no longer use the string `reg` when naming DataFrames. All the DataFrames after this checkpoint only contain data of the region(s) you select."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Join/merge the DataFrame with the county-region crosswalk\n",
    "\n",
    "# Load Restore Illinois Health Regions\n",
    "region_dict <- read_csv('P:\\\\tr-dol-eta\\\\ETA Class Notebooks\\\\xwalks\\\\reopening_regions.csv', col_types = \"ic\") %>%\n",
    "    mutate(county_fips_code = substr(fips,3,5)) %>%\n",
    "    select(county_fips_code, region)\n",
    "\n",
    "#Left join region to certified claimants data\n",
    "df_claimants <- left_join(df_claimants, region_dict, by.y = \"county_fips_code\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Investigate the number of claimants in each region and each cohort\n",
    "df_claimants %>% group_by(region, cohort) %>% summarize(n=n_distinct(ssn_id))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Subset the dataframe to a specific region\n",
    "# Replace ___ with your selected region\n",
    "df_claimants <- df_claimants %>% filter(region == ___)\n",
    "\n",
    "# See top records in the dataframe\n",
    "head(df_claimants)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **2. Create the Label**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A few people's benefit year starting date changed, adjust them to the earliest date\n",
    "df_claimants <- df_claimants %>%\n",
    "    group_by (ssn_id, cohort) %>%\n",
    "    mutate(byr_start_week = min(byr_start_week)) %>% # For each person in each cohort, adjust the 'byr_start_week' to the earliest date\n",
    "    ungroup() #return data to non-grouped form. If we don't include this code, you may get errors in the later analysis\n",
    "\n",
    "# Create week number field and keep the first 13 weeks of records for each person\n",
    "df_claimants <- df_claimants %>% \n",
    "    mutate(week_number = as.integer(difftime(week_end_date, byr_start_week, units = \"weeks\")) + 1) %>%\n",
    "    filter(week_number <= 13) # Only keep each person's first 13 records"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Checkpoint 2: Create the Label**\n",
    "\n",
    "In this project, we define claimants who left before the 13th week after program entry as **fast exiters** and their labels should be 0. We define claimants who stayed in the UI program for 13 weeks or longer as **slow exiters** and their labels should be 1. In the third line of the code, we have calculated how many weeks each claimant stayed in the UI program during the first 13 weeks, `stay_weeks`. Based on this information, what conditions should you use inside of the `case_when()` function to define the label?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the label\n",
    "# Replace ___ with the conditions we use to define the label\n",
    "df_label <- df_claimants %>% \n",
    "    group_by(ssn_id, cohort) %>%\n",
    "    summarize(stay_weeks = n()) %>% #count each person's number of records\n",
    "    mutate(label = case_when(___ ~  ___, ___ ~ ___)) %>% #create the label, if stayed 13 weeks or more, then label=1, otherwise, label=0\n",
    "    ungroup() #return data to non-grouped form. If we don't include this code, you may get errors in the later analysis\n",
    "\n",
    "#Check the top rows of df_label\n",
    "head(df_label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Understand what percent of claimants in each cohort left before the 13th week\n",
    "\n",
    "#Count number of people by cohort and label\n",
    "df_label_freq <- df_label %>% group_by(cohort,label) %>% summarize(freq = n())\n",
    "\n",
    "#Count number of claimants in each cohort\n",
    "df_cohort_pop <- df_label %>% group_by(cohort) %>% summarize(cohort_pop = n_distinct(ssn_id))\n",
    "\n",
    "#Left join the two DataFrame and Calculate the percentages of fast exiters and slow exiters in each cohort\n",
    "df_label_freq <- df_label_freq %>%\n",
    "    left_join(df_cohort_pop, by = 'cohort') %>%\n",
    "    mutate(percent = round((freq/cohort_pop),3))\n",
    "\n",
    "#Look at the statistics\n",
    "df_label_freq"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **3.Create the Features**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#First, we need to calculate average weekly total pay\n",
    "df_claimants <- df_claimants %>%\n",
    "    group_by(ssn_id, cohort) %>%\n",
    "    mutate(avg_total_pay = mean(total_pay))\n",
    "\n",
    "#Keep each person's first record, since we only need one set of features for each person\n",
    "df_feature <- df_claimants %>% \n",
    "    arrange(ssn_id, week_end_date) %>% #Sort the data ascendingly based on ssn_id and week_end_date\n",
    "    group_by(ssn_id, cohort) %>%\n",
    "    mutate(record_id = row_number()) %>% #create record ID for each person\n",
    "    filter(record_id == 1)  %>% #Keep each person's first record \n",
    "    ungroup() #return data to non-grouped form. If we don't include this code, you may get errors in the later analysis\n",
    "\n",
    "# See top records of df_feature\n",
    "head(df_feature)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate age at program entry and bottom code outliers\n",
    "df_feature <- df_feature %>%\n",
    "    mutate(age = as.integer(difftime(byr_start_week, birth_date, units='days')/365.25)) %>%\n",
    "    mutate(age = case_when(age<REDACTED ~ as.integer(REDACTED), TRUE~age)) # A small number of people are too young. We bottom code their ages\n",
    "\n",
    "# Recode gender, race, ethnicity, disability, and education\n",
    "df_feature <- df_feature %>%\n",
    "    mutate(gender = case_when(gender == 1 ~ 'Male', gender == 2 ~ 'Female', TRUE ~ 'Unknown'),\n",
    "           race = case_when(race == 1 ~ 'White', race == 2 ~ 'African_American', race %in% c(3, 4, 6) ~ 'Other_race', TRUE ~ 'Unknown'),\n",
    "           ethnicity = case_when(ethnicity == 1 ~ 'Hispanic', ethnicity == 2 ~ 'Not_Hispanic', TRUE ~ 'Unknown'),\n",
    "           disability = case_when(disability == 1 ~'Disabled', disability == 2 ~ 'Not_Disabled', TRUE ~ 'Unknown'),\n",
    "           education = case_when (education >= 1 & education <= 13 ~ \"Less_than_HS\",\n",
    "                                    education >= 14 & education <= 18 ~ \"HS_graduate_or_some_college\",\n",
    "                                    education >= 19 & education <= 20 ~ \"Associate\",\n",
    "                                    education >= 21 & education <= 22 ~ \"Bachelor\",\n",
    "                                    education >= 23 ~ \"Master_or_higher\",\n",
    "                                    TRUE ~ \"Other\" ))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Combine NAICS major codes based on grouping used for UI dashboard \n",
    "# Import NAICS code crosswalk\n",
    "naics_groups <- read_csv('P:\\\\tr-dol-eta\\\\ETA Class Notebooks\\\\xwalks\\\\naics_groups.csv', col_types = \"ccc\")    \n",
    "\n",
    "# Convert 6-digit NAICS codes to 2-digit NAICS codes by keeping only the first two characters\n",
    "df_feature <- df_feature %>% mutate(naics_maj_code = substr(naics_code,1,2))\n",
    "\n",
    "# Join NAICS groupings to claimant dataset\n",
    "df_feature <- df_feature %>% \n",
    "    left_join(naics_groups, by = 'naics_maj_code') %>%\n",
    "    mutate(naics_maj_code_rv = case_when(naics_maj_code %in% c('REDACTED') ~ 'REDACTED',TRUE ~ naics_maj_code_rv)) \n",
    "\n",
    "# Check the top records of df_feature\n",
    "head(df_feature)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Only keep the columns we need\n",
    "df_feature <- df_feature %>% \n",
    "    select(ssn_id, cohort, byr_start_week, # Variables to identify person and cohort\n",
    "           gender, race, ethnicity, disability, education, naics_maj_code_rv, occupation_code, # Categorical Variables\n",
    "           wages_2019, avg_spend_1, avg_spend_4, avg_spend_8, avg_total_pay, age) #Numeric variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Checkpoint 3: Check Missing Values**\n",
    "\n",
    "1. Check if any of your features has missing values. How are you going to deal with missing values?\n",
    "\n",
    "2. Assume the variable `wages_2019` has missing values. We imputed its missing values with the average earnings of claimants who have the same education level. Are you going to use the same imputation method or a different method?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Check which columns have missing values\n",
    "sapply(df_feature, function(x) sum(is.na(x)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Fill in missing values\n",
    "#Replace the first '___' with the variable you use to calculate group average earnings. \n",
    "#For example, if you want to calculate race level average, replace '___' with race. \n",
    "#Replace the second '___' with the values you use to fill in missing earnings\n",
    "df_feature <- df_feature %>%\n",
    "    group_by(___) %>%\n",
    "    mutate(wages_2019 = replace(wages_2019, is.na(wages_2019), ___))) %>%\n",
    "    ungroup()  #return data to non-grouped form. If we don't include this code, you may get errors in the later analysis\n",
    "\n",
    "#Check missing values again\n",
    "sapply(df_feature, function(x) sum(is.na(x)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Checkpoint 4: Convert Features into Proper Formats**\n",
    "\n",
    "Before we run a ML model, we need to scale all the numeric variables by using the `scale()` function and convert all the categorical variables into factor type by using the `factor()` function. Complete the code below to convert all the features into proper formats."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### **Numeric Features**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scale the numeric variables by cohort\n",
    "# Replace the '___' with the appropriate function and variables to get scaled features\n",
    "df_feature <- df_feature %>%\n",
    "    group_by(cohort) %>%\n",
    "    mutate(age_scaled = ___, avg_total_pay_scaled = ___, \n",
    "           avg_spend_1_scaled = ___, avg_spend_4_scaled = ___,\n",
    "           avg_spend_8_scaled = ___, wages_2019_scaled = ___) %>%\n",
    "    select(-c(age, avg_total_pay, avg_spend_1, avg_spend_4, avg_spend_8,wages_2019)) %>% # Remove columns we don't need\n",
    "    ungroup()  #return data to non-grouped form. If we don't include this code, you may get errors in the later analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Categorical Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use gender as an example\n",
    "# Convert character type into factor type; numeric variables can be converted into factor type as well\n",
    "# Replace the '___' with the appropriate function and variable to convert gender into factor type\n",
    "df_feature$gender <- ___\n",
    "\n",
    "#Check the types of ssn_id, gender, race. They should have <int>, <fct>, <chr> types, respectively.\n",
    "head(df_feature %>%  select(ssn_id, gender, race))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **6.Combine DataFrames**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Combine the label DataFrame (df_label) \n",
    "# and the feature DataFrame with dummy variables and scaled numeric variables (df_feature_trans)\n",
    "df_ml <- df_label[,c('ssn_id','cohort','label')] %>%\n",
    "    left_join(df_feature, on = c('ssn_id','cohort'))\n",
    "\n",
    "# See top records of the final DataFrame\n",
    "head(df_ml)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=red> Note that you need to create a \"Data\" folder in your \"ETA Training\" folder first. Then change the directory in write.csv() statements below. Replace \". .\" with your username.</font>\n",
    "\n",
    "> Note that the csv file name is `reg_ml_data.csv`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Export the data to a csv file. We will use it in the next notebook\n",
    "write.csv(df_ml, \"U:\\\\..\\\\ETA Training\\\\Data\\\\reg_ml_data.csv\", row.names=F)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.0.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}