Skip to content

Latest commit

 

History

History
58 lines (42 loc) · 3.5 KB

README.md

File metadata and controls

58 lines (42 loc) · 3.5 KB

Dispatch analysis

Data processing analysis for dispatch data across the Twin Cities metro area.

Data processing

  1. Set DB URI with DISPATCH_DB_URI environment variable.

Import data

  1. Put original files into the data/sources/ folder.
    • See lib/convert-sources.sh to see how it should be structured.
  2. Convert file formats into CSV and rename: bash lib/convert-sources.sh
  3. Import into database, one at a time.
    • Example: node lib/import.js --file="data/build/minneapolis--all--all--2017-01.csv"
    • Use the --force flag to delete the dispatch table and rebuild (useful for model changes)
    • TODO: Create a --reload flag that will delete all rows with the same file name.
  4. Get census tracts:
    • Metro area race data
      • curl "https://api.censusreporter.org/1.0/data/download/latest?table_ids=B03002&geo_ids=140|31000US33460&format=geojson" | tar -xOzf - acs2016_5yr_B03002_14000US27003050608/acs2016_5yr_B03002_14000US27003050608.geojson | node lib/geojson-to-postgis --table="census_tracts_B03002"
    • Metro area median household income
      • curl "https://api.censusreporter.org/1.0/data/download/latest?table_ids=B19013&geo_ids=140|31000US33460&format=geojson" | tar -xOzf - acs2016_5yr_B19013_14000US27003050608/acs2016_5yr_B19013_14000US27003050608.geojson | node lib/geojson-to-postgis --table="census_tracts_B19013"
  5. Get Minnesota cities from state legislature:
    • curl "https://www.gis.leg.mn/php/shptoGeojson.php?file=/geo/data/mcd/mcd2012" | node lib/geojson-to-postgis --table="mn_cities" --source-proj="EPSG:26915" --target-proj="EPSG:4326"
  6. Get Minneapolis precincts from Minneapolis open data:
    • curl "https://opendata.arcgis.com/datasets/02ab4e12be434fee9d266531f563a6e8_0.zip" | shp2json | node lib/geojson-to-postgis --table="mpls_precincts"
  7. Get Minneapolis neightborhoods from Minneapolis open data:
    • curl "https://opendata.arcgis.com/datasets/7f88316841ce471faa33c89035fb69e8_0.zip" | shp2json | node lib/geojson-to-postgis --table="mpls_neighborhoods"

Analysis

For hexbin analysis

  1. Create hexbins and import into database (database queries are much faster than doing anything JS).
    • node lib/hexbin.js | node lib/geojson-to-postgis.js --table="dispatch_hexes"
    • Can change the hexbin options with:
      • --units: Defaults to miles.
      • --side: length of side, defaults to 0.25.
  2. Run query and create geojson: node lib/hexbin-analysis.js > data/build/hexbin-analysis.geo.json
    • Can use the --responding-type="fire" flag, but note not all data has this information.
      • node lib/hexbin-analysis.js --responding-type="fire" > data/build/hexbin-analysis-fire.geo.json
      • node lib/hexbin-analysis.js --responding-type="police" > data/build/hexbin-analysis-police.geo.json

For race by census tract

  1. Create CSV: node lib/race-analysis.js > data/build/race-analysis.csv

For median income by census tract

  1. Create CSV: node lib/income-analysis.js > data/build/income-analysis.csv

For MPD hour analysis

  1. Create JSON: node lib/hour-analysis > data/build/hour-analysis.json

For MPD neighborhood analysis

  1. Create JSON: node lib/neighborhood-analysis > data/build/neighborhood-analysis.json