code review edits

Author: ejm714

README.md

@@ -158,6 +158,7 @@ Options:
  - [ ] **A.1 Informed consent**: If there are human subjects, have they given informed consent, where subjects affirmatively opt-in and have a clear understanding of the data uses to which they consent?
  - [ ] **A.2 Collection bias**: Have we considered sources of bias that could be introduced during data collection and survey design and taken steps to mitigate those?
  - [ ] **A.3 Limit PII exposure**: Have we considered ways to minimize exposure of personally identifiable information (PII) for example through anonymization or not collecting information that isn't relevant for analysis?
+ - [ ] **A.4 Downstream bias mitigation**: Have we considered ways to enable testing downstream results for biased outcomes (e.g. collecting data on protected group status like race or gender)?
 
 ## B. Data Storage
  - [ ] **B.1 Data security**: Do we have a plan to protect and secure data (e.g., encryption at rest and in transit, access controls on internal users and third parties, access logs, and up-to-date software)?

deon/assets/checklist.yml

@@ -1,5 +1,5 @@
 title: Data Science Ethics Checklist
-sections: 
+sections:
   - title: Data Collection
     section_id: A
     lines:
@@ -14,7 +14,7 @@ sections:
           line: Have we considered ways to minimize exposure of personally identifiable information (PII) for example through anonymization or not collecting information that isn't relevant for analysis?
         - line_id: A.4
           line_summary: Downstream bias mitigation
-          line: Have we considered ways to include demographic data (e.g. race, gender) where possible to enable testing downstream results for biased outcomes?
+          line: Have we considered ways to enable testing downstream results for biased outcomes (e.g. collecting data on protected group status like race or gender)?
   - title: Data Storage
     section_id: B
     lines:

deon/assets/examples_of_ethical_issues.yml

@@ -18,12 +18,12 @@
       url: https://www.wired.com/2007/12/why-anonymous-data-sometimes-isnt/
 - line_id: A.4
   links:
-    - text: Reports of credit bias in Apple card credit limits spark negative press and investigation.
-      url: https://www.wired.com/story/the-apple-card-didnt-see-genderand-thats-the-problem/
+    - text: In six major cities, Amazon's same day delivery service excludes many predominantly black neighborhoods.
+      url: https://www.bloomberg.com/graphics/2016-amazon-same-day/
     - text: Facial recognition software is significanty worse at identifying people with darker skin.
       url: https://www.theregister.co.uk/2018/02/13/facial_recognition_software_is_better_at_white_men_than_black_women/
     - text: -- Related academic study.
-      url: http://proceedings.mlr.press/v81/buolamwini18a.html 
+      url: http://proceedings.mlr.press/v81/buolamwini18a.html
 - line_id: B.1
   links:
     - text: Personal and financial data for more than 146 million people was stolen in Equifax data breach.
@@ -60,7 +60,7 @@
   links:
     - text: Misleading chart shown at Planned Parenthood hearing distorts actual trends of abortions vs. cancer screenings and preventative services.
       url: https://www.politifact.com/truth-o-meter/statements/2015/oct/01/jason-chaffetz/chart-shown-planned-parenthood-hearing-misleading-/
-    - text: Georgia Dept. of Health graph of COVID-19 cases falsely suggests a steeper decline when dates are ordered by total cases rather than chronologically
+    - text: Georgia Dept. of Health graph of COVID-19 cases falsely suggests a steeper decline when dates are ordered by total cases rather than chronologically.
       url: https://www.vox.com/covid-19-coronavirus-us-response-trump/2020/5/18/21262265/georgia-covid-19-cases-declining-reopening
 - line_id: C.4
   links:
@@ -72,16 +72,18 @@
       url: https://www.bbc.com/news/magazine-22223190
 - line_id: D.1
   links:
-    - text: In six major cities, Amazon's same day delivery service excludes many predominantly black neighborhoods.
-      url: https://www.bloomberg.com/graphics/2016-amazon-same-day/
     - text: Variables used to predict child abuse and neglect are direct measurements of poverty, unfairly targeting low-income families for child welfare scrutiny.
       url: https://www.wired.com/story/excerpt-from-automating-inequality/
+    - text: Amazon scraps AI recruiting tool that showed bias against women.
+      url: https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G
     - text: Criminal sentencing risk asessments don't ask directly about race or income, but other demographic factors can end up being proxies.
       url: https://www.themarshallproject.org/2015/08/04/the-new-science-of-sentencing
     - text: Creditworthiness algorithms based on nontraditional criteria such as grammatic habits, preferred grocery stores, and friends' credit scores can perpetuate systemic bias.
       url: https://www.whitecase.com/publications/insight/algorithms-and-bias-what-lenders-need-know
 - line_id: D.2
   links:
+    - text: Apple credit card offers smaller lines of credit to women than men.
+      url: https://www.wired.com/story/the-apple-card-didnt-see-genderand-thats-the-problem/
     - text: Google Photos tags two African-Americans as gorillas.
       url: https://www.forbes.com/sites/mzhang/2015/07/01/google-photos-tags-two-african-americans-as-gorillas-through-facial-recognition-software/#12bdb1fd713d
     - text: With COMPAS, a risk-assessment algorithm used in criminal sentencing, black defendants are almost twice as likely as white defendants to be mislabeled as likely to reoffend.
@@ -92,8 +94,6 @@
       url: https://www.liebertpub.com/doi/pdf/10.1089/big.2016.0047
     - text: Google's speech recognition software doesn't recognize women's voices as well as men's.
       url: https://www.dailydot.com/debug/google-voice-recognition-gender-bias/
-    - text: Amazon scraps AI recruiting tool that showed bias against women.
-      url: https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G
     - text: Google searches involving black-sounding names are more likely to serve up ads suggestive of a criminal record than white-sounding names.
       url: https://www.technologyreview.com/s/510646/racism-is-poisoning-online-ad-delivery-says-harvard-professor/
     - text: -- Related academic study.