Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added checklist item for downstream bias mitigation #119

Merged
merged 5 commits into from
Dec 12, 2020

Conversation

glipstein
Copy link
Contributor

Context:
This PR focuses on a missing piece in A. Data Collection about considering ways to include data that will facilitate downstream bias measurement and mitigation, for instance by including demographic data for protected groups (e.g. race, gender). This seems to be a huge existing barrier to being able to trust ML systems not to codify or perpetuate existing inequities. It's also in line with Ibram Kendi's quote on "ignoring race" that "Terminating racial categories is potentially the last, not the first, step in the antiracist struggle."

Considerations for review:

  • Updated checklist in yml and docs
  • Updated examples in yml and docs
    • added apple pay example from list in issue Process for contributing to Examples / Resources docs #98
    • moved up facial recognition example from fairness across groups
    • moved amazon recruiting bias example from proxy discrimination to fairness across groups (since it seemed a better fit there, and the list otherwise had almost all google examples)
  • I also had come across this article "How Algorithms Discriminate Based on Data They Lack: Challenges, Solutions, and Policy Implications." Not sure there's a place for things like this, but dropping here for reference https://www.jstor.org/stable/10.5325/jinfopoli.8.2018.0078
  • If this is merged, will need to create an issue for the translated files to reflect the updates

Also

@glipstein glipstein requested review from pjbull and ejm714 October 7, 2020 04:07
@codecov
Copy link

codecov bot commented Oct 7, 2020

Codecov Report

Merging #119 (dd9a8b9) into master (60b7f8d) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #119   +/-   ##
=======================================
  Coverage   96.80%   96.80%           
=======================================
  Files           6        6           
  Lines         188      188           
=======================================
  Hits          182      182           
  Misses          6        6           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 60b7f8d...dd9a8b9. Read the comment docs.

@glipstein glipstein requested a review from jayqi October 7, 2020 04:17
Copy link
Contributor

@ejm714 ejm714 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggested rewordings and different locations for examples. @glipstein take a look at let me know your thoughts

@ejm714 ejm714 force-pushed the Add-A4-downstream-bias-mitigation branch from e639c88 to 69e30d5 Compare December 12, 2020 00:35
@ejm714 ejm714 self-requested a review December 12, 2020 00:38
Copy link
Member

@jayqi jayqi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@ejm714 ejm714 merged commit a7a794d into master Dec 12, 2020
@ejm714 ejm714 deleted the Add-A4-downstream-bias-mitigation branch December 12, 2020 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants