Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

Build Fairness Library #5105

Open
12 of 15 tasks
ArjunSubramonian opened this issue Apr 8, 2021 · 38 comments
Open
12 of 15 tasks

Build Fairness Library #5105

ArjunSubramonian opened this issue Apr 8, 2021 · 38 comments
Assignees
@ArjunSubramonian

Comments

@ArjunSubramonian
Copy link
Contributor

ArjunSubramonian commented Apr 8, 2021 •
edited
Loading

Motivation: As models and datasets become increasingly large and complex, it is critical to evaluate the fairness of models according to multiple definitions of fairness and mitigate bias in learned representations. This library aims to make fairness metrics, fairness training tools, and bias mitigation algorithms extremely easy to use and accessible to researchers and practitioners of all levels.

Success Criteria:

  • Create a fairness library, and apply it to the Textual Entailment model, publishing an analysis for where the present models fall short and where they should improve.
  • Write a blog post and guide chapter and add a model and demo for the implementations of the fairness metrics and bias mitigation algorithms, and explain the broader impact.

Milestones

Implement the following:

Fairness Metrics

  • Independence, Separation, Sufficiency
  • Sparse Annotations for Ground-Truth
  • Dataset Bias Amplification, Model Bias Amplification

Training-Time Fairness Algorithms (with and without Demographics):

  • Through Adversarial Learning (with Demographics)
  • Minimax (without Demographics)
  • Repeated Loss Minimization (without Demographics)

Bias Mitigation Algorithms:

  • Linear projection, Hard debiasing, OSCaR, Iterative Null Space Projection
  • Bias direction methods: Classification Normal, Two Means, Paired PCA, PCA
  • Contextualized word embeddings

Bias Metrics:

  • WEAT, Embedding Coherence Test, NLI

Communication:

  • blog post
  • guide chapter
  • demo
  • contribute binary gender bias-mitigated model for SNLI to allennlp-models
  • contribute binary gender bias-mitigated model for SNLI to demos
@ArjunSubramonian ArjunSubramonian self-assigned this Apr 8, 2021
@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

28 similar comments
@github-actions
Copy link

github-actions bot commented Jul 2, 2021

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Aug 2, 2021

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Dec 9, 2021

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Jan 7, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Feb 7, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Mar 8, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Apr 6, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented May 5, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Jun 3, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Jul 5, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

github-actions bot commented Aug 3, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

8 similar comments
@github-actions
Copy link

github-actions bot commented Sep 1, 2022

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

@github-actions
Copy link

@ArjunSubramonian this is just a friendly ping to make sure you haven't forgotten about this issue 😜

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@ArjunSubramonian ArjunSubramonian

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ArjunSubramonian