Quantifying Fairness and Unintended Bias in Machine Learning Models
Problem
When a machine learning model helps decide who gets a loan, an interview, an insurance quote or a medical follow-up, fairness is not optional. Models learn from historical data, and historical data often carries correlations between a sensitive attribute, such as a demographic group, and the outcome being predicted. A model can pick up those correlations and repeat them, even when nobody intended it to.
Researchers have proposed many algorithms that try to make deep learning predictions fairer. But, as the paper points out, there was no common framework for analysing and comparing what these methods can actually do. Each was tested on its own data, with its own setup and its own choice of fairness metric. There is also no single, universally accepted definition of fairness, so a method that looks good on one metric may look very different on another. For a business, that makes it hard to choose a method, defend the choice, or know how a model will behave when the data is skewed.
Also applicable to
- Recruitment and HR: screening and ranking models that must treat candidates fairly.
- Financial services and banking: credit, lending and fraud models under regulatory scrutiny.
- Insurance: pricing and underwriting models that must avoid unfair discrimination.
- Healthcare: risk and triage models that should perform equally well across patient groups.
- Public sector: eligibility and allocation decisions that require transparency.
Solution / Approach
The Elementera team's research on fairness measurement appears in Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness Metrics, a paper co-authored by Elementera co-founder Soroush Mehri with Charan Reddy, Deepak Sharma, Adriana Romero Soriano, Samira Shabanian and Sina Honari, presented in the NeurIPS 2021 Datasets and Benchmarks Track.
The work builds the common framework the field was missing. According to the paper and its open-source code, it:
- Introduces a dataset built for testing bias. CI-MNIST is a variant of the MNIST digits dataset in which the authors introduce different, controlled types of correlation between dataset features and the eligibility criterion, so bias can be dialled up and studied.
- Adds a real-world counterpart. The UCI Adult dataset is used with an added sensitive feature correlated with eligibility.
- Proposes several biased evaluation setups, including imbalanced and correlated settings, that reflect how skewed data appears in practice.
- Evaluates recent debiasing algorithms side by side in one framework, including LAFTR (in several variants), CFAIR and FFVAE, against standard neural network baselines.
- Measures results with multiple established fairness metrics, because no one metric captures fairness on its own.
Why this matters for a company using AI: fairness claims are only as good as the measurement behind them. A shared benchmark with controlled bias lets a team see how a mitigation method behaves as data becomes more skewed, and on which fairness definition it helps, before the model affects real people.
Impact
The benchmark gives practitioners and researchers a rigorous, common way to compare bias mitigation methods for deep learning, with the code and dataset tools released publicly so experiments can be reproduced. For businesses, this research means:
- Evidence-based model choices: compare debiasing methods on the same data and metrics.
- Stress-testing before deployment: see how methods behave as bias in the data increases.
- Clearer reporting: measure fairness on several definitions rather than one convenient number.
- Measurement expertise: the Elementera team's work on how to measure AI systems rigorously, and not only how to build them, carries through to how Elementera measures AI behaviour today.
Technologies
- Representation learning: training models to learn features that support fair predictions.
- Bias mitigation algorithms: LAFTR, CFAIR and FFVAE, compared in one framework.
- Fairness metrics: multiple established definitions applied side by side.
- Synthetic and real-world benchmarks: CI-MNIST with controlled correlations, and the Adult dataset.
Read the paper: Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness Metrics (NeurIPS 2021 Datasets and Benchmarks). Code: github.com/charan223/FairDeepLearning.
Ex-Microsoft. 10+ years of experience in AI/ML, MSc from Université de Montréal (Mila). US patent holder.
AI/ML scientist with 20+ years of research & industry experience. Ex-Google DeepMind, MIT, Harvard, Mila. McGill-Mila PhD.