We are looking for the datasets that can be used for evaluating fairness models in machine learning on real data. Could you recommend a labeled dataset in which the labeling reveals some unfair decision process. E.g., unfair decisions in hiring, courts, healthcare etc.
I would recommend trying a disruptive approach, which follows the human process against unfair decisions:
-identify worst cases you could think of, with bias, discrimination, unfair decisions observed and documented.
-identify steps which led to such decisions
You can perform machine learning, deep learning, reinforcement learning to get to a bad systematically unfair "automated decision agent".
You can define a similarity measure between the "automated decision agent" you are trying to build, and these negative reference scenarios embedded in the unfair "automated decision agents" you have gathered.
As the learning system progresses towards the target "automated decision agent" you can iterate similarity measure computations with the bad unfair references, and when you get too close, an alarm is raised.
There are many ways to address reducing this risk at the next learning batch: introduce a repulsive gradient along the negative reference(s), build a Lagragian driving you away from it, etc...
Does it help you build a robust algorithm?
Have a look at https://www.kaggle.com/garethjns/titanicsexism-fairness-in-ml
I would recommend trying a disruptive approach, which follows the human process against unfair decisions:
-identify worst cases you could think of, with bias, discrimination, unfair decisions observed and documented.
-identify steps which led to such decisions
You can perform machine learning, deep learning, reinforcement learning to get to a bad systematically unfair "automated decision agent".
You can define a similarity measure between the "automated decision agent" you are trying to build, and these negative reference scenarios embedded in the unfair "automated decision agents" you have gathered.
As the learning system progresses towards the target "automated decision agent" you can iterate similarity measure computations with the bad unfair references, and when you get too close, an alarm is raised.
There are many ways to address reducing this risk at the next learning batch: introduce a repulsive gradient along the negative reference(s), build a Lagragian driving you away from it, etc...
Does it help you build a robust algorithm?
Dear Dr. Safro!
You presented a relevant challenge. I agree with Dr. Francesco. Hereby I want to add value to this discussion via a case-study approach (COVID-19):
1) Kolla Bhanu Prakash et al., International Journal of Emerging Trends in Engineering Research, 8(5), May 2020, 2199 - 2204 Open Access, Availableat:
Article Analysis, Prediction and Evaluation of COVID-19 Datasets usi...
2) Jie Sheng et al. (2020). COVID‐19 Pandemic in the New Era of Big Data Analytics: Methodological Innovations and Future Research Directions, British Journal of Management, 2 November 2020, Open Access, Available at:
https://onlinelibrary.wiley.com/doi/10.1111/1467-8551.12441
3) Mohammadreza Nemati et al. (2020). Machine-Learning Approaches in COVID-19 Survival Analysis and Discharge-Time Likelihood Prediction Using Clinical Data
Patterns, Volume 1, Issue 5, 14 August 2020, Open Access, Available at:
Article Machine-Learning Approaches in COVID-19 Survival Analysis an...
4) Sheetal, A., Feng, Z., & Savani, K. (2020). Using machine learning to generate novel hypotheses: Increasing optimism about COVID-19 makes people less willing to justify unethical behaviors. Psychological Science, 31, 1222–1235. Open Access Available at:
Article Using Machine Learning to Generate Novel Hypotheses: Increas...
Yours sincerely, Bulcsu Szekely
Dear Dr. Safro!
You presented a relevant challenge. I agree with Dr. Francesco and Bulcsu Szekely
Dear Dr. Safro!
You presented a relevant challenge. I agree with Dr. Francesco and Bulcsu Szekely
If the response is helpful then click on Recommend to help others. Thank you Thank you.
- Badges
- Science topic