Amazon’s internal machine learning system has been dropped due to its bias against female applicants. The software consistently downgraded female candidates compared to male ones.
The project was introduced to speed up the hiring process. However, when the company realized that the software was not producing gender-neutral results it was modified to neutralize the bias. However, since it could not be ensured that other biases had not crept into the program, the program was scrapped in its entirety last year.
It should be realized AI systems are based on historical data, thus it should not be a surprise that they often perpetuate existing biases. Thus the existing bias against women was perpetuated due to its existence in the data fed to the AI systems. According to Reuters,
Amazon’s program penalized applicants who attended all-women’s colleges, as well as any resumes that contained the word “women’s” (as might appear in the phrase “women’s chess club”).
https://www.youtube.com/watch?v=_NdCuzTuWdo
