Gender Shades

Summary

In her TED talk, Joy explores how algorithmic bias, or the 'Coded Gaze,' affects facial recognition technology. Using her experience and her MIT thesis, Gender Shades, she demonstrates how commercial gender classification systems performed worse on darker-skinned women. Testing systems from IBM, Microsoft, and Face++, the results showed a significant bias towards lighter-skinned and male subjects. Joy emphasizes the need for more diverse data sets, transparency, and accountability in AI to avoid perpetuating bias in critical areas such as hiring and predictive analytics, urging society to ensure ethical and inclusive AI development.

Takeaways

• 😀 Joy researches how computers detect, recognize, and classify faces, focusing on algorithmic bias, which she refers to as the 'Coded Gaze.'
• 😀 In her TED talk, Joy shared her personal experience with facial recognition systems that worked well on lighter-skinned individuals but failed to detect or misgender her darker skin.
• 😀 After her TED talk, Joy tested her own image across various facial analysis demos, revealing issues with detection and misgendering.
• 😀 Joy discovered that the demos provided only two labels (male and female), ignoring gender identity and biological sex distinctions.
• 😀 Joy's project, 'Gender Shades,' was inspired by the desire to understand if the poor results were due to her unique features or part of a broader pattern.
• 😀 The 'Gender Shades' project aimed to evaluate gender classification systems' performance across different skin types and genders using a dataset of over a thousand images of parliament members.
• 😀 The dataset included a mix of African and European countries to assess how systems performed on lighter and darker skin types.
• 😀 Joy evaluated three companies: IBM, Microsoft, and Face++, testing their gender classification accuracy across the dataset.
• 😀 Microsoft achieved the highest accuracy at 94% overall, but all companies showed better performance with male and lighter-skinned individuals.
• 😀 Analysis revealed all companies performed worse on darker females, with IBM showing the largest accuracy gap (34%) between lighter males and darker females.
• 😀 Joy was shocked to see that commercial products misgendered one in three women of color, with a near-coin toss accuracy for darker-skinned women.
• 😀 The project highlighted the impact of a lack of diversity in training datasets and the need for better separation of results across gender and skin type.
• 😀 The failure of machine learning models in this context has broader implications for other AI systems, such as facial recognition and predictive analytics, which can influence important life decisions like hiring or loans.
• 😀 Joy stressed the need for more transparency and accountability in AI systems, as they are vulnerable to bias and misuse, particularly in critical areas such as civil rights and gender equity.
• 😀 Joy concluded that without ethical and inclusive AI, we risk undoing the progress made in civil rights and gender equity, under the guise of 'machine neutrality.'

Q & A

What is the concept of the 'Coded Gaze' introduced by Joy?

The 'Coded Gaze' refers to algorithmic bias in facial recognition systems, where systems are often more accurate for lighter-skinned individuals, leading to misidentification or inaccurate results for people with darker skin tones.

What issue did Joy face when testing a facial recognition system?

Joy discovered that the facial recognition system performed well on her lighter-skinned friend's face but struggled to detect her own face unless she wore a white mask, highlighting the system's bias towards lighter skin tones.

What was Joy's experience after her TED talk was posted?

After her TED talk was posted, Joy tested her speaker image across various facial analysis demos. Two of these demos failed to detect her face, while the others misgendered her, highlighting the inadequacies in the systems.

How did Joy explore the issue of gender classification in facial recognition?

Joy embarked on a project called 'Gender Shades' to assess how gender classification systems performed across different people's faces, focusing on the impact of skin type and gender identity on the results.

What dataset did Joy use for her 'Gender Shades' project?

For the 'Gender Shades' project, Joy created a dataset of over a thousand images of parliament members from three African and three European countries, chosen to represent a range of skin types and genders.

Which companies' facial recognition systems did Joy evaluate in her study?

Joy evaluated the facial recognition systems of IBM, Microsoft, and Face++, a company with access to one of the largest datasets of Chinese faces.

What were the general results of the facial recognition tests Joy conducted?

The systems generally performed better on males than females and better on lighter-skinned individuals than darker-skinned individuals. Microsoft achieved the highest accuracy, but all companies showed biases, especially toward darker-skinned females.

Which company performed best in detecting faces based on the study's results?

Microsoft performed best in the study, achieving 94% accuracy across the entire dataset.

What did Joy discover about the performance differences between lighter and darker-skinned subjects?

Joy found that all companies performed worse on darker-skinned females, with significant error rate differences, particularly in IBM's results, where there was a 34% difference in error rates between lighter males and darker females.

What broader issue did Joy highlight with her research on facial recognition systems?

Joy emphasized the lack of diversity in training data and the absence of accuracy results based on factors like gender and skin tone. She warned that these issues could lead to biased applications of facial recognition and other predictive systems, potentially causing harm in areas like hiring, loan granting, and other AI-driven decisions.