Explainable AI explained! | #5 Counterfactual explanations and adversarial attacks

Summary

In this video, the presenter explores counterfactual explanations, a powerful approach to understanding machine learning model predictions. By using a stroke prediction example, the concept of counterfactuals is illustrated, where a user is informed about changes they can make to alter their risk assessment. The video delves into methods for calculating counterfactuals, including white box and black box approaches, and showcases practical implementation using the Dice library in Python. This local explainability method offers valuable insights into how minimal changes in input features can lead to different outcomes, making it a crucial tool in explainable AI.

Takeaways

• 😀 Counterfactual explanations help individuals understand what changes can be made to achieve a different outcome in machine learning predictions.
• 😀 In the stroke prediction example, a counterfactual suggested that John could lower his stroke risk by reducing his BMI to a specific value.
• 😀 A counterfactual is defined as the smallest change in input features that leads to a different prediction, highlighting its effectiveness in explainable AI.
• 😀 The concept of counterfactuals has roots in psychology and has been adapted for use in machine learning since 2017.
• 😀 Counterfactuals are often calculated using optimization problems that seek to find minimal changes to input data to achieve a desired output.
• 😀 There are two main approaches to calculating counterfactuals: white box (accessing model internals) and black box (relying on output queries).
• 😀 Using model internals allows for more efficient calculations through methods like gradient descent, while black box approaches may require extensive querying.
• 😀 The Python library Dice provides tools for generating diverse counterfactual explanations by perturbing input data intelligently.
• 😀 Counterfactuals can yield multiple valid solutions, reflecting the Rashomon effect, where different explanations can all be considered reasonable.
• 😀 Ensuring the feasibility of counterfactuals is essential, and libraries like Dice allow users to specify constraints to generate realistic changes.

Q & A

What are counterfactuals in the context of machine learning?

Counterfactuals are scenarios that suggest what changes a specific input feature would need to undergo to alter the output prediction of a machine learning model. For instance, if a patient is predicted to have a high risk of stroke, a counterfactual might indicate that reducing their body mass index (BMI) could lower that risk.

How do counterfactual explanations differ from feature importance methods?

Counterfactual explanations provide actionable insights on how to change input features to achieve a different outcome, whereas feature importance methods, like SHAP and LIME, primarily indicate which features most significantly affect predictions without suggesting modifications.

What is the main goal of counterfactual explanations?

The main goal of counterfactual explanations is to inform individuals about the minimal changes they can make to their attributes to achieve a more favorable prediction outcome.

Can you explain the Rashomon effect in relation to counterfactuals?

The Rashomon effect refers to the idea that there can be multiple valid counterfactuals that lead to the same desired outcome. For example, a loan could be approved by either increasing income or improving credit history, indicating that there are various paths to the same result.

What is the significance of using counterfactuals in AI safety?

In AI safety, counterfactuals are significant because they help identify and mitigate vulnerabilities in machine learning models by exploring how small changes in input can lead to drastically different predictions, thus informing the development of more robust models.

What are the two main approaches to computing counterfactuals?

The two main approaches to computing counterfactuals are white box approaches, where the internal workings of the model are known and can be leveraged (e.g., using gradients), and black box approaches, which rely on querying the model multiple times without accessing its internal parameters.

How does the 'DICE' library help in generating counterfactuals?

The DICE (Diverse Counterfactual Explanations) library facilitates the generation of multiple diverse counterfactual explanations by using various optimization methods to perturb input features while considering feasibility constraints.

What is the role of the distance function in generating counterfactuals?

The distance function in generating counterfactuals helps ensure that the modified inputs remain as similar as possible to the original inputs, effectively minimizing the changes required to achieve a different prediction.

What is an example of a counterfactual explanation for a patient at risk of stroke?

An example counterfactual for a patient named John, predicted to have a 90% stroke risk, could state: 'If you reduce your body mass index (BMI) to 25, your stroke risk prediction would drop to 70%.'

What was the historical context of counterfactuals before their application in machine learning?

Counterfactuals have been used in psychology for a long time, with a theoretical definition presented by philosopher David Lewis in 1973. Their application in machine learning started gaining attention with research published in 2017.