Explainable Artificial Intelligence (VTR UGE 21), day 4 of 5

Reliability of Explanations

Prof. Dr. Utku Kose, Süleyman Demirel University

Reliability lab

Part A runs the deletion and insertion tests in the browser. Part B classifies four reliability diagrams by eye. Part C computes two fairness measures by hand.

Part A: Deletion and insertion

A model of eight features explains twenty cases. Choose an explanation method and run the deletion and insertion tests: Features that support the prediction are removed first, or added first, and the probability of the predicted class is recorded. A faithful explanation makes the deletion curve fall fast and the insertion curve rise fast. Every run is compared with a random ranking, which any method must beat.

Scoreboard

RunExplanationNoiseDeletion AUCInsertion AUC FaithfulnessBeats random?

Open in ColabContinue in Colab, section 6: the same test with SHAP, LIME and a random forest on real data.

Part B: Calibration by eye

Each diagram plots the observed frequency against the mean predicted probability in bins, for one model. The dashed diagonal is perfect calibration. Decide what each model does and press Check the diagrams.

Open in ColabContinue in Colab, section 1: reliability diagrams of three real models, before and after recalibration.

Part C: Fairness measures by hand

A credit model was applied to 100 applicants of each group. Compute the two fairness measures of the lecture from the counts and press Check my values.

GroupApplicantsWould repayApproved, of those who would repay Approved, of those who would not
A10060455
B10050182

Open in ColabContinue in Colab, section 5: a full fairness audit and the proxy that carries the gap.