Would you deploy this model?
A neural network classifies 569 breast biopsies as benign or malignant. Each prediction comes from a model that never saw that biopsy during training. Move the threshold and watch what a 97 percent accuracy figure hides.
Wrong and sure
The most confident mistakes at the current threshold. A confident error looks exactly like a confident correct answer.
Does 0.9 mean 90 percent?
Why this risk, for this patient?
A gradient boosting model predicts coronary artery disease from 13 clinical variables of the UCI Cleveland data. Edit any value. The Shapley values are computed exactly, over all 8,192 coalitions of variables, against 16 reference patients.
SHAP: how each value moves the risk
LIME: a local linear approximation
What would change the prediction?
Shapley values by hand
Pick three variables. They become players in a cooperative game whose payoff is the predicted risk. An absent player takes its values from 16 reference patients and the results are averaged; every other variable keeps this patient's value. The Shapley value is a player's marginal contribution averaged over every order of arrival.
Every coalition
Every order of arrival
Spoof the sensor
Each example is one real hour of continuous glucose monitoring from an older adult with type 1 diabetes, and in each case glucose fell below 70 mg/dL within the next 30 minutes. The alert model warned correctly. Drag the reported readings, or run the attacker, and try to silence the warning while staying within 15 mg/dL of every true reading.