¶ SHAP: SHapley Additive exPlanations
SHAP (SHapley Additive exPlanations) is a general framework for explaining machine learning model predictions using concepts from cooperative game theory. It connects local explanation methods with Shapley values, providing a principled way to attribute a model’s output to its input features.
The library is available as an open-source Python package and supports many model types, including tree ensembles, deep learning models, NLP pipelines, linear models, and fully model-agnostic explainers.
¶ Model description
SHAP is not a predictive model itself, but an explainability framework for interpreting the outputs of machine learning models. Its central idea is to assign each feature a contribution value based on how much that feature changes the prediction relative to a baseline. These contributions are computed using the theory of Shapley values, which come from game theory and describe fair credit allocation among multiple participants.
A major strength of SHAP is that it unifies several explanation approaches under a single theoretical framework while also offering specialized algorithms for different model families. For example, TreeExplainer provides fast exact explanations for tree ensembles, DeepExplainer and GradientExplainer support neural networks, and KernelExplainer offers a model-agnostic method that can be applied to almost any prediction function.
Key traits of SHAP:
- Game-theoretic explanations: Uses Shapley values to attribute predictions to features.
- Model flexibility: Supports tree models, neural networks, NLP pipelines, linear models, and arbitrary black-box functions.
- Local explanations: Explains individual predictions in terms of feature contributions.
- Global interpretation tools: Aggregates local explanations to show overall feature importance and interaction patterns.
- Rich visualization ecosystem: Includes waterfall plots, force plots, scatter plots, beeswarm plots, bar plots, text explanations, and image explanations.
¶ SHAP
Figure 1 (conceptually based on the SHAP documentation) illustrates the workflow of SHAP explanations:
- A trained model and input data are provided to a SHAP explainer.
- The explainer computes feature contribution values for each prediction.
- These values show how features push the prediction higher or lower relative to a baseline prediction.
- The explanations can be visualized locally for single predictions or globally across an entire dataset.
- Different explainer types are available depending on the model family, such as TreeExplainer, DeepExplainer, GradientExplainer, LinearExplainer, and KernelExplainer.
¶ Intended uses & limitations
SHAP is intended for:
- Explaining individual model predictions in an interpretable and theoretically grounded way.
- Understanding feature importance across a dataset.
- Debugging and validating models by revealing which features drive predictions.
- Supporting transparency and trust in machine learning workflows across tabular, image, text, and other domains.
Limitations:
- SHAP is an explainability method, not a predictive model, so it depends entirely on the behavior of the underlying model.
- Some explainer types, especially KernelExplainer, can be computationally expensive on large datasets or complex models.
- Explanations are only as meaningful as the underlying data and model assumptions, and should not automatically be interpreted as causal effects.
- Different SHAP explainers use different approximation strategies, so explanation quality and runtime can vary depending on the model class.
¶ BibTeX entry and citation info
@inproceedings{lundberg2017unified,
title={A Unified Approach to Interpreting Model Predictions},
author={Lundberg, Scott M and Lee, Su-In},
booktitle={Advances in Neural Information Processing Systems},
year={2017}
}