Estimands: Shapley vs Banzhaf

An estimand is the quantity your attribution procedure targets. In nlp-shap, estimands are explicit types — not implicit behaviour of an estimator. Monte Carlo coalition averaging does not automatically estimate Shapley values; unweighted means over sampled coalitions estimate the Banzhaf index unless Shapley weights are applied.

For Shapley axioms and uniqueness, see Shapley values and axioms. For cooperative-game foundations, see Cooperative games.

Shapley value (estimand)

The Shapley estimand uses factorial coalition weights:

\[\phi_i(v) = \sum_{S \subseteq N \setminus \{i\}} \frac{|S|!\,(|N|-|S|-1)!}{|N|!} \bigl(v(S \cup \{i\}) - v(S)\bigr)\]
\[w_{\mathrm{Shapley}}(k, n) = \frac{k!\,(n-k-1)!}{n!}\]

Implemented in ShapleyAggregator.

Axiomatic summary: efficiency, symmetry, dummy player, additivity — uniquely characterizing \(\phi\) among cooperative-game allocation rules (Shapley, 1953). See Shapley values and axioms for formal statements.

Banzhaf index (estimand)

The Banzhaf index (also called the Banzhaf–Coleman power index in voting theory) uses uniform weights over coalitions \(S \subseteq N \setminus \{i\}\):

\[\beta_i(v) = \frac{1}{2^{|N|-1}} \sum_{S \subseteq N \setminus \{i\}} \bigl(v(S \cup \{i\}) - v(S)\bigr)\]

Equivalently, \(w_{\mathrm{Banzhaf}}(k, n) = 2^{-(n-1)}\) for every coalition size \(k\).

Implemented in BanzhafAggregator.

Interpretation: \(\beta_i\) measures the average marginal contribution across all coalitions, or — in threshold games — the probability that player \(i\) is pivotal when each player independently joins with probability \(1/2\). Banzhaf indices need not satisfy efficiency: \(\sum_i \beta_i(v) \neq v(N)\) in general.

Banzhaf (1965) introduced the index in weighted-voting analysis; Dubey & Shapley (1979) developed its mathematical properties on the space of characteristic functions.

When Shapley and Banzhaf diverge

For additive games \(v(S) = \sum_{i \in S} a_i\), both indices recover \(a_i\).

For non-additive games they can disagree substantially. A standard example is the majority/threshold game with \(|N|=3\) and \(v(S)=1\) iff \(|S| \ge 2\), else \(0\):

Estimand

Per-player value

Shapley

\(2/6 \approx 0.333\)

Banzhaf

\(0.5\)

The test suite locks this divergence on identical coalition samples.

Choosing an estimand in practice

Prefer Shapley when:

  • Attributions must sum to the total utility gain (efficiency).

  • You need the classical fairness axioms for stakeholder review.

  • Reporting aligns with SHAP literature and local accuracy guarantees.

Prefer Banzhaf when:

  • You care about swing / pivotal influence per token.

  • You want uniform coalition averaging without factorial weights.

  • You explicitly study voting-power-style sensitivity.

Regardless of choice, label outputs honestly. nlp-shap separates estimands from estimators so archives and papers never misreport Banzhaf output as Shapley.

Monte Carlo and estimand confusion

A common pitfall in explainability codebases is to:

  1. Sample coalitions \(S_1, \dots, S_m\) with an estimator

  2. Average marginal contributions without Shapley weights

  3. Label the result “Shapley”

Step 2 estimates a Banzhaf-style quantity (uniform coalition average), not \(\phi\). Strumbelj & Kononenko (2014) discuss coalition-based feature contributions; Fryer et al. (2021) analyze when Shapley axioms justify feature selection in ML. Always apply the estimand aggregator that matches the quantity you intend to report.

Labelling in nlp-shap

Every ExplainResult carries an Estimand label. Run archives persist the same label in RunManifestPayload so downstream analysis and compliance reviews can verify which estimand was used.

The EstimandAggregator protocol defines the aggregation contract; concrete implementations live in nlp_shap.estimation.estimands.

References

  • Shapley, L. S. (1953). A value for n-person games. In H. W. Kuhn & A. W. Tucker (Eds.), Contributions to the Theory of Games II (pp. 307–317). Princeton University Press. DOI:10.1515/9781400881970-018

  • Banzhaf, J. F. (1965). Weighted voting doesn’t work: A mathematical analysis. Rutgers Law Review, 19(2), 317–343.

  • Dubey, P., & Shapley, L. S. (1979). Mathematical properties of the Banzhaf power index. Mathematics of Operations Research, 4(2), 99–131. DOI:10.1287/moor.4.2.99

  • Strumbelj, E., & Kononenko, I. (2014). Explaining prediction models and individual predictions with feature contributions. Knowledge-Based Systems, 41, 78–84. DOI:10.1016/j.knosys.2013.10.012

  • Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30. ML Anthology; arXiv:1705.07874

  • Fryer, D. V., Strümke, I., & Nguyen, H. D. (2021). Shapley values for feature selection: The good, the bad, and the axioms. IEEE Access, 9, 144352–144360. DOI:10.1109/ACCESS.2021.3115252; arXiv:2102.10936

  • Ethayarajh, K., & Jurafsky, D. (2021). Attention flows are Shapley value explanations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) (pp. 49–54). ACL Anthology; arXiv:2105.14652