Shapley vs Banzhaf on toy cooperative games

Business context: Attribution reports shown to legal, risk, and product teams must name the estimand being reported. Shapley and Banzhaf answer different fairness questions; mislabelling one as the other breaks audit trails and can invalidate compliance reviews.

Goal: Walk through every public estimand API in nlp-shap 0.1.1 — aggregators, coalition weights, labelled results, manifest wire format, and plugin resolution.

Prerequisites: nlp-shap 0.1.1+, Python 3.12.

Theory: See the estimands theory page.

Expected output: Divergence on a majority game; agreement on an additive game; ExplainResult and RunManifest carry the estimand label end to end.

from itertools import product

from nlp_shap import (
    BanzhafAggregator,
    Estimand,
    ExplainResult,
    RunManifest,
    ShapleyAggregator,
    parse_manifest,
)
from nlp_shap.domain.estimands import estimand_to_wire
from nlp_shap.plugins import PluginGroup, PluginRegistry

Why the estimand label matters

Regulators and enterprise buyers increasingly ask which cooperative-game value was computed, not only the attribution vector. nlp-shap treats the estimand as first-class metadata so archived runs remain interpretable months later.

Majority game — non-additive divergence

Three players; payoff is 1 when at least two are present. Interaction between players means Shapley and Banzhaf generally disagree — a realistic sanity check before trusting LLM attributions.

NUM_PLAYERS = 3

masks = [tuple(bits) for bits in product([False, True], repeat=NUM_PLAYERS)]


def majority_payoff(mask: tuple[bool, ...]) -> float:
    return 1.0 if sum(mask) >= 2 else 0.0


payoffs = [majority_payoff(mask) for mask in masks]
print("coalitions:", len(masks))
print("first payoffs:", payoffs[:4])

coalitions: 8
first payoffs: [0.0, 0.0, 0.0, 1.0]

Aggregator labels and coalition weights

Each aggregator exposes its estimand and the coalition weight formula it applies. Weights explain why identical samples can yield different player scores.

shapley_agg = ShapleyAggregator()
banzhaf_agg = BanzhafAggregator()

print("labels:", shapley_agg.estimand, banzhaf_agg.estimand)

for coalition_size in range(NUM_PLAYERS):
    print(
        f"size {coalition_size}:",
        "shapley",
        round(shapley_agg.coalition_weight(coalition_size, NUM_PLAYERS), 4),
        "banzhaf",
        round(banzhaf_agg.coalition_weight(coalition_size, NUM_PLAYERS), 4),
    )

labels: shapley banzhaf
size 0: shapley 0.3333 banzhaf 0.25
size 1: shapley 0.1667 banzhaf 0.25
size 2: shapley 0.3333 banzhaf 0.25

Aggregate identical samples

Product teams comparing vendors should demand this check: same coalitions, same payoffs, different estimands → different attributions unless the game is additive.

shapley = shapley_agg.aggregate(masks, payoffs)
banzhaf = banzhaf_agg.aggregate(masks, payoffs)

print("Shapley:", shapley)
print("Banzhaf:", banzhaf)
assert shapley != banzhaf

Shapley: [0.3333333333333333, 0.3333333333333333, 0.3333333333333333]
Banzhaf: [0.5, 0.5, 0.5]

Additive game — when estimands agree

For linear value functions, both estimands recover the underlying coefficients. This is the case where estimand choice is less consequential — useful when explaining the feature to non-technical stakeholders.

ADDITIVE_PLAYERS = 2
coefficients = (1.0, 2.0)
additive_masks = [
    tuple(bits) for bits in product([False, True], repeat=ADDITIVE_PLAYERS)
]
additive_payoffs = [
    sum(
        coef for coef, present in zip(coefficients, mask, strict=True) if present
    )
    for mask in additive_masks
]

additive_shapley = shapley_agg.aggregate(additive_masks, additive_payoffs)
additive_banzhaf = banzhaf_agg.aggregate(additive_masks, additive_payoffs)

print("additive Shapley:", additive_shapley)
print("additive Banzhaf:", additive_banzhaf)
print("match:", additive_shapley == additive_banzhaf == list(coefficients))

additive Shapley: [1.0, 2.0]
additive Banzhaf: [1.0, 2.0]
match: True

Label explain results

ExplainResult stores the estimand beside the attribution vector so dashboards and exports cannot silently swap definitions.

shapley_result = ExplainResult(
    estimand=Estimand.SHAPLEY,
    values=tuple(shapley),
)
banzhaf_result = ExplainResult(
    estimand=Estimand.BANZHAF,
    values=tuple(banzhaf),
)

print(shapley_result.estimand, shapley_result.values)
print(banzhaf_result.estimand, banzhaf_result.values)
assert shapley_result.estimand != banzhaf_result.estimand

shapley (0.3333333333333333, 0.3333333333333333, 0.3333333333333333)
banzhaf (0.5, 0.5, 0.5)

Persist estimand metadata in run manifests

Run archives serialize the estimand as a stable wire value for JSON manifests — the hook compliance teams use to verify what was shipped to production.

manifest = RunManifest(estimand=Estimand.SHAPLEY, run_id="toy-majority")
payload = manifest.to_dict()
restored = parse_manifest(payload)

print("wire:", payload)
print("round-trip estimand:", restored.estimand)
print("estimand_to_wire:", estimand_to_wire(Estimand.BANZHAF))
assert restored.estimand is Estimand.SHAPLEY
assert restored.run_id == "toy-majority"

wire: {'estimand': 'shapley', 'run_id': 'toy-majority'}
round-trip estimand: shapley
estimand_to_wire: banzhaf

Resolve estimand plugins from packaging entry points

Downstream apps select shapley or banzhaf by name through the plugin registry — the same mechanism production configs will use.

registry = PluginRegistry()
registry.load_entry_points(PluginGroup.ESTIMANDS)

print("registered:", registry.names(PluginGroup.ESTIMANDS))
resolved_shapley = registry.resolve(PluginGroup.ESTIMANDS, "shapley")
resolved_banzhaf = registry.resolve(PluginGroup.ESTIMANDS, "banzhaf")

type(resolved_shapley).__name__, type(resolved_banzhaf).__name__

registered: ('banzhaf', 'shapley')

('ShapleyAggregator', 'BanzhafAggregator')