milia.cluster.stability#

Resampling checks of whether a clustering is real or an artefact of one sample.

  • strength(): prediction strength (Tibshirani and Walther, 2005).

  • bootstrap(): per-cluster bootstrap Jaccard stability (Hennig, 2007).

  • subsample(): subsample agreement by adjusted Rand index (Ben-Hur et al., 2002).

  • calibrated(): each of the three against a structureless reference, so values compare across k.

  • audit(): the full trust audit of one chosen clustering.

Every function takes a clusterer estimator, cloned and refitted at each k.

Functions

audit(X, labels, estimator[, splits, ...])

Full trust audit of one clustering: internal validity plus prediction strength and stability.

bootstrap(X, labels, estimator[, resamples, ...])

Per-cluster bootstrap stability (Hennig, 2007).

calibrated(X, ks, estimator[, draws, ...])

Resampling checks against structureless references, so k values can be compared.

comembership(test, predicted)

Smallest, over test clusters, share of member pairs the prediction keeps together.

nearest(points, centroids, ids)

Label of each point's nearest centroid.

null(X[, kind, random_state])

Data of the same shape and range as X with no cluster structure.

onedirection(train, test, k, estimator)

Prediction strength of a training-half clustering on the test half.

strength(X, ks, estimator[, splits, ...])

Prediction strength (Tibshirani and Walther, 2005): does clustering one half predict the other?

subsample(X, estimator, k[, reps, fraction, ...])

Mean adjusted Rand agreement between clusterings of pairs of random subsamples (Ben-Hur et al., 2002).

trust(X, estimator, k, splits, resamples, ...)

The three resampling checks at one k: prediction strength, smallest-cluster stability and subsample ARI.

milia.cluster.stability.comembership(test, predicted)#

Smallest, over test clusters, share of member pairs the prediction keeps together.

Parameters:
  • test (ndarray) – Labels of a clustering of the test half.

  • predicted (ndarray) – Labels the training clustering assigns to the same points.

Return type:

float

Returns:

The co-membership agreement of the worst-kept cluster (1 when no cluster has two members).

Examples

>>> comembership(np.array([0, 0, 1, 1]), np.array([5, 5, 6, 7]))
0.0
milia.cluster.stability.nearest(points, centroids, ids)#

Label of each point’s nearest centroid.

Parameters:
  • points (ndarray) – Points, shape (n, d).

  • centroids (ndarray) – Centroids, shape (k, d).

  • ids (ndarray) – Label of each centroid.

Return type:

ndarray

Returns:

One label per point.

Examples

>>> nearest(np.array([[0.1], [0.9]]), np.array([[0.0], [1.0]]), np.array([7, 9])).tolist()
[7, 9]
milia.cluster.stability.onedirection(train, test, k, estimator)#

Prediction strength of a training-half clustering on the test half.

Parameters:
Return type:

float

Returns:

The co-membership agreement (0 when either half collapses to one cluster).

Examples

>>> from milia import make
>>> half = np.r_[np.zeros((5, 1)), np.full((5, 1), 9.0)]
>>> onedirection(half, half + 0.1, 2, make("clusterer", "kmeans"))
1.0
milia.cluster.stability.strength(X, ks, estimator, splits=10, random_state=None)#

Prediction strength (Tibshirani and Walther, 2005): does clustering one half predict the other?

The data is split in random halves; each half is clustered, and the test half’s clusters are checked against the assignments the training half’s centroids predict. Values near 1 mean the k-cluster structure replicates; 0.8 is the usual bar.

Parameters:
  • X (ArrayLike) – Data.

  • ks (Sequence[int]) – Candidate numbers of clusters.

  • estimator (BaseEstimator) – Clusterer, refitted at each k.

  • splits (int) – Random half splits, each scored in both directions.

  • random_state (int | None) – Seed.

Return type:

DataFrame

Returns:

One row per k with strength and strengthstd.

Examples

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0)
>>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20)
>>> from milia import make
>>> kmeans = make("clusterer", "kmeans", n_init=5)
>>> table = strength(X, [3, 5], kmeans, splits=3, random_state=0)
>>> bool(table.loc[0, "strength"] > table.loc[1, "strength"])
True
milia.cluster.stability.bootstrap(X, labels, estimator, resamples=100, random_state=None)#

Per-cluster bootstrap stability (Hennig, 2007).

The data is resampled with replacement and reclustered; each original cluster’s best Jaccard match among the bootstrap clusters is averaged over resamples. Above 0.75 is usually read as stable, below 0.5 as dissolved.

Parameters:
  • X (ArrayLike) – Data.

  • labels (ArrayLike) – The clustering being audited.

  • estimator (BaseEstimator) – Clusterer, refitted at the clustering’s k.

  • resamples (int) – Bootstrap resamples.

  • random_state (int | None) – Seed.

Return type:

dict[int, float]

Returns:

Cluster label to stability.

Raises:

ValueError – If labels does not have one entry per row.

Examples

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0)
>>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20)
>>> from milia import make
>>> kmeans = make("clusterer", "kmeans", n_init=5)
>>> min(bootstrap(X, truth, kmeans, resamples=5, random_state=0).values()) > 0.9
True
milia.cluster.stability.subsample(X, estimator, k, reps=15, fraction=0.8, shared=None, random_state=None, detail=False)#

Mean adjusted Rand agreement between clusterings of pairs of random subsamples (Ben-Hur et al., 2002).

Works on a precomputed distance matrix too: each subsample then takes the matching rows and columns.

Parameters:
  • X (ArrayLike) – Data, or a square distance matrix for a precomputed clusterer.

  • estimator (BaseEstimator) – Clusterer.

  • k (int) – Number of clusters.

  • reps (int) – Subsample pairs.

  • fraction (float) – Subsample size as a share of the rows.

  • shared (int | None) – Fewest shared points for a pair to count; max(10, k) by default.

  • random_state (int | None) – Seed.

  • detail (bool) – Also return every pair’s ARI and each subsample’s smallest cluster size.

Return type:

float | tuple[float, ndarray, ndarray]

Returns:

The mean ARI (NaN if no pair shared enough points), or with detail (mean, aris, minorities).

Raises:

ValueError – If fraction is not in (0, 1].

Examples

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0)
>>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20)
>>> from milia import make
>>> kmeans = make("clusterer", "kmeans", n_init=5)
>>> subsample(X, kmeans, 3, reps=4, random_state=0) > 0.9
True
milia.cluster.stability.null(X, kind='uniform', random_state=None)#

Data of the same shape and range as X with no cluster structure.

The reference that calibrated() compares trust metrics against.

Parameters:
  • X (ArrayLike) – Data.

  • kind (Literal['uniform', 'permutation']) – "uniform" draws within each feature’s range; "permutation" shuffles each feature independently, keeping its marginal distribution.

  • random_state (int | None) – Seed.

Return type:

ndarray

Returns:

Array of the shape of X.

Raises:

ValueError – If kind is unknown.

Examples

>>> X = np.arange(10.0).reshape(5, 2)
>>> sorted(null(X, "permutation", 0)[:, 0].tolist()) == sorted(X[:, 0].tolist())
True
milia.cluster.stability.trust(X, estimator, k, splits, resamples, reps, fraction, seed)#

The three resampling checks at one k: prediction strength, smallest-cluster stability and subsample ARI.

Parameters:
  • X (ndarray) – Data.

  • estimator (BaseEstimator) – Clusterer.

  • k (int) – Number of clusters.

  • splits (int) – Prediction-strength splits.

  • resamples (int) – Bootstrap resamples.

  • reps (int) – Subsample pairs.

  • fraction (float) – Subsample size.

  • seed (int) – Seed of all three.

Return type:

dict[str, float]

Returns:

strength, stability and subsample.

Examples

>>> from milia import make
>>> X = np.r_[np.zeros((15, 2)), np.full((15, 2), 9.0)] + np.random.default_rng(0).normal(size=(30, 2))
>>> sorted(trust(X, make("clusterer", "kmeans"), 2, 2, 2, 2, 0.8, 0))
['stability', 'strength', 'subsample']
milia.cluster.stability.calibrated(X, ks, estimator, draws=5, splits=5, resamples=25, reps=15, fraction=0.8, kind='uniform', random_state=0)#

Resampling checks against structureless references, so k values can be compared.

Raw prediction strength and stability fall as k grows even on data with no clusters, so a raw comparison across k favours small k. Each check is also run on draws structureless datasets (null()), and the excess over that null is reported.

Parameters:
  • X (ArrayLike) – Data.

  • ks (Sequence[int]) – Candidate numbers of clusters.

  • estimator (BaseEstimator) – Clusterer.

  • draws (int) – Structureless datasets per k.

  • splits (int) – Prediction-strength splits.

  • resamples (int) – Bootstrap resamples.

  • reps (int) – Subsample pairs.

  • fraction (float) – Subsample size.

  • kind (Literal['uniform', 'permutation']) – Kind of null (see null()).

  • random_state (int) – Seed.

Return type:

DataFrame

Returns:

One row per k with each metric, its null mean and nullstd, excess and z, plus excess, the mean excess over the three.

Raises:

ValueError – If draws < 1.

Examples

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0)
>>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20)
>>> from milia import make
>>> kmeans = make("clusterer", "kmeans", n_init=5)
>>> table = calibrated(X, [3], kmeans, draws=1, splits=2, resamples=2, reps=2)
>>> bool(table.loc[0, "strengthexcess"] > 0)
True
milia.cluster.stability.audit(X, labels, estimator, splits=10, resamples=100, random_state=0)#

Full trust audit of one clustering: internal validity plus prediction strength and stability.

Parameters:
  • X (ArrayLike) – Data.

  • labels (ArrayLike) – The clustering being audited.

  • estimator (BaseEstimator) – Clusterer used to refit during resampling.

  • splits (int) – Prediction-strength splits.

  • resamples (int) – Bootstrap resamples.

  • random_state (int) – Seed.

Return type:

dict[str, float]

Returns:

k, the internal metrics, strength, stability (smallest cluster) and meanstability.

Examples

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0)
>>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20)
>>> from milia import make
>>> kmeans = make("clusterer", "kmeans", n_init=5)
>>> row = audit(X, truth, kmeans, splits=2, resamples=3)
>>> row["k"], row["stability"] > 0.9
(3, True)