milia.cluster.ablation#
Ablation of a clustering pipeline, one axis at a time and in pairs.
Hold the settled recipe fixed, vary one step (scaler, feature set, reducer, number of components), and measure what changes: internal validity, cluster balance, agreement with the reference labelling, and recovery of a known group.
Functions
|
Vary one pipeline step and measure the clustering at each setting. |
|
Spread and mode of the k chosen by several algorithms. |
|
A metric over every pair of settings of two steps (e.g. feature set by reducer). |
|
The ablation metrics of one labelling. |
|
Rank the rows within each column of an |
|
The recommended k and the resulting cluster sizes for each clusterer on the same input. |
- milia.cluster.ablation.measure(X, labels, k, reference, cohort)#
The ablation metrics of one labelling.
- Parameters:
- Return type:
- Returns:
silhouette,balance,smallestand, when given,ari,nmiandrecovery.
Examples
>>> import numpy as np >>> X = np.r_[np.zeros((5, 2)), np.ones((5, 2))] >>> labels = np.repeat([0, 1], 5) >>> measure(X, labels, 2, labels, labels == 0)["recovery"] 1.0
- milia.cluster.ablation.ablate(values, build, estimator, k, reference=None, cohort=None, axis='value')#
Vary one pipeline step and measure the clustering at each setting.
Hold the settled recipe fixed, change one step (scaler, feature set, reducer, number of components), and record what moves.
- Parameters:
values (
Sequence[Hashable]) – Settings of the varied step, e.g. scaler names.build (
Callable[[Hashable],ArrayLike]) – Builds the clustering input for one setting.estimator (
BaseEstimator) – Clusterer.k (
int) – Number of clusters.reference (
ArrayLike|None) – Labels of the settled recipe, forari.cohort (
ArrayLike|None) – Boolean mask of a known group, forrecovery.axis (
str) – Name of the varied step, used as the first column.
- Return type:
- Returns:
One row per setting.
Examples
>>> import numpy as np >>> rng = np.random.default_rng(0) >>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0) >>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20) >>> from milia import make >>> kmeans = make("clusterer", "kmeans", n_init=5) >>> from milia.features.scale import scale >>> table = ablate(["none", "minmax"], lambda name: scale(X, name), kmeans, 3, truth, axis="scaler") >>> table["ari"].tolist() [1.0, 1.0]
- milia.cluster.ablation.interaction(first, second, build, estimator, k, metric='silhouette')#
A metric over every pair of settings of two steps (e.g. feature set by reducer).
- Parameters:
first (
Sequence[Hashable]) – Settings of the first step (rows).second (
Sequence[Hashable]) – Settings of the second step (columns).build (
Callable[[Hashable,Hashable],ArrayLike]) – Builds the clustering input for a pair of settings.estimator (
BaseEstimator) – Clusterer.k (
int) – Number of clusters.
- Return type:
- Returns:
Table of the metric, first step by second step.
Examples
>>> import numpy as np >>> rng = np.random.default_rng(0) >>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0) >>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20) >>> from milia import make >>> kmeans = make("clusterer", "kmeans", n_init=5) >>> interaction(["raw"], [1.0, 2.0], lambda a, b: X * b, kmeans, 3).shape (1, 2)
- milia.cluster.ablation.rankshift(table, ascending=False)#
Rank the rows within each column of an
interaction()table.- Parameters:
table (
DataFrame) – Output ofinteraction().ascending (
bool) – Rank ascending (for lower-is-better metrics).
- Return type:
- Returns:
(ranks, changes)– the rank table, and how many rows’ ranks differ across columns. Zero changes means the two steps do not interact.
Examples
>>> import pandas as pd >>> ranks, changes = rankshift(pd.DataFrame({"pca": [0.5, 0.3], "umap": [0.2, 0.4]}, index=["a", "b"])) >>> changes 2
- milia.cluster.ablation.reference(X, estimators, ks=range(2, 11), strategy='rank')#
The recommended k and the resulting cluster sizes for each clusterer on the same input.
Running the same recipe on another dataset with this function is the transfer check: do the algorithms still agree on k?
- Parameters:
X (
ArrayLike) – Clustering input.estimators (
Mapping[str,BaseEstimator]) – Name to clusterer.strategy (
str) – Strategy ofrecommend().
- Return type:
- Returns:
One row per clusterer with
kandsizes(e.g."20/20/20").
Examples
>>> import numpy as np >>> rng = np.random.default_rng(0) >>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0) >>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20) >>> from milia import make >>> kmeans = make("clusterer", "kmeans", n_init=5) >>> reference(X, {"kmeans": kmeans}, ks=range(2, 5)).values.tolist() [['kmeans', 3, '20/20/20']]
- milia.cluster.ablation.consensus(ks)#
Spread and mode of the k chosen by several algorithms.
- Parameters:
ks (
Mapping[str,int] |Sequence[int]) – Algorithm to its chosen k, or the chosen values.- Return type:
- Returns:
low,highandmode.
Examples
>>> consensus({"kmeans": 3, "gmm": 3, "spectral": 5}) {'low': 3, 'high': 5, 'mode': 3}