milia.cluster.conformal#

Split-conformal prediction sets for cluster membership (Lei, Rinaldo and Wasserman, 2015).

Instead of forcing each meter into one cluster, it gets the set of every cluster whose centroid lies within a calibrated distance. A meter with a set of two or more sits between archetypes; one with an empty set fits none.

Functions

centroids(model)

A fitted clusterer's centroids.

confusion(predicted)

How often each pair of clusters appears together in a prediction set.

sets(X, estimator[, alpha, test, random_state])

Split-conformal cluster prediction sets (Lei, Rinaldo and Wasserman, 2015).

setsizes(predicted)

Number of rows with each prediction-set size.

Classes

ConformalSets(centroids, tau, alpha)

A calibrated membership rule: centroids and the distance threshold tau.

class milia.cluster.conformal.ConformalSets(centroids, tau, alpha)#

Bases: object

A calibrated membership rule: centroids and the distance threshold tau.

Examples

>>> rule = ConformalSets(centroids=np.array([[0.0], [10.0]]), tau=2.0, alpha=0.1)
>>> [s.tolist() for s in rule.predict([[1.0], [5.0], [9.5]])]
[[0], [], [1]]
distances(X)#

Distance from every row to every centroid.

Parameters:

X (ArrayLike) – Rows.

Return type:

ndarray

Returns:

Array of shape (n, clusters).

predict(X)#

The prediction set of each row: every cluster whose centroid lies within tau.

An empty set means the row fits no archetype; a set of two or more means it sits between archetypes.

Parameters:

X (ArrayLike) – Rows.

Return type:

list[ndarray]

Returns:

One array of cluster indices per row.

milia.cluster.conformal.centroids(model)#

A fitted clusterer’s centroids.

Parameters:

model (BaseEstimator) – A fitted clusterer exposing centroids_ (milia) or cluster_centers_ (scikit-learn).

Return type:

ndarray

Returns:

The centroids, shape (clusters, features).

Raises:

TypeError – If the clusterer has no centroids, as for a density clusterer.

Examples

>>> import numpy as np
>>> from milia import make
>>> centroids(make("clusterer", "kmeans", n_clusters=1).fit(np.ones((3, 2)))).tolist()
[[1.0, 1.0]]
milia.cluster.conformal.sets(X, estimator, alpha=0.1, test=0.3, random_state=0)#

Split-conformal cluster prediction sets (Lei, Rinaldo and Wasserman, 2015).

The clusterer is fitted on a training split; tau is the conformal quantile of the held-out rows’ nearest-centroid distances, so a new exchangeable row’s nearest cluster is in its set with probability at least 1 - alpha.

Parameters:
  • X (ArrayLike) – Data.

  • estimator (BaseEstimator) – A centroid clusterer already set to its k.

  • alpha (float) – Miscoverage level.

  • test (float) – Share of rows held out for calibration.

  • random_state (int) – Split seed.

Return type:

ConformalSets

Returns:

The calibrated ConformalSets.

Examples

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> centres = np.repeat(np.array([[0.0, 0.0], [6.0, 6.0], [0.0, 6.0]]), 20, axis=0)
>>> X, truth = centres + rng.normal(scale=0.5, size=(60, 2)), np.repeat([0, 1, 2], 20)
>>> from milia import make
>>> rule = sets(X, make("clusterer", "kmeans", n_clusters=3), alpha=0.1)
>>> rule.centroids.shape, len(rule.predict(X[:2]))
((3, 2), 2)
milia.cluster.conformal.setsizes(predicted)#

Number of rows with each prediction-set size.

Parameters:

predicted (list[ndarray]) – Output of ConformalSets.predict().

Return type:

Series

Returns:

Set size to count, sorted by size.

Examples

>>> setsizes([np.array([0]), np.array([0, 1]), np.array([1])]).to_dict()
{1: 2, 2: 1}
milia.cluster.conformal.confusion(predicted)#

How often each pair of clusters appears together in a prediction set.

Parameters:

predicted (list[ndarray]) – Output of ConformalSets.predict().

Return type:

DataFrame

Returns:

a, b and count columns, most frequent pair first.

Examples

>>> confusion([np.array([0, 1]), np.array([1, 0]), np.array([2])]).values.tolist()
[[0, 1, 2]]