Key concepts#

From series to features#

Methods in milia work on a matrix with one row per series, or per window of a series. milia.data turns a table of readings into normalised daily profiles and day tensors, and milia.features turns those into named features: the shape of the mean day, calendar and magnitude summaries for clustering, and shape, rolling and spectral features of a window for anomaly detection. Features carry names, so a result can be read back in the units of the problem.

Estimators and the registry#

Every learned component is a scikit-learn estimator with fit and predict, so it clones, pickles, tunes with GridSearchCV and composes in a Pipeline. Detectors and clusterers are registered by name: milia.available("detector") lists them, and milia.make("detector", "lof", n_neighbors=30) builds one from its validated configuration.

Trust before use#

A clustering is real only if it survives resampling. milia scores each candidate number of groups by internal validity and by resampling checks (prediction strength, per-cluster bootstrap and subsample agreement), and it calibrates those checks against structureless reference data, so a split that the data does not support is not chosen. A retrieval carries a trust radius, so a query that is far from every labelled case is reported as such.

Prediction strength and cluster-wise stability.

Two of the resampling checks that a clustering has to pass.#

Conformal thresholds#

Anomaly scores have no natural scale. milia sets the threshold from held-out normal data with a split-conformal quantile, so that, for exchangeable data, the expected share of normal cases that are flagged is bounded by the level alpha that you choose. A threshold can also be set per group, so that a group with a different normal range is held to its own standard.

Reproducible runs#

Randomness is seeded from named conditions, and a benchmark writes a run folder with a provenance record: the settings, the package versions and the input files. Methods are compared with significance tests on per-unit results, not by a single average.