milia.anomaly#
Anomaly detection: detectors, ensembles, conformal thresholds, injection and group scores.
- class milia.anomaly.BaseDetector#
Bases:
OutlierMixin,BaseEstimator,ABCShared fit, scoring, thresholding and calibration for every detector.
Subclasses implement two hooks and declare their hyperparameters as keyword arguments of
__init__(stored unchanged, scikit-learn style), always includingcontamination:_fit(X)learns from validated training data._anomaly(X)returns one score per row, higher meaning more anomalous.
- Variables:
offset – Subtracted from
score_samplesto givedecision_function.threshold – The same cut-off on the anomaly scale (
-offset_).calibrated – Whether
threshold_came fromcalibrate().n_features_in – Number of features seen during
fit.
Examples
A detector is written by implementing the two hooks:
>>> import numpy as np >>> class Norm(BaseDetector): ... def __init__(self, contamination=0.1): ... self.contamination = contamination ... ... def _fit(self, X): ... self.centre_ = X.mean(axis=0) ... ... def _anomaly(self, X): ... return np.linalg.norm(X - self.centre_, axis=1) >>> Norm().fit(np.random.default_rng(0).normal(size=(100, 2))).predict([[0.0, 0.0], [9.0, 9.0]]).tolist() [1, -1]
- fit(X, y=None)#
Fit the detector on training data assumed to be (mostly) normal.
- Parameters:
- Return type:
- Returns:
The fitted detector.
- calibrate(X, alpha=0.1)#
Replace the threshold with a split-conformal one from held-out normal data.
- Parameters:
- Return type:
- Returns:
The detector, with
threshold_andoffset_updated.
- score_samples(X)#
Normality score of each row: higher means more normal.
This is scikit-learn’s convention; the “higher means more anomalous” score used in the energy literature is
-score_samples(X).- Parameters:
X (
ArrayLike) – Rows to score, with the number of features seen infit.- Return type:
- Returns:
One score per row.
Examples
>>> import numpy as np >>> from milia import make >>> rng = np.random.default_rng(0) >>> train, query = rng.normal(size=(200, 2)), np.array([[0.0, 0.0], [8.0, 8.0]]) >>> scores = make("detector", "ecod").fit(train).score_samples(query) >>> bool(scores[0] > scores[1]) True
- decision_function(X)#
Normality score shifted by the threshold: negative for outliers, positive for inliers.
- predict(X)#
Label each row an inlier (
+1) or an outlier (-1).The threshold is the training
contaminationquantile, or the conformal threshold aftercalibrate().Examples
>>> import numpy as np >>> from milia import make >>> rng = np.random.default_rng(0) >>> train, query = rng.normal(size=(200, 2)), np.array([[0.0, 0.0], [8.0, 8.0]]) >>> make("detector", "iforest").fit(train).predict(query).tolist() [1, -1]
- flag(X)#
Boolean anomaly mask:
Truewhere a row is an outlier.Examples
>>> import numpy as np >>> from milia import make >>> rng = np.random.default_rng(0) >>> train, query = rng.normal(size=(200, 2)), np.array([[0.0, 0.0], [8.0, 8.0]]) >>> make("detector", "knn").fit(train).flag(query).tolist() [False, True]
Modules
Base class for anomaly detectors, following scikit-learn's outlier-detector API. |
|
Change-point detection: when did a meter's behaviour change? |
|
Split-conformal thresholds on anomaly scores, globally or per group. |
|
Detector ensembles: several detectors' scores rescaled to a common range and combined. |
|
SHAP explanations of an Isolation Forest flag. |
|
Anomaly scores for a group of meters rather than a single one. |
|
Synthetic anomalies to inject into otherwise normal profiles. |
|
Registered anomaly detectors, one module per model (see |
|
Alarm triage: rank entities by their worst event. |