milia.recommend.retrieval#
Content-based retrieval: predict an entity’s outcome from its nearest labelled neighbours.
Two split-conformal calibrations on held-out labelled rows say how far to believe a retrieval.
The trust radius bounds novelty. A query’s score is the distance to its nearest labelled neighbour, and the radius \(\hat q\) is the conformal quantile of the held-out scores; a query farther than \(\hat q\) has no trustworthy precedent, e.g. a customer from another population or one changed by new DER.
The error interval bounds accuracy. With the held-out outcomes, the residual \(|y - \hat y|\) is calibrated into an interval \(\hat y \pm \varepsilon\); in its locally adaptive form the residual is divided by the spread of the neighbours’ outcomes \(\hat\sigma_k\), giving \(\hat y \pm \varepsilon\, \hat\sigma_k\), wider where the neighbours disagree [Romano et al., 2019].
Either calibration can be made per group, such as a network or a regime (Mondrian conformal prediction, Vovk et al., 2005), so that its guarantee holds within each group rather than only on average over them.
Functions
|
Share of a target population trusted under a radius calibrated on the source population. |
Classes
|
Mean outcome of the k nearest labelled neighbours, with a conformal trust radius. |
- class milia.recommend.retrieval.KNNRetriever(n_neighbors=5, metric='minkowski')#
Bases:
RegressorMixin,BaseEstimatorMean outcome of the k nearest labelled neighbours, with a conformal trust radius.
- Parameters:
- Variables:
tau – Trust radius set by
calibrate()(infuntil then).scores – Held-out distances to the nearest labelled neighbour, kept by
calibrate()forpvalue().epsilon – Error quantile set by
calibrate()with outcomes (nanuntil then).group_tau – Trust radius of each group, when calibrated with
groups.group_epsilon – Error quantile of each group, when calibrated with outcomes and
groups.
Examples
>>> rng = np.random.default_rng(0) >>> X = rng.normal(size=(100, 2)) >>> model = KNNRetriever(n_neighbors=5).fit(X[:70], X[:70, 0]).calibrate(X[70:], alpha=0.1) >>> model.trusted([[0.0, 0.0], [20.0, 20.0]]).tolist() [True, False]
- fit(X, y)#
Index the labelled rows.
- Parameters:
- Return type:
- Returns:
The fitted retriever, with an infinite
tau_untilcalibrate().
- predict(X)#
Mean outcome of each row’s nearest labelled neighbours.
Examples
>>> import numpy as np >>> rng = np.random.default_rng(0) >>> X = rng.normal(size=(80, 2)) >>> y = 2 * X[:, 0] >>> model = KNNRetriever(n_neighbors=3).fit(X[:60], y[:60]) >>> float(np.corrcoef(model.predict(X[60:]), y[60:])[0, 1]) > 0.9 True
- distance(X)#
Distance from each row to its nearest labelled neighbour.
- spread(X)#
Standard deviation of the outcomes of each row’s nearest labelled neighbours.
Floored at a millionth of the standard deviation of all labelled outcomes, so that a query whose neighbours agree exactly still has a positive scale.
Examples
>>> model = KNNRetriever(n_neighbors=2).fit([[0.0], [1.0], [10.0], [11.0]], [0.0, 2.0, 5.0, 5.0]) >>> model.spread([[0.5], [10.5]]).round(3).tolist() [1.0, 0.0]
- calibrate(X, alpha=0.1, y=None, groups=None, adaptive=False)#
Calibrate the trust radius and, given outcomes, the error interval on held-out labelled rows.
The trust radius
tau_is the split-conformal quantile of the held-out rows’ distances to their nearest labelled neighbour, so a new exchangeable row lies within it with probability at least1 - alpha. Withy, the error quantileepsilon_is the same quantile of the residuals|y - predict(X)|, divided byspread()whenadaptive, andinterval()then covers a new exchangeable outcome with probability at least1 - alpha. Withgroups, both are also calibrated within each group.- Parameters:
X (
ArrayLike) – Held-out labelled features, not used infit.alpha (
float) – Miscoverage level.y (
ArrayLike|None) – Held-out outcomes; the error interval is calibrated only when given.groups (
ArrayLike|None) – Group of each held-out row, such as its network; calibrates per group when given.adaptive (
bool) – Scale the residuals by the neighbours’ spread.
- Return type:
- Returns:
The retriever, calibrated.
Examples
>>> import numpy as np >>> rng = np.random.default_rng(0) >>> X = rng.normal(size=(80, 2)) >>> y = 2 * X[:, 0] >>> model = KNNRetriever().fit(X[:60], y[:60]).calibrate(X[60:]) >>> bool(np.isfinite(model.tau_)), model.trusted([[50.0, 50.0]]).tolist() (True, [False])
- trusted(X, groups=None)#
Whether each row has a labelled neighbour within the trust radius.
- Parameters:
- Return type:
- Returns:
Boolean mask;
Falsemarks a query with no trustworthy precedent.
Examples
>>> model = KNNRetriever(n_neighbors=1).fit([[0.0], [10.0]], [0.0, 0.0]) >>> model = model.calibrate([[0.5], [1.0], [10.2], [10.4]], alpha=0.4, groups=["a", "a", "b", "b"]) >>> model.trusted([[0.8], [10.8], [0.8]], groups=["a", "b", "c"]).tolist() [True, False, False]
- pvalue(X)#
Conformal p-value of each row’s distance to its nearest labelled neighbour.
A row is trusted at level
alphawhen its p-value exceedsalpha; a small p-value marks a query that lies farther from every labelled row than all but a few held-out rows did.- Parameters:
X (
ArrayLike) – Query features.- Return type:
- Returns:
One p-value per row, in
(0, 1].- Raises:
ValueError – If the retriever was not calibrated.
Examples
>>> import numpy as np >>> rng = np.random.default_rng(0) >>> X = rng.normal(size=(100, 2)) >>> model = KNNRetriever().fit(X[:60], X[:60, 0]).calibrate(X[60:]) >>> model.pvalue([[0.0, 0.0], [20.0, 20.0]]).round(3).tolist() [0.78, 0.024]
- interval(X, groups=None)#
Conformal interval for each row’s outcome, from
calibrate()with outcomes.- Parameters:
- Return type:
- Returns:
Lower and upper bounds.
- Raises:
ValueError – If the error interval was not calibrated, or a group was never calibrated.
Examples
>>> import numpy as np >>> rng = np.random.default_rng(1) >>> X = rng.uniform(size=(400, 1)) >>> y = X[:, 0] + rng.normal(scale=0.1, size=400) >>> model = KNNRetriever(n_neighbors=10).fit(X[:200], y[:200]).calibrate(X[200:300], y=y[200:300]) >>> lower, upper = model.interval(X[300:]) >>> 0.8 <= float(np.mean((y[300:] >= lower) & (y[300:] <= upper))) <= 1.0 True
- set_score_request(*, sample_weight='$UNCHANGED$')#
Configure whether metadata should be requested to be passed to the
scoremethod.Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with
enable_metadata_routing=True(seesklearn.set_config()). Please check the User Guide on how the routing mechanism works.The options for each parameter are:
True: metadata is requested, and passed toscoreif provided. The request is ignored if metadata is not provided.False: metadata is not requested and the meta-estimator will not pass it toscore.None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.str: metadata should be passed to the meta-estimator with this given alias instead of the original name.
The default (
sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.Added in version 1.3.
Parameters#
- sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED
Metadata routing for
sample_weightparameter inscore.
Returns#
- selfobject
The updated object.
- Return type:
- milia.recommend.retrieval.coverage(train, cal, target, alpha=0.1)#
Share of a target population trusted under a radius calibrated on the source population.
About
1 - alphain distribution; well below it under domain shift, e.g. another utility’s customers or customers changed by new DER.- Parameters:
- Return type:
- Returns:
The trusted share of
target.
Examples
>>> import numpy as np >>> rng = np.random.default_rng(0) >>> X = rng.normal(size=(80, 2)) >>> y = 2 * X[:, 0] >>> coverage(X[:40], X[40:60], X[60:] + 10.0) 0.0