milia.core.conformal#
The split-conformal quantile, shared by every conformal method in milia.
Anomaly thresholds, cluster prediction sets and retrieval trust radii all reduce
to the same step: take the finite-sample-corrected 1 - alpha quantile of a
set of calibration nonconformity scores. That step lives here once.
Functions
|
The corrected quantile level |
|
Conformal p-value of each value against calibration scores: |
|
Split-conformal threshold from held-out calibration scores. |
- milia.core.conformal.level(n, alpha)#
The corrected quantile level
ceil((n + 1)(1 - alpha)) / n, capped at 1.The plain
1 - alphaquantile ofncalibration scores under-covers by about1 / n; the(n + 1)correction restores the finite-sample guarantee of split-conformal prediction.- Parameters:
- Return type:
- Returns:
The quantile level to take of the calibration scores.
- Raises:
ValueError – If
n < 1oralphais outside(0, 1).
Examples
>>> level(100, 0.1) 0.91 >>> level(10, 0.1) 1.0
- milia.core.conformal.quantile(scores, alpha)#
Split-conformal threshold from held-out calibration scores.
The threshold is the
ceil((n + 1)(1 - alpha))-th smallest of thenscores, an exact order statistic as in twiga’scalculate_conformal_value; interpolating between order statistics would give a threshold slightly wider than the guarantee needs. A new exchangeable score exceeds it with probability at mostalpha. Whennis too small for that rank to exist, the largest score is returned and the guarantee holds only approximately.- Parameters:
- Return type:
- Returns:
The calibrated threshold.
Examples
>>> scores = np.array([0.1, 0.2, 0.15, 0.9, 0.3, 0.25, 0.18, 0.22, 0.12, 0.4]) >>> round(quantile(scores, alpha=0.1), 2) 0.9 >>> float(quantile(np.arange(1.0, 101.0), alpha=0.1)) # the 91st smallest of 100 91.0
- milia.core.conformal.pvalue(scores, values)#
Conformal p-value of each value against calibration scores:
(1 + #{s_i >= v}) / (n + 1).A value lies within the threshold of
quantile()at levelalphawhen its p-value exceedsalpha(Vovk et al., 2005), so a small p-value marks a query that conforms less than all but a few calibration cases. Ported from twiga’sMultivariateConformal.p_value.- Parameters:
- Return type:
- Returns:
P-values in
(0, 1], with the shape ofvalues.
Examples
>>> pvalue(np.arange(1.0, 10.0), [0.5, 5.0, 20.0]).round(2).tolist() [1.0, 0.6, 0.1]