Retrieval and Ranking on Real Networks
Chapter 1 stated retrieval and ranking for the Low Voltage (LV) network and tested them on a synthetic case base whose outcomes followed a voltage model of four features. On that case base the nearest cases estimated a new transformer’s hours outside the band with a mean absolute error of 1.17 hours, and a conformal trust radius refused most of the queries drawn from networks with longer feeders. Both results rested on the model that generated the cases, in which the features that drive the outcome are known and measured. On a real network neither holds by construction. The features available to a Distribution System Operator (DSO) are those of its smart meters, its asset register and its network model, the outcome follows from a power flow, and the population of a new request may differ from the one the case base was built on in ways the features do not record.
This chapter applies the two steps to the AusNet networks of Part 5, in the form of two decisions a DSO takes routinely. The first is whether to approve a request to connect rooftop Photovoltaic (PV) without a network study, which retrieval answers from simulated customers that resemble the request. The second is the order in which transformers enter a reinforcement programme, which ranking answers from what is observable today. The chapter asks which features make a retrieved estimate accurate enough to approve or refuse a connection, when the trust radius detects that an estimate should not be used, how a decision can be given a guaranteed rate of error, and whether what can be observed today, including the violation events of Part 5, orders the transformers of later years on this network and on a second one.
Retrieval and Ranking in Distribution Planning
Estimating the effect of a new connection without a full study is the purpose of the hosting capacity methods of distribution planning. Stochastic methods simulate many placements of PV on a feeder and report the share of placements that violate a limit [1], and surveys of the field distinguish these from deterministic, optimisation-based and streamlined methods by their cost and their reliance on a network model [2]. Model-free and clustering-based estimates have been proposed for networks whose models are incomplete [3], [4]. These methods estimate the capacity of a feeder or a transformer, whereas a request concerns one connection point, and the voltage at a point depends on the impedance between it and the transformer. The voltage rise caused by a connection that injects active power \(P_q\) and reactive power \(Q_q\) through a Thevenin impedance \(R_q + jX_q\) at nominal voltage \(V\) is approximately
\[ \Delta V_q \approx \frac{R_q P_q + X_q Q_q}{V}. \tag{7.1}\]
The approximation makes the position of a connection a candidate feature for retrieval, alongside the shape and size of demand that Part 5 used to group customers. Figure 7.1 contrasts the two. Customers of similar shape may stand anywhere on a feeder, whereas the customers nearest in position share the impedance of Equation 7.1.
The evaluation of a retrieval on real data raises two further questions, the design of the split and the shift of the population. Customers of a network are not independent cases. Several customers are assigned the measured load profile of the same household, and all customers of a transformer share its rating and its upstream impedance. A random split of customers therefore places near-duplicates of a test case in the case base, a form of leakage that inflates the measured accuracy [5]. Roberts and colleagues showed that data with hierarchical or spatial structure require blocked splits that hold out whole groups [6], and the groups here are the transformer and the household. A model may also be applied to queries whose distribution differs from that of its training data, either in the features or in the relation of the outcome to the features [7]. Conformal guarantees assume exchangeability and fail under such a shift, unless the calibration is reweighted for a known change in the features [8] or made separately within each group, as in Equation 6.10. Whether the trust radius of Equation 6.5 detects a given shift is therefore an empirical question.
The ranking of transformers for reinforcement is usually made from the asset register and from measured loading, and it is judged by the outcomes of later years, when adoption has grown. The Normalised Discounted Cumulative Gain (NDCG) of Equation 6.14 judges the top of such a list, and Wang and colleagues showed that it separates good from poor rankings consistently as the list grows [9]. A list of a few dozen transformers is well below the size at which that result applies, and Kendall’s coefficient of Equation 6.15, computed over the whole list, complements it. The next section states the quantities that both steps estimate.
Clusters in the Decision Chain
Part 5 grouped the households of AusNet by the shape and by the level of their demand, and the violations of HPK11 by their duration, depth and timing. A natural expectation is that these groups predict the outcomes this part estimates and orders, and the evidence of this chapter is that they do not (Table 7.1). The voltage of a connection is set by its position on the network: the transformer and the impedance at the connection point explained 90.8% of the variance of every customer’s highest voltage, and the midday demand of its household, with a median of 0.42 kW against a 5 kWp array, added nothing. Customers are moreover assigned the profiles of households independently of their position, so that the archetype explained 0.03% of the variance of the impedance. On HPK11 the history of over-voltage is no better a predictor: among the 35 transformers that exceeded 1.10 pu at 20% PV, their hours at 20% and at 80% agreed with a Kendall’s \(\tau_b\) of 0.010.
Clusters nonetheless serve the decisions, as structure rather than as predictors, which is also how recommender systems use them: to scale to large catalogues, to serve new users and to treat groups of users equally [10]. This chapter uses them in three roles (Figure 7.2). As strata, the demand clusters define the groups within which a guarantee of the error of a decision is calibrated, so that it holds for every segment of customers rather than on average, as Equation 6.10 requires [11]. As representatives, clusters of transformers identify the few transformers whose study best covers the others, the competence that case-based reasoning seeks in a case base [12]. As groups of a programme, the constraints that the regimes of violation separate, thermal and voltage, define separate rankings, because their mitigation measures differ, as asset management ranks by failure mode [13]. Chapter 3 adds two further roles, a mitigation measure recommended for each regime of violation and the equity of curtailment across the demand clusters.
Labelled Risk Sets
A labelled risk set assigns to each customer of a network its outcomes under a stated adoption, and the features that are known before any study. Four sets were simulated with popo on the AusNet network models of Part 5 (Table 7.2). Each customer was given a 5 kWp array with inverters set to the AS/NZS 4777.2 Australia A volt-VAr and volt-watt characteristics. With \(v_q(t)\) the voltage at its connection point in half-hour \(t\) of a clear summer day, its voltage outcome is the highest voltage of the day,
\[ y_q^{\mathrm{v}} = \max_t v_q(t). \tag{7.2}\]
Each customer was separately given a 7.36 kW charger in use from 18:00 to 21:00. With \(p_q(t)\) its demand, including the charger, on the day of highest evening demand, the network peaks in the half-hour
\[ t^\star = \arg\max_t \sum_j p_j(t), \tag{7.3}\]
and the thermal outcome of the customer is its contribution to that peak,
\[ y_q^{\mathrm{t}} = p_q(t^\star). \tag{7.4}\]
The voltage outcome decides a connection directly, by comparison with the upper limit \(\overline{v} = 1.10\) pu of AS 61000.3.100. The thermal outcome is a contribution, which a DSO compares with the remaining headroom of the transformer.
The first set covers HPK11, each of whose 5,274 residential customers is assigned the measured load profile of one of the 342 AusNet households, and the second CRE21, the second urban network of the same models, with 3,374 customers on 70 transformers. The third and fourth keep HPK11 and its assignment of customers to transformers but replace the AusNet households with households of London and of the Basque Country (GoiEner), each on a summer day of its own record and excluding households whose demand exceeds 20 kW. The four sets thus differ in two controlled ways. CRE21 changes the network and keeps the households, and the London and GoiEner sets change the households and keep the network. On HPK11 34.9% of customers exceeded \(\overline{v}\), against 20.2% on CRE21, although the median impedance at the connection point was almost equal, 0.076 and 0.077 \(\Omega\).
The features of a customer form four groups,
\[ \mathbf{x}_q = \bigl(\mathbf{s}_q, \mathbf{m}_q, \mathbf{a}_q, R_q, X_q\bigr). \tag{7.5}\]
The shape \(\mathbf{s}_q\) holds the first three principal components of its household’s mean day, each day normalised by its own peak, fitted on the AusNet households. The size \(\mathbf{m}_q\) holds the mean demand, the mean daily evening peak between 17:00 and 21:00 and the mean demand between 10:00 and 15:00. The asset features \(\mathbf{a}_q\) are the rating of its transformer and the number of single-phase customers it supplies. The position is the resistance \(R_q\) and reactance \(X_q\) of the Thevenin impedance at the connection point, obtained from a fault study of the network model. The shape and the size are known from the smart meter and the asset features from the register. The position requires a model of the LV network, which not every DSO holds, but it requires no power flow under the new connection.
Retrieval of the Risk of a New Connection
A request to connect is a query from a customer whose household may not have been simulated, on a transformer that may not have been studied, and the evaluation reproduces this position (Figure 7.3). Each split held out 20% of the transformers and, independently, 20% of the households, and tested only customers on a held-out transformer whose load profile comes from a held-out household. A second 20% of each calibrated the trust radius and the intervals, and customers on the remaining transformers with the profiles of the remaining households formed the case base \(\mathcal{D}_1\). The split was repeated 200 times.
The estimate is the mean outcome of the 10 nearest cases of Equation 6.3, on features standardised over the case base. Four sets of features were compared, adding the groups of Equation 7.5 in turn, against four baselines. The population mean estimates every query by the mean of the case base. The archetype, demand-cluster and demand-tercile means estimate it by the mean of the customers of the same group of Part 5, the segmentations a utility would apply to a new customer. A linear regression on all features shows what a global model achieves with the same information, and a blend of the two, \(\beta \hat y_q^{\mathrm{kNN}} + (1 - \beta) \hat y_q^{\mathrm{lin}}\) with \(\beta\) chosen on the calibration customers, interpolates a retrieval with a parametric model as nearest-neighbour language models do [14]. Describing a new customer by its features rather than by its history is the remedy that recommender systems apply to the cold start, where factorisation machines add such side information to the identities of users and items [15]. Each method was scored by its mean absolute error over the \(n_{\mathrm{test}}\) test customers \(\mathcal{T}\) of a split,
\[ \mathrm{MAE} = \frac{1}{n_{\mathrm{test}}} \sum_{q \in \mathcal{T}} \lvert y_q - \hat y_q \rvert. \tag{7.6}\]
For the voltage, the error matters through the decision it implies. A connection is approved when its estimated highest voltage is within the limit, \(\hat y_q^{\mathrm{v}} \le \overline{v}\), and the costly error is the approval of a connection that exceeds it. Its rate among the over-limit customers \(\mathcal{T}^{+} = \{q \in \mathcal{T} : y_q^{\mathrm{v}} > \overline{v}\}\) is
\[ \mathrm{MA} = \frac{1}{\lvert \mathcal{T}^{+} \rvert} \sum_{q \in \mathcal{T}^{+}} \mathbb{1}\bigl\{\hat y_q^{\mathrm{v}} \le \overline{v}\bigr\}, \tag{7.7}\]
and the rate of refusals among the customers within the limit is defined in the same way.
The segmentations of Part 5 did not reduce the voltage error at all (Figure 7.4). The population mean erred by 7.88 mpu, or 1.8 V at 230 V, and the archetype, demand-cluster and tercile means by 7.89 mpu each. Nearest cases on the shape of demand were worse than the population mean, at 8.21 mpu, and adding the size of demand and the asset features brought them back only to 7.93 mpu. Adding the position halved the error, to 4.22 mpu or about 1.0 V, and the linear regression on the same features reduced it further, to 3.94 mpu, and the blend of the two to 3.86 mpu. For the thermal outcome the order was different. The demand-cluster mean, at 1.38 kW, improved on the population mean of 1.46 kW, nearest cases on shape alone were the worst estimate, at 1.74 kW, and each group of features reduced the error, to 1.26 kW with all four, against 1.22 kW for the linear regression and 1.20 kW for the blend.
The decisions followed the errors. Because the mean highest voltage, 1.095 pu, lies just within the limit, the population mean and every segmentation mean approved all of the over-limit connections, and nearest cases on shape and size approved 96.6% of them. With the position, the nearest cases approved 28.7% of the over-limit connections and refused 11.0% of those within the limit, and the linear regression 31.0% and 8.5%. An estimate from the position therefore removes most of the approvals that would breach the limit, at the cost of refusing about one connection in ten that would not, and a DSO could send the refused requests to a study rather than decline them.
These differences held split by split (Table 7.3). Adding the position reduced the voltage error by 3.72 mpu in all 200 splits, whereas adding the size of demand to its shape changed it by 0.005 mpu and reduced it in only 49.0% of the splits, with a Wilcoxon p-value of 0.978 [16]. For the thermal outcome the size of demand reduced the error by 0.158 kW in 86.5% of the splits. The linear regression improved on the nearest cases by 0.277 mpu and 0.034 kW, in 73.0% and 75.0% of the splits. The blend improved on the linear regression by 0.084 mpu in 65.5% of the splits, and by 0.021 kW on average although in only 46.0% of them, a gain concentrated in the splits in which the regression erred most. The voltage of a connection is therefore retrieved by where it stands rather than by how its household uses energy, as the variance decomposition of Part 5 anticipated, and its demand contributes to the thermal outcome through its size more than through its shape. That a linear regression improved on the nearest cases, and their blend on both, also indicates that the outcome varies smoothly with the position, as Equation 7.1 suggests, while the nearest cases correct it where it does not.
Choosing the Transformers to Study
The case base of a utility is built from the studies it can afford, and with thousands of transformers the choice of which to study determines how well the others are retrieved. The clustering of Part 5 offers one choice. The transformers are described by their rating, customers, impedance and demand, standardised, and clustered by k-means into as many clusters \(\mathcal{C}_1, \dots, \mathcal{C}_m\) as there are studies, with centres \(\boldsymbol{\mu}_k\); the transformer studied in each cluster is its medoid,
\[ j_k = \arg\min_{j \in \mathcal{C}_k} \lVert \mathbf{a}_j - \boldsymbol{\mu}_k \rVert, \tag{7.8}\]
where \(\mathbf{a}_j\) are the features of transformer \(j\). This is the competence that case-based reasoning seeks in a case base, a few cases that cover the others [12]. The medoids were compared with transformers drawn at random, for budgets from 4 to 20 of the 43 transformers, by the error of retrieval for the customers of the transformers not studied, over 40 repeats in which the households were also split in half, so that no household was both studied and retrieved.
The medoids improved on a random choice only modestly, and not at every budget (Figure 7.5). With 12 to 20 transformers studied they reduced the voltage error by 1.2% to 3.6%, to 4.26 mpu against 4.37 mpu at 20, and the thermal error by up to 2.8%. With 8 they were worse, at 5.70 mpu against 4.99 mpu. A medoid is a typical transformer, whereas retrieval for the others also needs the unusual ones, at the edges of the feature space, and with few studies the clusters cannot hold both. A selection that covers the extremes, by the distance of every transformer to the nearest one studied, is the alternative that the notion of competence suggests.
These errors were measured on customers of the network the case base was built on. A request may come from a household unlike any simulated, or from another network, and the next section examines whether the trust radius recognises such requests.
Refusal Under Shift
The trust radius of Equation 6.5 refuses the queries that do not resemble the case base. Its calibration at \(\alpha = 0.1\) guarantees that at most 10% of exchangeable queries are refused, and the adaptive interval of Equation 6.9 that at least 90% of the trusted outcomes are covered. Both were calibrated on HPK11 with all features of Equation 7.5 and applied to the held-out HPK11 customers, to every customer of CRE21, and to the London and GoiEner customers on the held-out HPK11 transformers. A shift the radius detects lowers the share of queries trusted, and a shift it misses lowers the coverage of those it trusts. Figure 7.6 illustrates the two cases.
On the held-out HPK11 customers the radius trusted 87.7% of queries and the voltage interval covered 89.0% of the trusted outcomes (Figure 7.7). The trusted share fell slightly below the nominal 90%, as expected of queries that are new in two respects at once, and the coverage held. The households of other populations were refused more often, 20.8% of the London and 29.0% of the GoiEner queries, and the coverage of those trusted held at 88.3% and 87.7%. The radius therefore detected the change of households, conservatively, since the refused queries were estimated with errors only about 12% larger than the trusted ones. Single splits varied widely around these means, by more than ten points of coverage, which is why a conformal guarantee is read over repeated splits, as the remark of Chapter 1 states.
The change of network was not detected. The radius trusted 85.9% of the CRE21 customers, almost as many as of HPK11, but the voltage interval covered only 77.7% of their outcomes, and the error of the trusted queries rose from 4.25 to 5.48 mpu. The customers of CRE21 resemble those of HPK11 in every feature, including the impedance at the connection point, but the network responds differently, as the lower share of customers above \(\overline{v}\) shows. The relation between features and outcome had changed while the features had not, and a radius on the features cannot detect such a shift. The thermal outcome, which depends on the household more than on the network, was covered at 90.2% on CRE21. A model applied to a network on which it was not calibrated therefore requires the guarantee of Equation 6.10, with calibration cases from that network, and a few studies on each new network provide them. On the network of the case base, however, the radius guarantees only how often a resembling request is refused, not how often a trusted estimate leads to a wrong decision, which the next section addresses.
Decisions with a Guaranteed Error
The decision that matters to a DSO is whether to approve a connection, and with the position the nearest cases still approved 28.7% of the connections that exceeded the limit. This section calibrates the decision itself, so that its rate of error is bounded, and sends the requests on which the estimate is too uncertain to a network study.
The calibration follows conformal risk control [17], which extends the guarantee of split conformal prediction (Equation 6.6) from coverage to the expected value of any loss that does not increase as a decision becomes more cautious. Language models use the same construction to withhold generated claims until the rate of false ones falls below a target [18]. The uncertainty of an estimate is measured by the spread of the highest voltages of the \(k\) nearest cases of request \(q\),
\[ \hat\sigma_q = \Bigl( \frac{1}{k} \sum_{j \in \mathcal{N}_k(q)} \bigl( y_j - \hat y_q \bigr)^2 \Bigr)^{1/2}, \tag{7.9}\]
and a margin \(\lambda \ge 0\) is counted in units of this spread. A request is approved when its estimate, raised by the margin, is within the limit,
\[ \hat y_q + \lambda \hat\sigma_q \le \overline{v}, \tag{7.10}\]
and is otherwise sent to a study. The loss of a calibration customer \(i\) that exceeds the limit is whether it would have been approved,
\[ L_i(\lambda) = \mathbb{1}\bigl\{ \hat y_i + \lambda \hat\sigma_i \le \overline{v} \bigr\}, \tag{7.11}\]
which does not increase with \(\lambda\). Over the \(n^{+}\) calibration customers above the limit, its mean is
\[ \hat R(\lambda) = \frac{1}{n^{+}} \sum_{i=1}^{n^{+}} L_i(\lambda), \tag{7.12}\]
and the margin is the smallest whose corrected mean is at most the target \(\alpha\),
\[ \hat\lambda = \inf \Bigl\{ \lambda : \frac{n^{+}}{n^{+} + 1} \hat R(\lambda) + \frac{1}{n^{+} + 1} \le \alpha \Bigr\}. \tag{7.13}\]
If a new over-limit customer is exchangeable with those of the calibration set, the probability that it is approved is bounded by the target,
\[ \mathbb{E}\bigl[ L_{n^{+} + 1}(\hat\lambda) \bigr] \le \alpha. \tag{7.14}\]
A second rule also refuses. A request is decided automatically when the band of \(\lambda\) spreads around its estimate lies wholly on one side of the limit,
\[ \bigl| \hat y_q - \overline{v} \bigr| > \lambda \hat\sigma_q, \tag{7.15}\]
approved below it and refused above it, and is sent to a study otherwise. Its loss is a wrong automatic decision,
\[ L_i(\lambda) = \mathbb{1}\bigl\{ | \hat y_i - \overline{v} | > \lambda \hat\sigma_i \bigr\}\, \mathbb{1}\bigl\{ (\hat y_i > \overline{v}) \ne (y_i > \overline{v}) \bigr\}, \tag{7.16}\]
which also does not increase with \(\lambda\), and it is calibrated by Equation 7.13 over every calibration customer. Figure 7.8 illustrates both steps. This arrangement, in which an inexpensive estimate decides where it is certain and an exact computation is reserved for the remaining cases, corresponds to speculative decoding in language models, in which a small model proposes and a large one verifies [19].
Both rules were calibrated on the calibration customers of each of the 200 splits, at targets from 1% to 20%, and applied to the test customers (Figure 7.9). The achieved error stayed at its target within sampling error. At \(\alpha = 0.10\) the first rule approved 9.8% of the over-limit connections and the second decided 9.4% of the requests wrongly, and at \(\alpha = 0.05\) the errors were 5.2% and 4.8%, with standard errors of the mean between 0.3 and 0.6 points. The guarantee is obtained at the cost of the requests sent to a study. To approve at most 5% of the over-limit connections, the first rule studied 57.1% of the requests, and at 10% it studied 47.8%; because it never refuses, every likely refusal is studied as well. The second rule studied 37.0% of the requests at an error of 5% and 19.0% at 10%. Since every request would otherwise be studied, a calibrated retrieval removed between 63% and 81% of the studies at a stated rate of wrong decisions. The guarantee rests on exchangeability, which Figure 7.7 showed to fail on another network, where the calibration must use customers of that network.
The guarantee of Equation 7.14 holds on average over the over-limit customers, and a segment of customers may bear more than its share of the error. With the demand clusters of Part 5 as strata, the margin can be calibrated within each cluster \(z\) on its own \(n_z^{+}\) over-limit customers,
\[ \hat\lambda_z = \inf \Bigl\{ \lambda : \frac{n_z^{+}}{n_z^{+} + 1} \hat R_z(\lambda) + \frac{1}{n_z^{+} + 1} \le \alpha \Bigr\}, \tag{7.17}\]
where \(\hat R_z\) is the mean loss of Equation 7.12 over the cluster, so that the guarantee holds within each cluster, as in Equation 6.10. At a target of 5%, the pooled margin approved 5.8% of the over-limit connections of the cluster of lowest demand and 5.6% of the cluster of highest demand, above the target, and 4.2% of the remaining cluster (Figure 7.10). Calibrated within each cluster, the margins approved 4.2%, 3.2% and 4.4%, within the target for every cluster, at the cost of studying between 68.0% and 73.9% of the requests rather than between 56.3% and 57.9%. A utility that must treat its customers equally can therefore buy a guarantee for each segment with roughly a quarter more studies.
Retrieval and its decisions serve a request for one connection. A reinforcement programme instead concerns every transformer of a network at once, which is a question of ranking.
Ranking Transformers Now for the Network Later
A reinforcement programme examines transformers in order, and the order is set before the adoption it guards against has arrived. The evaluation therefore ranks the 43 residential transformers of HPK11 by what is observable at 20% penetration in DSS-AU, and judges the order by the violation hours each transformer incurred at 80%. For PV, the relevance \(\mathrm{rel}_i\) of a transformer is its hours above \(\overline{v}\) at 80% PV and 20% Electric Vehicle (EV) penetration. For EV, it is its hours below 0.94 pu or above its rating at 80% EV and 20% PV penetration. Figure 7.11 illustrates why the two need not behave alike.
The record of DSS-AU shows the difference before any score is fitted (Figure 7.12). Under EV charging the transformers that were most loaded at 20% were largely those with the most violation hours at 80%. Under PV, 35 of the 44 transformers already exceeded \(\overline{v}\) at 20%, for a median of 113 hours, but the transformers with the most hours at 20% were not those with the most at 80%, because the hours grew at rates that differed between transformers.
Five scores were compared against a random order. Customers per kVA and today’s violation hours are the two heuristics a utility holds without a model, the latter counting hours above \(\overline{v}\) for PV and hours above 90% of the rating for EV. Three pointwise models, the five nearest transformers, a linear regression and gradient-boosted trees, were fitted to the later hours from all of today’s features, including the impedance of the connection points and the composition of each transformer’s customers, and scored each transformer when left out of the fit. A listwise ranker of the kind Chapter 1 described learns from many lists, and one network supplies a single list of 43 transformers, so the pointwise models are the learned scores of this chapter. The NDCG at 5 used five grades, the quintiles of the later hours, and Kendall’s \(\tau_b\) was computed on the hours themselves. Intervals were obtained from 1,000 bootstrap resamples of the transformers.
For EV the heuristics sufficed (Figure 7.13). Customers per kVA placed the five worst transformers first, with an NDCG at 5 of 1.000 and a \(\tau_b\) of 0.604, and today’s hours reached 0.963 and 0.562. The gradient-boosted trees reached the highest \(\tau_b\), 0.637, but its interval from 0.548 to 0.714 overlapped those of both heuristics. Charging adds load in proportion to the number of customers, so the transformer that is most heavily loaded per kVA today remains so at 80%.
For PV the heuristics failed. Today’s hours above \(\overline{v}\) reached a \(\tau_b\) of only 0.196, and customers per kVA 0.053, no better than the random order at 0.048. The learned scores reached 0.504 to 0.573, with intervals that excluded both heuristics, and so recovered much of the later order that today’s hours missed. The NDCG at 5 was less informative. A random order reached 0.742, with an interval from 0.468 to 0.927 that contained every learned score, because, with 43 transformers in five grades, a random choice of five places often attains a high gain. The top of a single short list cannot separate the scores, and the order of the whole list is the more reliable criterion for a programme of this size.
Ranking by Today’s Events
Today’s hours count how long a transformer violates its limits, but not how, in one long afternoon or in many brief excursions, slightly or far beyond the limit. Part 5 described each violation event by its duration, its depth and its timing (Equation 4.2, Figure 4.9) and grouped the events of HPK11 into eight regimes. The same description characterises a transformer by its events today. For transformer \(j\) and kind \(\kappa\) of violation, with \(\mathcal{E}_{j\kappa}\) its events at 20% penetration, the severity is the summed excess integral,
\[ S_{j\kappa} = \sum_{e \in \mathcal{E}_{j\kappa}} I_e, \tag{7.18}\]
which weighs depth and duration together. Each transformer is further described, for each kind, by the number of its events, the mean of the features of Equation 4.2 over them, its longest and deepest event, and the number of days with an event and with one sustained for at least one and for at least two hours. It is also described by the share of its event hours in each regime \(r\) of Part 5,
The regimes were refitted with the description, sample, seed and number of regimes of Part 5, and reproduced its eight regimes exactly. Because the hours ignore depth, the transformers were ranked both against their hours at 80% and against their severity at 80%, Equation 7.18 applied to the later events.
For EV charging, the severity of today’s events was the best order (Figure 7.14). It reached a \(\tau_b\) of 0.712 against the hours at 80%, with an interval from 0.627 to 0.796 that excluded today’s hours at 0.562, and 0.772 against the later severity. No learned score improved on it, and gradient-boosted trees on the events and assets together reached 0.662. The severity weighs the depth of today’s evening excursions, which the hours ignore, and the transformers whose excursions were deepest at 20% were those that violated most at 80%. For PV, today’s events contained little information on the later order. The severity reached 0.195, no more than the hours, the regime shares 0.172 and trees on the event features 0.367, while trees on the events with the asset features reached 0.586, no more than trees on hours and assets at 0.575. The timing and depth of today’s over-voltage thus add nothing to the position of a transformer in the network, which accounts for its later order.
Onset and Severity of PV Violations
The failure of today’s history to order the transformers for PV has a cause in the data. At 20% penetration, one customer in five has an array, and the violations of a transformer reflect where those first adopters happen to connect more than how much it can host; at 80% almost every customer has one, and the structure of the network decides. Two quantities separate the questions a DSO asks. The hosting capacity of transformer \(j\) is the penetration at which its over-voltage first breaches the criterion,
\[ \mathrm{HC}_j = \min \bigl\{ p : D_j(p) \ge 87.6 \bigr\}, \tag{7.20}\]
interpolated linearly between the simulated levels, and states when a transformer will need action; the hours and the margin at 80% state how severe its violations will then be. Two scores that do not depend on the first adopters were added. The voltage sensitivity measured from today’s record is the slope \(\beta_j\) of the regression of the highest customer voltage on the power of the transformer over the midday half-hours,
\[ v_j^{\max}(t) = a_j + \beta_j P_j(t) + \epsilon_j(t), \tag{7.21}\]
expressed per customer as \(-\beta_j n_j\), since a transformer of many customers changes its total power more for the same export of each. The screening study is the mean highest voltage of the transformer’s customers with every customer adopting, from the labelled risk set.
Today’s history ordered the onset and not the severity (Figure 7.15). Today’s hours ordered the hosting capacity with a \(\tau_b\) of 0.919, and the hours at 80% with 0.196. The severity was ordered by the structure of the network: the number of customers supplied, a quantity of the asset register, reached 0.560 for the hours and 0.598 for the margin at 80%, as high as trees on all of today’s features at 0.575 and 0.555, followed by the screening study at 0.504 and the measured sensitivity at 0.415. The sensitivity also ordered the hosting capacity, at 0.542, and adding it to the trees raised their order of the hosting capacity from 0.770 to 0.798. A transformer of many customers carries more export at high adoption, and the measured sensitivity adds how strongly its voltage responds. For a programme that asks when each transformer will need action, today’s history suffices; for one that asks how severe the violations will become, the register and a measurement of sensitivity serve better than the history.
A Second Network
A ranking learned on one network serves a utility with many only if it transfers. DSS-US records 30 pole transformers of a SMART-DS feeder in Austin, Texas, under the same 25 scenarios, judged against ANSI C84.1 Range A, 0.95 to 1.05 pu. Few of its transformers violate a limit even at 80%, three above 1.05 pu under PV and six overloaded under EV charging, so that hours would give almost every transformer the same relevance. The US transformers were therefore ranked by their margin to the limit. With \(x_j(t)\) the highest customer voltage under PV, or the loading under EV charging, of transformer \(j\) at half-hour \(t\) of the year at 80%, the relevance is its 99th percentile,
\[ \mathrm{rel}_j = Q_{0.99}\bigl( \{ x_j(t) \}_t \bigr). \tag{7.22}\]
The scores used what both records hold, today’s hours of each violation, the 99th percentiles of the loading and of the highest voltage, and the rating, standardised within each network, so that a model fitted on HPK11 ranks the US transformers by their position relative to their own network.
The transfer held for EV charging and not for PV (Figure 7.16). Today’s peak loading ordered the later loading with a \(\tau_b\) of 0.591, and gradient-boosted trees fitted on HPK11 alone reached 0.652, with an interval from 0.526 to 0.769, against 0.416 for the same model fitted within DSS-US. For PV, today’s highest voltage reached 0.425, the best of any score, while the learned scores reached 0.214 to 0.320 within DSS-US and 0.260 from HPK11. On the US feeder the voltage margin today ordered the margin later. The hours of HPK11 could not show such an order, because 35 of its 44 transformers were already beyond the limit at 20%.
Learning to Rank Across Networks
The scores so far were pointwise, each fitted to the outcome of a transformer and ranking only as a consequence. Chapter 1 described rankers that learn the order itself, pairwise and listwise, and a single list cannot test them. Liu frames learning to rank at the level of queries [20]. A ranker \(f\) is judged by its mean loss over the \(n\) queries \(q\) of a training set,
\[ \hat R(f) = \frac{1}{n} \sum_{q=1}^{n} L(f; q), \tag{7.23}\]
and Lan and colleagues showed that this risk approaches its expectation at a rate of \(1/\sqrt{n}\), so that a listwise ranker generalises with the number of lists rather than of items [21]. A pointwise score is not thereby inferior. Cossock and Zhang bounded the ranking error of a list of \(m\) items by the regression error of the score that orders it,
\[ 1 - \mathrm{NDCG} \le \frac{1}{\mathrm{IDCG}} \Bigl( \sum_{i=1}^{m} w_i^2 \Bigr)^{1/2} \Bigl( \sum_{i=1}^{m} \bigl( f_i - \mathrm{rel}_i \bigr)^2 \Bigr)^{1/2}, \tag{7.24}\]
with the weights \(w_i\) of Equation 6.13, so that a score that estimates the relevance accurately also orders it well [22]. The bound is sufficient but not necessary, since a perfect order can come from a score with a large regression error.
Each network supplies several lists when each pair of a penetration today and a higher one later is taken as a query: today 20% or 40% of the technology, later 60% or 80%, four queries per technology on each network. The relevance is the margin of Equation 7.22, graded into quintiles within each list for the listwise ranker. LambdaMART [23], [24] and pointwise gradient-boosted trees learned from the four lists of one network and ranked those of the other, with the features of both records standardised within each list. Their orders were also combined with the margin today by reciprocal rank fusion, which scores item \(i\) by its ranks \(r_s(i)\) under the scores \(s\) being combined,
\[ \mathrm{RRF}(i) = \sum_{s} \frac{1}{60 + r_s(i)}, \tag{7.25}\]
and requires no training [25]. Hybrid search engines that retrieve context for language models merge their keyword and vector rankings in the same way.
No learned ranker improved on the margin today when it learned from the other network (Figure 7.17). For EV charging on HPK11, the margin today ordered the later loading with a mean \(\tau_b\) of 0.919 over the four queries, against 0.628 for the pointwise trees and 0.588 for LambdaMART learned on DSS-US; on DSS-US the four scores lay between 0.621 and 0.651. For PV, the margin today reached 0.322 on HPK11 and 0.538 on DSS-US, the learned rankers 0.182 to 0.321. LambdaMART fell below the pointwise trees in two of the four cases and exceeded them by at most 0.032 in the other two, as Equation 7.24 allows and as four training lists from one network provide few terms to the average of Equation 7.23. The fusion was never the worst score and stayed within 0.14 of the better of its two components, at 0.785 and 0.629 for EV and 0.337 and 0.400 for PV, a property of value to a utility that cannot determine in advance which score will transfer.
Ranking by Risk Under Uncertain Adoption
The rankings so far ordered the transformers by their outcome in one future. A DSO that plans maintenance or replacement for thousands of transformers ranks them instead by risk, the likelihood of a failure to meet a limit combined with its consequence [13], and the future in which the likelihood is assessed is itself uncertain. The scenarios of DSS-AU provide that uncertainty. With adoption of each technology at the planning horizon distributed over the levels \(x \in \{0, 0.2, \dots, 0.8\}\) with probabilities \(\pi(x)\), here proportional to a normal density centred at 0.4 with a standard deviation of 0.2, a scenario \(s\) of EV penetration \(e_s\) and PV penetration \(p_s\) has the probability
\[ w_s = \pi(e_s)\, \pi(p_s). \tag{7.26}\]
Thermal and voltage violations are ranked separately, since their mitigation measures differ, transformer replacement for the one and voltage management for the other, and since the regimes of Part 5 separate them. For a constraint \(c\), with \(D_{jc}(s)\) the hours of violation of transformer \(j\) in scenario \(s\), its likelihood is the probability of breaching the hosting criterion of Equation 4.1,
\[ P_{jc} = \sum_{s} w_s\, \mathbb{1}\bigl\{ D_{jc}(s) > 87.6 \bigr\}, \tag{7.27}\]
its expected hours of violation are
\[ H_{jc} = \sum_{s} w_s\, D_{jc}(s), \tag{7.28}\]
and its risk weighs them by the \(n_j\) customers it supplies, in customer-hours outside the limits,
\[ R_{jc} = n_j\, H_{jc}. \tag{7.29}\]
Figure 7.18 illustrates the likelihood as the probability of the futures that breach and the ranking by likelihood and consequence. A utility cannot simulate every transformer, so the likelihood was also learned from what is observable today. A classifier estimates the probability \(\hat p(\mathbf{x}_j, e, p)\) that a transformer with today’s observables \(\mathbf{x}_j\) breaches the criterion at penetrations \(e\) and \(p\), fitted on the scenarios beyond today of the other transformers, and the learned likelihood weighs it by Equation 7.26,
\[ \hat P_{jc} = \frac{\sum_{s \in \mathcal{S}} w_s\, \hat p(\mathbf{x}_j, e_s, p_s)}{\sum_{s \in \mathcal{S}} w_s}, \tag{7.30}\]
where \(\mathcal{S}\) is the set of scenarios beyond today.
Under this distribution of adoption, the mean likelihood of breaching the criterion was 0.156 for overload and 0.797 for voltage on HPK11: most transformers will breach the voltage criterion, and the risk of each is therefore told apart by its expected hours and its customers rather than by its likelihood (Figure 7.19). The learned likelihood ordered the risk with a \(\tau_b\) of 0.726 for the thermal and 0.796 for the voltage constraint, with Brier scores of 0.078 and 0.065. For a programme that can inspect one transformer in five, the order of the learned likelihood captured 62.3% of the thermal risk and 41.0% of the voltage risk, against ceilings of 66.4% and 41.8% for the order of the risk itself (Figure 7.20). Customers per kVA captured 64.4% of the thermal risk but only 29.7% of the voltage risk, and today’s violation hours 60.7% and 24.6%. The learned likelihood thus served both programmes, whereas each heuristic served only the thermal one. Its probabilities did not transfer to DSS-US, whose transformers rarely breach the criterion: learned on HPK11, the classifier reached Brier scores of 0.336 and 0.889 there, so that a utility must calibrate the probabilities on its own network before it ranks by them.
Conclusion
This chapter applied retrieval and ranking to labelled risk sets of the AusNet networks, in which every customer was assigned its simulated voltage and thermal outcome together with the features known before a study. Retrieval was evaluated on transformers and households held out together, over 200 splits. The voltage of a new connection was retrieved by its position on the network. Nearest cases with the Thevenin impedance of the connection point erred by 4.22 mpu against 7.88 mpu for the population mean, and approved 28.7% of the connections that exceeded the limit, whereas the segmentations of Part 5 approved all of them. The thermal contribution was retrieved by the size of demand, with an error of 1.26 kW against 1.46 kW.
The trust radius detected a change of households and missed a change of network. It refused 20.8% and 29.0% of the queries from two other populations, but trusted 85.9% of the customers of CRE21, whose voltage intervals then covered 77.7% rather than 90% of the outcomes. A model transferred to a new network therefore requires calibration on that network. On the network of the case base, conformal risk control bounded the rate of wrong decisions: at a target of 5%, a rule that approves, refuses or studies made 4.8% wrong decisions while studying 37.0% of the requests. Ranking transformers for the network of later years depended on the technology. For EV charging, customers per kVA today ordered the transformers as well as any learned score, whereas for PV today’s violation hours contained little information on the later order, and pointwise models fitted to the features raised Kendall’s \(\tau_b\) from 0.196 to between 0.504 and 0.573. The severity of today’s events ordered the transformers for EV charging better than any other score, at 0.712, while for PV the events added nothing to the features of the network. On a second network, a model fitted on HPK11 ordered the EV loading margins of 30 US transformers at 0.652. Learned across networks, neither pointwise trees nor LambdaMART improved on the margin today, and a reciprocal rank fusion of the two was the most robust order.
The clusters of Part 5 entered the chain as structure rather than as predictors. Calibrated within each demand cluster, the decision rule kept the approval error of every cluster within a target of 5%, where the pooled rule exceeded it in two clusters, at the cost of about a quarter more studies. Studying the medoids of clusters of transformers reduced the error of retrieval for the others by at most 3.6% against a random choice, and not with few studies. Ranked by risk under uncertain adoption, separately for the thermal and the voltage constraint, a learned likelihood captured 62.3% of the thermal and 41.0% of the voltage risk when one transformer in five was inspected. For PV, today’s history ordered the onset of violations, with a \(\tau_b\) of 0.919 against the hosting capacity, but not their severity at high adoption, which the number of customers supplied ordered at 0.560.
The limitations follow from the simulations. Customers are assigned the profiles of households independently of their position, so that the analysis cannot represent a network in which types of household cluster geographically, and the adopters of each scenario are drawn at random, so that the first adopters of a real network, who may not be random, could order the onset differently. The risk sets fix one array size, one charger and one day for each outcome, and the replacement populations are placed on the AusNet network and irradiance, so the shift they represent is one of households rather than of climate. The position features require a model of the LV network, and where a DSO has none, a proxy such as the distance along the feeder would have to replace them. The ranking uses two networks, 43 transformers on HPK11 and 30 on DSS-US, and four lists on each, too few for a listwise ranker to show the advantage that many queries give it. Chapter 3 turns from estimating and ordering outcomes to choosing the mitigation measure that changes them.
Exercises
- Remove the resistance and reactance from the features of the
retrievalcell and add the distance of each customer from its transformer along the feeder instead. How much of the reduction of the voltage error of Figure 7.4, and of the over-limit approvals, does the distance recover? - Calibrate the trust radius and the interval of the
shiftcell on CRE21 customers alone, held out by transformer and household as for HPK11, and apply them to the CRE21 test customers. Does the coverage of the voltage interval return to 90%? - Replace the five grades of the
rankingcell with three and with ten. How do the NDCG at 5 of the random order and its interval in Figure 7.13 change, and which number of grades gives the narrowest intervals? - Rank the transformers for PV by the mean resistance of their connection points alone. How close does this single feature come to the learned scores of Figure 7.13?
- Rank the transformers of HPK11 by their margin, Equation 7.22, rather than by their hours. Does today’s highest voltage then order the later margins as it did on DSS-US?
- Calibrate the second rule of the
decisionscell on CRE21 customers held out by transformer and household, and apply it to the remaining CRE21 customers. How many requests must be studied for an error of 5%?