milia.features.profile#
Named, interpretable features of each meter, for clustering.
Input is a meter pivot (DatetimeIndex by meter columns); output is one row
per meter. extract() assembles the groups switched on in a
ProfileConfig; ProfileFeatures wraps it as a scikit-learn
transformer so it can open a Pipeline.
Functions
|
Autocorrelation of each meter's readings at several lags. |
|
Daily peak, peak-to-average, weekday/weekend and morning/evening energy ratios. |
|
Skewness, kurtosis, quartiles and inter-quartile range of daily totals. |
|
Feature matrix with one row per meter, built from the groups a config enables. |
|
Mean, standard deviation, maximum, minimum and total of daily energy. |
|
Hour-of-day mean profiles per meter: weekly and/or seasonal. |
|
Elementwise ratio with infinities and NaN replaced by zero. |
|
Peak hour, morning (06-10) to evening (17-21) ratio and mean daily load factor. |
|
Describe each meter by the shape of its mean day, divided by its peak, which discards its size. |
|
Describe each meter by the size of its demand: mean demand, mean evening peak and mean midday demand. |
|
Latest day-on-day changes at lags 1, 2, 3 and 7, and the least-squares slope of daily energy. |
|
Coefficient of variation and mean 30-day rolling standard deviation of daily energy. |
Classes
|
Which feature groups |
|
Scikit-learn transformer from a meter pivot to a meters-by-features matrix. |
- class milia.features.profile.ProfileConfig(*, profile=True, normalized=False, period='both', hemisphere='south', magnitude=False, volatility=False, trend=False, shape=False, autocorr=False, calendar=False, distribution=False)#
Bases:
BaseModelWhich feature groups
extract()builds.- Variables:
profile (
bool) – Hour-of-day profiles (weekly 7x24 and/or seasonal 4x24).normalized (
bool) – Divide readings by the meter’s daily maximum before profiling.period (
Literal['weekly','seasonal','both']) –"weekly","seasonal"or"both"profiles.hemisphere (
Literal['north','south']) – Season convention for seasonal profiles.magnitude (
bool) – Mean, spread and total of daily consumption.volatility (
bool) – Coefficient of variation and 30-day rolling spread.trend (
bool) – Recent day-on-day changes and the linear slope.shape (
bool) – Peak hour, morning/evening ratio and load factor.autocorr (
bool) – Autocorrelation at a few lags.calendar (
bool) – Weekday/weekend and peak-to-average ratios.distribution (
bool) – Skewness, kurtosis and quantiles of daily totals.
Examples
>>> ProfileConfig(profile=False, magnitude=True).magnitude True >>> ProfileConfig(profile=False) Traceback (most recent call last): ... pydantic_core._pydantic_core.ValidationError: ...
- anygroup()#
At least one feature group must be switched on.
Runs automatically after construction.
- Return type:
- Returns:
The validated config.
- Raises:
ValueError – If every group is switched off.
- milia.features.profile.ratio(numerator, denominator)#
Elementwise ratio with infinities and NaN replaced by zero.
- Parameters:
- Return type:
- Returns:
The ratio, zero where it is undefined.
Examples
>>> import pandas as pd >>> ratio(pd.Series([1.0, 1.0, 0.0]), pd.Series([2.0, 0.0, 0.0])).tolist() [0.5, 0.0, 0.0]
- milia.features.profile.profile(data, hemisphere='south', period='both')#
Hour-of-day mean profiles per meter: weekly and/or seasonal.
Weekly columns are named
w_<day>_<hour>(7 x 24) and seasonal columnss_<season>_<hour>(4 x 24), seasons following the hemisphere.- Parameters:
- Return type:
- Returns:
One row per meter, one column per profile cell.
- Raises:
ValueError – If
periodis not one of the three options.
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> profile(pivot, period="weekly").shape (4, 168)
- milia.features.profile.shape(data)#
Peak hour, morning (06-10) to evening (17-21) ratio and mean daily load factor.
The load factor is daily energy over 24 times the daily peak, averaged over days.
- Parameters:
data (
DataFrame) – Meter pivot, aDatetimeIndexby meter-column frame.- Return type:
- Returns:
One row per meter –
peak_hour,morning_evening_ratio,load_factor.
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> shape(pivot).shape (4, 3)
- milia.features.profile.autocorr(data, lags=(1, 2, 3, 6, 12))#
Autocorrelation of each meter’s readings at several lags.
Strong autocorrelation at the daily lag marks a routine household; weak autocorrelation an irregular one.
- Parameters:
- Return type:
- Returns:
One row per meter, one
autocorr_lag<k>column per lag (0 when the series is too short).
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> autocorr(pivot, lags=(1, 24)).shape (4, 2)
- milia.features.profile.calendar(data)#
Daily peak, peak-to-average, weekday/weekend and morning/evening energy ratios.
Ratios with an undefined denominator are reported as zero.
- Parameters:
data (
DataFrame) – Meter pivot, aDatetimeIndexby meter-column frame.- Return type:
- Returns:
One row per meter –
daily_peak,peak_to_avg_ratio,weekday_weekend_ratio,morning_evening_ratio.
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> calendar(pivot).shape (4, 4)
- milia.features.profile.distribution(data)#
Skewness, kurtosis, quartiles and inter-quartile range of daily totals.
Describes how a meter’s daily energy is spread across days, not within them.
- Parameters:
data (
DataFrame) – Meter pivot, aDatetimeIndexby meter-column frame.- Return type:
- Returns:
One row per meter –
skewness,kurtosis,q25,q50,q75,iqr.
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> distribution(pivot).shape (4, 6)
- milia.features.profile.magnitude(daily)#
Mean, standard deviation, maximum, minimum and total of daily energy.
The size features that peak-normalised shape features deliberately leave out.
- Parameters:
daily (
DataFrame) – Daily totals, days by meters (pivot.resample("D").sum()).- Return type:
- Returns:
One row per meter –
mean,std,max,min,total.
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> magnitude(pivot.resample("D").sum()).shape (4, 5)
- milia.features.profile.volatility(daily)#
Coefficient of variation and mean 30-day rolling standard deviation of daily energy.
With fewer than 30 days the rolling term falls back to the plain standard deviation.
- Parameters:
daily (
DataFrame) – Daily totals, days by meters (pivot.resample("D").sum()).- Return type:
- Returns:
One row per meter –
cv,rolling_std_30d.
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> volatility(pivot.resample("D").sum()).shape (4, 2)
- milia.features.profile.trend(daily)#
Latest day-on-day changes at lags 1, 2, 3 and 7, and the least-squares slope of daily energy.
Changes beyond the available history are reported as zero.
- Parameters:
daily (
DataFrame) – Daily totals, days by meters (pivot.resample("D").sum()).- Return type:
- Returns:
One row per meter –
change_lag1,change_lag2,change_lag3,change_lag7,slope.
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> trend(pivot.resample("D").sum()).shape (4, 5)
- milia.features.profile.extract(data, config=None)#
Feature matrix with one row per meter, built from the groups a config enables.
The pivot is cleaned first (
milia.data.prep.clean()). Withnormalizedset, readings are divided by each meter’s daily maximum before the profile features, so they describe when energy is used rather than how much; the other groups always use the cleaned readings.- Parameters:
data (
DataFrame) – Meter pivot.config (
ProfileConfig|None) – Feature groups; weekly and seasonal profiles only by default.
- Return type:
- Returns:
Meters by features.
Examples
>>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot >>> extract(pivot).shape # 7 x 24 weekly cells, 24 for the one season covered (4, 192) >>> extract(pivot, ProfileConfig(profile=False, magnitude=True, shape=True)).columns.tolist()[:3] ['mean', 'std', 'max']
- class milia.features.profile.ProfileFeatures(config=None)#
Bases:
TransformerMixin,BaseEstimatorScikit-learn transformer from a meter pivot to a meters-by-features matrix.
Stateless:
fitonly records the feature names. Because it turns time rows into meter rows, it belongs at the start of a pipeline whose later steps work on meters.- Parameters:
config (
ProfileConfig|None) – Feature groups to extract.
Examples
>>> from sklearn.pipeline import Pipeline >>> from milia import make >>> from milia.data import synthetic >>> pivot = synthetic.generate(meters=9, days=14, steps=24).pivot >>> pipe = Pipeline([("features", ProfileFeatures()), ("cluster", make("clusterer", "kmeans", n_clusters=3))]) >>> pipe.fit_predict(pivot).shape (9,)
- fit(X, y=None)#
Record the names of the features the config produces.
- Parameters:
- Return type:
- Returns:
The fitted transformer.
- milia.features.profile.sizes(cube, evening=slice(34, 42, None), midday=slice(20, 30, None))#
Describe each meter by the size of its demand: mean demand, mean evening peak and mean midday demand.
These are the size features by which a new connection’s contribution to the evening peak is retrieved from similar customers; the shape of the day is left out on purpose.
- Parameters:
- Return type:
- Returns:
One row per meter –
annual,eveningandmidday, in kW.
Examples
>>> sizes(np.ones((2, 3, 48))).round(1).values.tolist() [[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]]
- milia.features.profile.shapes(cube)#
Describe each meter by the shape of its mean day, divided by its peak, which discards its size.
Under-recording of a constant share of every reading leaves the shape unchanged, so these features are invariant to it, while they lose the size of the demand that the estimate of a peak contribution needs.
- Parameters:
cube (
ndarray) – Power in kW, shape(meters, days, steps).- Return type:
- Returns:
One row per meter and one column per step; a meter that never draws power has a flat zero shape.
Examples
>>> shapes(np.ones((1, 2, 4)) * np.array([1.0, 2.0, 4.0, 2.0])).values.tolist() [[0.25, 0.5, 1.0, 0.5]]