milia.features.profile#

Named, interpretable features of each meter, for clustering.

Input is a meter pivot (DatetimeIndex by meter columns); output is one row per meter. extract() assembles the groups switched on in a ProfileConfig; ProfileFeatures wraps it as a scikit-learn transformer so it can open a Pipeline.

Functions

autocorr(data[, lags])

Autocorrelation of each meter's readings at several lags.

calendar(data)

Daily peak, peak-to-average, weekday/weekend and morning/evening energy ratios.

distribution(data)

Skewness, kurtosis, quartiles and inter-quartile range of daily totals.

extract(data[, config])

Feature matrix with one row per meter, built from the groups a config enables.

magnitude(daily)

Mean, standard deviation, maximum, minimum and total of daily energy.

profile(data[, hemisphere, period])

Hour-of-day mean profiles per meter: weekly and/or seasonal.

ratio(numerator, denominator)

Elementwise ratio with infinities and NaN replaced by zero.

shape(data)

Peak hour, morning (06-10) to evening (17-21) ratio and mean daily load factor.

shapes(cube)

Describe each meter by the shape of its mean day, divided by its peak, which discards its size.

sizes(cube[, evening, midday])

Describe each meter by the size of its demand: mean demand, mean evening peak and mean midday demand.

trend(daily)

Latest day-on-day changes at lags 1, 2, 3 and 7, and the least-squares slope of daily energy.

volatility(daily)

Coefficient of variation and mean 30-day rolling standard deviation of daily energy.

Classes

ProfileConfig(*[, profile, normalized, ...])

Which feature groups extract() builds.

ProfileFeatures([config])

Scikit-learn transformer from a meter pivot to a meters-by-features matrix.

class milia.features.profile.ProfileConfig(*, profile=True, normalized=False, period='both', hemisphere='south', magnitude=False, volatility=False, trend=False, shape=False, autocorr=False, calendar=False, distribution=False)#

Bases: BaseModel

Which feature groups extract() builds.

Variables:
  • profile (bool) – Hour-of-day profiles (weekly 7x24 and/or seasonal 4x24).

  • normalized (bool) – Divide readings by the meter’s daily maximum before profiling.

  • period (Literal['weekly', 'seasonal', 'both']) – "weekly", "seasonal" or "both" profiles.

  • hemisphere (Literal['north', 'south']) – Season convention for seasonal profiles.

  • magnitude (bool) – Mean, spread and total of daily consumption.

  • volatility (bool) – Coefficient of variation and 30-day rolling spread.

  • trend (bool) – Recent day-on-day changes and the linear slope.

  • shape (bool) – Peak hour, morning/evening ratio and load factor.

  • autocorr (bool) – Autocorrelation at a few lags.

  • calendar (bool) – Weekday/weekend and peak-to-average ratios.

  • distribution (bool) – Skewness, kurtosis and quantiles of daily totals.

Examples

>>> ProfileConfig(profile=False, magnitude=True).magnitude
True
>>> ProfileConfig(profile=False)
Traceback (most recent call last):
...
pydantic_core._pydantic_core.ValidationError: ...
anygroup()#

At least one feature group must be switched on.

Runs automatically after construction.

Return type:

ProfileConfig

Returns:

The validated config.

Raises:

ValueError – If every group is switched off.

milia.features.profile.ratio(numerator, denominator)#

Elementwise ratio with infinities and NaN replaced by zero.

Parameters:
  • numerator (Series) – Numerator series.

  • denominator (Series) – Denominator series, aligned with numerator.

Return type:

Series

Returns:

The ratio, zero where it is undefined.

Examples

>>> import pandas as pd
>>> ratio(pd.Series([1.0, 1.0, 0.0]), pd.Series([2.0, 0.0, 0.0])).tolist()
[0.5, 0.0, 0.0]
milia.features.profile.profile(data, hemisphere='south', period='both')#

Hour-of-day mean profiles per meter: weekly and/or seasonal.

Weekly columns are named w_<day>_<hour> (7 x 24) and seasonal columns s_<season>_<hour> (4 x 24), seasons following the hemisphere.

Parameters:
  • data (DataFrame) – Meter pivot.

  • hemisphere (str) – "north" or "south", for the season of each timestamp.

  • period (Literal['weekly', 'seasonal', 'both']) – "weekly", "seasonal" or "both".

Return type:

DataFrame

Returns:

One row per meter, one column per profile cell.

Raises:

ValueError – If period is not one of the three options.

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> profile(pivot, period="weekly").shape
(4, 168)
milia.features.profile.shape(data)#

Peak hour, morning (06-10) to evening (17-21) ratio and mean daily load factor.

The load factor is daily energy over 24 times the daily peak, averaged over days.

Parameters:

data (DataFrame) – Meter pivot, a DatetimeIndex by meter-column frame.

Return type:

DataFrame

Returns:

One row per meter – peak_hour, morning_evening_ratio, load_factor.

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> shape(pivot).shape
(4, 3)
milia.features.profile.autocorr(data, lags=(1, 2, 3, 6, 12))#

Autocorrelation of each meter’s readings at several lags.

Strong autocorrelation at the daily lag marks a routine household; weak autocorrelation an irregular one.

Parameters:
Return type:

DataFrame

Returns:

One row per meter, one autocorr_lag<k> column per lag (0 when the series is too short).

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> autocorr(pivot, lags=(1, 24)).shape
(4, 2)
milia.features.profile.calendar(data)#

Daily peak, peak-to-average, weekday/weekend and morning/evening energy ratios.

Ratios with an undefined denominator are reported as zero.

Parameters:

data (DataFrame) – Meter pivot, a DatetimeIndex by meter-column frame.

Return type:

DataFrame

Returns:

One row per meter – daily_peak, peak_to_avg_ratio, weekday_weekend_ratio, morning_evening_ratio.

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> calendar(pivot).shape
(4, 4)
milia.features.profile.distribution(data)#

Skewness, kurtosis, quartiles and inter-quartile range of daily totals.

Describes how a meter’s daily energy is spread across days, not within them.

Parameters:

data (DataFrame) – Meter pivot, a DatetimeIndex by meter-column frame.

Return type:

DataFrame

Returns:

One row per meter – skewness, kurtosis, q25, q50, q75, iqr.

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> distribution(pivot).shape
(4, 6)
milia.features.profile.magnitude(daily)#

Mean, standard deviation, maximum, minimum and total of daily energy.

The size features that peak-normalised shape features deliberately leave out.

Parameters:

daily (DataFrame) – Daily totals, days by meters (pivot.resample("D").sum()).

Return type:

DataFrame

Returns:

One row per meter – mean, std, max, min, total.

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> magnitude(pivot.resample("D").sum()).shape
(4, 5)
milia.features.profile.volatility(daily)#

Coefficient of variation and mean 30-day rolling standard deviation of daily energy.

With fewer than 30 days the rolling term falls back to the plain standard deviation.

Parameters:

daily (DataFrame) – Daily totals, days by meters (pivot.resample("D").sum()).

Return type:

DataFrame

Returns:

One row per meter – cv, rolling_std_30d.

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> volatility(pivot.resample("D").sum()).shape
(4, 2)
milia.features.profile.trend(daily)#

Latest day-on-day changes at lags 1, 2, 3 and 7, and the least-squares slope of daily energy.

Changes beyond the available history are reported as zero.

Parameters:

daily (DataFrame) – Daily totals, days by meters (pivot.resample("D").sum()).

Return type:

DataFrame

Returns:

One row per meter – change_lag1, change_lag2, change_lag3, change_lag7, slope.

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> trend(pivot.resample("D").sum()).shape
(4, 5)
milia.features.profile.extract(data, config=None)#

Feature matrix with one row per meter, built from the groups a config enables.

The pivot is cleaned first (milia.data.prep.clean()). With normalized set, readings are divided by each meter’s daily maximum before the profile features, so they describe when energy is used rather than how much; the other groups always use the cleaned readings.

Parameters:
  • data (DataFrame) – Meter pivot.

  • config (ProfileConfig | None) – Feature groups; weekly and seasonal profiles only by default.

Return type:

DataFrame

Returns:

Meters by features.

Examples

>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=4, days=35, steps=24).pivot
>>> extract(pivot).shape  # 7 x 24 weekly cells, 24 for the one season covered
(4, 192)
>>> extract(pivot, ProfileConfig(profile=False, magnitude=True, shape=True)).columns.tolist()[:3]
['mean', 'std', 'max']
class milia.features.profile.ProfileFeatures(config=None)#

Bases: TransformerMixin, BaseEstimator

Scikit-learn transformer from a meter pivot to a meters-by-features matrix.

Stateless: fit only records the feature names. Because it turns time rows into meter rows, it belongs at the start of a pipeline whose later steps work on meters.

Parameters:

config (ProfileConfig | None) – Feature groups to extract.

Examples

>>> from sklearn.pipeline import Pipeline
>>> from milia import make
>>> from milia.data import synthetic
>>> pivot = synthetic.generate(meters=9, days=14, steps=24).pivot
>>> pipe = Pipeline([("features", ProfileFeatures()), ("cluster", make("clusterer", "kmeans", n_clusters=3))])
>>> pipe.fit_predict(pivot).shape
(9,)
fit(X, y=None)#

Record the names of the features the config produces.

Parameters:
  • X (DataFrame) – Meter pivot.

  • y (None) – Ignored; present for API consistency.

Return type:

ProfileFeatures

Returns:

The fitted transformer.

transform(X)#

Extract the features of a pivot (see extract()).

Parameters:

X (DataFrame) – Meter pivot.

Return type:

DataFrame

Returns:

Meters by features.

get_feature_names_out(input_features=None)#

Names of the output features, recorded in fit().

Parameters:

input_features (None) – Ignored; present for API consistency.

Return type:

ndarray

Returns:

Feature names, in column order.

milia.features.profile.sizes(cube, evening=slice(34, 42, None), midday=slice(20, 30, None))#

Describe each meter by the size of its demand: mean demand, mean evening peak and mean midday demand.

These are the size features by which a new connection’s contribution to the evening peak is retrieved from similar customers; the shape of the day is left out on purpose.

Parameters:
  • cube (ndarray) – Power in kW, shape (meters, days, steps).

  • evening (slice) – Steps of the evening peak window (17:00 to 21:00 for half-hourly days).

  • midday (slice) – Steps of the midday window (10:00 to 15:00 for half-hourly days).

Return type:

DataFrame

Returns:

One row per meter – annual, evening and midday, in kW.

Examples

>>> sizes(np.ones((2, 3, 48))).round(1).values.tolist()
[[1.0, 1.0, 1.0], [1.0, 1.0, 1.0]]
milia.features.profile.shapes(cube)#

Describe each meter by the shape of its mean day, divided by its peak, which discards its size.

Under-recording of a constant share of every reading leaves the shape unchanged, so these features are invariant to it, while they lose the size of the demand that the estimate of a peak contribution needs.

Parameters:

cube (ndarray) – Power in kW, shape (meters, days, steps).

Return type:

DataFrame

Returns:

One row per meter and one column per step; a meter that never draws power has a flat zero shape.

Examples

>>> shapes(np.ones((1, 2, 4)) * np.array([1.0, 2.0, 4.0, 2.0])).values.tolist()
[[0.25, 0.5, 1.0, 0.5]]