milia.data.profiles#
Daily profiles and day tensors built from a meter pivot.
These replace the per-notebook copies of “hour-of-day mean, then peak
normalise” and of season_tensor.
Functions
|
Mean daily profile of each meter. |
|
Peak demand per household of a group of |
|
Mean of the rows of an array in each group, such as the mean daily profile of each cluster. |
|
The row of each group with the smallest total Euclidean distance to the others of its group. |
|
Lay a cube of daily readings out as a meter pivot, the inverse of |
|
The rows of a pivot that fall on the given days. |
|
Normalised mean daily profile of each meter, peak-normalised by default. |
|
Position of each timestamp within its day. |
|
Readings per day implied by the index spacing. |
|
Meter-by-day-by-step tensor, the input of Tucker and CROCS. |
|
Each meter's mean daily profile of every week, normalised by its largest absolute value. |
- milia.data.profiles.steps(index)#
Readings per day implied by the index spacing.
- Parameters:
index (
DatetimeIndex) – A regularDatetimeIndex.- Return type:
- Returns:
Readings per day, e.g. 48 for half-hourly data.
- Raises:
ValueError – If the spacing does not divide a day.
Examples
>>> import pandas as pd >>> steps(pd.date_range("2024-01-01", periods=96, freq="30min")) 48
- milia.data.profiles.select(pivot, dates=None)#
The rows of a pivot that fall on the given days.
- Parameters:
pivot (
DataFrame) – Meter readings.dates (
DatetimeIndex|None) – Days to keep; all rows whenNone.
- Return type:
- Returns:
The selected rows.
Examples
>>> import pandas as pd, numpy as np >>> pivot = pd.DataFrame({"a": np.arange(48.0)}, index=pd.date_range("2024-01-01", periods=48, freq="h")) >>> len(select(pivot, pd.to_datetime(["2024-01-02"]))) 24
- milia.data.profiles.slot(index, count)#
Position of each timestamp within its day.
- Parameters:
index (
DatetimeIndex) – Timestamps.count (
int) – Readings per day.
- Return type:
- Returns:
Integers from
0tocount - 1.
Examples
>>> import pandas as pd >>> slot(pd.to_datetime(["2024-01-01 00:30", "2024-01-01 23:30"]), 48).tolist() [1, 47]
- milia.data.profiles.daily(pivot, dates=None)#
Mean daily profile of each meter.
- Parameters:
pivot (
DataFrame) – Meter readings with a regularDatetimeIndex.dates (
DatetimeIndex|None) – Restrict to these days, e.g. one season; all days when omitted.
- Return type:
- Returns:
One row per meter, one column per step of the day.
Examples
>>> import pandas as pd, numpy as np >>> index = pd.date_range("2024-01-01", periods=48, freq="h") >>> pivot = pd.DataFrame({"a": np.tile(np.arange(24.0), 2)}, index=index) >>> daily(pivot).shape, float(daily(pivot).loc["a", 23]) ((1, 24), 23.0)
- milia.data.profiles.shape(pivot, dates=None, how='peak')#
Normalised mean daily profile of each meter, peak-normalised by default.
The representation most of the clustering chapters use: when in the day a meter consumes, with its size removed.
- Parameters:
pivot (
DataFrame) – Meter readings.dates (
DatetimeIndex|None) – Restrict to these days; all days when omitted.how (
Literal['peak','energy','zscore','minmax']) – Normalisation, seenormalize().
- Return type:
- Returns:
Array of shape
(meters, steps); withhow="peak"each row peaks at 1.
Examples
>>> import pandas as pd, numpy as np >>> index = pd.date_range("2024-01-01", periods=48, freq="h") >>> pivot = pd.DataFrame({"a": np.tile(np.arange(24.0), 2)}, index=index) >>> float(shape(pivot).max()) 1.0
- milia.data.profiles.tensor(pivot, dates=None, count=None)#
Meter-by-day-by-step tensor, the input of Tucker and CROCS.
- Parameters:
pivot (
DataFrame) – Meter readings over whole days.dates (
DatetimeIndex|None) – Restrict to these days; all days when omitted.count (
int|None) – Readings per day; inferred from the index when omitted.
- Return type:
- Returns:
Array of shape
(meters, days, steps), meters in the pivot’s column order.- Raises:
ValueError – If the rows do not form whole days.
Examples
>>> import pandas as pd, numpy as np >>> index = pd.date_range("2024-01-01", periods=48, freq="h") >>> pivot = pd.DataFrame({"a": np.arange(48.0), "b": np.ones(48)}, index=index) >>> tensor(pivot).shape (2, 2, 24)
- milia.data.profiles.weekly(cube, days=7)#
Each meter’s mean daily profile of every week, normalised by its largest absolute value.
The normalisation keeps the sign, so a net load that exports keeps its negative midday.
- Parameters:
- Return type:
- Returns:
Shape
(meters, weeks, steps); a week of zero readings stays zero.
Examples
>>> cube = np.ones((1, 15, 4)) * np.array([1.0, 2.0, -4.0, 0.0]) >>> weekly(cube).shape, weekly(cube)[0, 0].tolist() ((1, 2, 4), [0.25, 0.5, -1.0, 0.0])
- milia.data.profiles.pivot(cube, start='2020-01-01', freq='30min')#
Lay a cube of daily readings out as a meter pivot, the inverse of
tensor().- Parameters:
- Return type:
- Returns:
One row per reading and one column per meter (
0tometers - 1).
Examples
>>> pivot(np.zeros((2, 1, 48))).shape (48, 2)
- milia.data.profiles.groupmean(values, groups)#
Mean of the rows of an array in each group, such as the mean daily profile of each cluster.
- Parameters:
- Return type:
- Returns:
A dictionary from each group, in sorted order, to the mean of its rows, of shape
values.shape[1:].
Examples
>>> means = groupmean(np.array([[1.0, 2.0], [3.0, 4.0], [10.0, 20.0]]), np.array(["a", "a", "b"])) >>> means["a"].tolist(), means["b"].tolist() ([2.0, 3.0], [10.0, 20.0])
- milia.data.profiles.medoids(values, groups)#
The row of each group with the smallest total Euclidean distance to the others of its group.
The medoid summarises a group by one of its own members, so the summary keeps the shape and the peak of a real household that a mean smooths away. The mean minimises the sum of squared distances and the medoid the sum of distances, which makes it less sensitive to outliers.
- Parameters:
- Return type:
- Returns:
A dictionary from each group, in sorted order, to the index of its medoid among all rows.
Examples
>>> rows = np.array([[0.0], [1.0], [1.2], [10.0]]) >>> medoids(rows, np.array([0, 0, 0, 1])) {0: 1, 1: 3}
- milia.data.profiles.diversity(values, sizes, draws=100, random_state=0)#
Peak demand per household of a group of
nhouseholds, as a share of the mean individual peak.The demands of households do not peak at the same time, so the peak of the sum of
nof them is less thanntimes the mean peak of one. The ratio of the first to the second, the coincidence factor, falls from one for a single household towards the ratio of the peak of the mean profile to the mean peak asngrows. It is the quantity that a representative profile built from a mean omits.- Parameters:
- Return type:
- Returns:
A frame with one row per size and the columns
size,mean(coincidence factor averaged over the draws),lowandhigh(5th and 95th percentiles).
Examples
>>> flat = np.array([[1.0, 0.0], [0.0, 1.0]]) >>> diversity(flat, [1, 2], draws=5)["mean"].round(2).tolist() [1.0, 0.5]