milia.features.events#

Occurrence, severity and timing of events in a series.

A voltage excursion, a thermal overload, a reverse-flow episode and a PV export window are the same question asked of different signals: how often does the event happen, how bad is it, how long does it last and when. events() answers it for any boolean event mask and severity series; band() and exports() build the mask and severity for the two common cases. eventmap() describes each event of a set, for example every limit violation on a network, by its duration, depth and the time of day and year at which it happens, so that events can be clustered.

Module Attributes

NAMES

Column names of the events() vector.

CIRCULAR_NAMES

the hour of day as the mean of its angle on the 24-hour clock.

MAP_NAMES

Column names of the eventmap() matrix.

REPRESENTATIONS

its magnitude, or its events with clock or circular time.

DAYREPRESENTATIONS

the readings, their shape, summary statistics, events, and events with statistics.

Functions

band(values, low, high[, step, circular])

events() for excursions outside a band, severity being the distance outside.

daysummary(days[, steps])

Level, spread and extremes of each day, with the hours of its largest and smallest reading on the circle.

describe(days, kind, representation)

Describe each day of a behaviour by its magnitude or by its events.

eventmap(start, duration, depth[, floor, ...])

Describe each event by its duration, depth and the time of day and of year at which it happens.

events(mask, severity[, step, circular])

Seven event features: step count, episode count, mean/max/total severity, and the timing of the event.

eventset(days, high[, export, step])

Events of export, high demand and flat readings, seven events() features each.

exports(net[, step, circular])

events() for export (negative net load), severity being the export depth.

exposure(events, units[, group, hours, ...])

Hours of event in each group, and the number of events, of every unit.

represent(days, name[, high])

Represent each day by its readings, its shape, summary statistics, events, or events with statistics.

runlength(mask)

Longest run of consecutive True values along the last axis.

milia.features.events.NAMES = ('steps', 'episodes', 'meanseverity', 'maxseverity', 'totalseverity', 'meanhour', 'hourspread')#

Column names of the events() vector.

milia.features.events.CIRCULAR_NAMES = ('steps', 'episodes', 'meanseverity', 'maxseverity', 'totalseverity', 'hoursin', 'hourcos')#

the hour of day as the mean of its angle on the 24-hour clock.

Type:

Column names with circular=True

milia.features.events.MAP_NAMES = ('duration', 'depth', 'hoursin', 'hourcos', 'daysin', 'daycos')#

Column names of the eventmap() matrix.

milia.features.events.events(mask, severity, step=0.5, circular=False)#

Seven event features: step count, episode count, mean/max/total severity, and the timing of the event.

The timing is the mean hour and its spread by default. A linear mean misplaces an event that spans midnight: hours 20 to 2 average to 14.75. With circular=True the timing is instead the mean sine and cosine of the event’s hours on the 24-hour clock, which places such an event at night and stays continuous across midnight.

Every computation is vectorised over the leading axes, so a whole set of days, shape (days, steps), is described in one call.

Parameters:
  • mask (ArrayLike) – Boolean event indicator, steps along the last axis.

  • severity (ArrayLike) – Non-negative severity, the shape of mask; only entries inside an event count.

  • step (float) – Hours per step, to convert a step index to an hour of day.

  • circular (bool) – Encode the timing on the circle (see CIRCULAR_NAMES).

Return type:

ndarray

Returns:

The features along a new last axis (see NAMES, or CIRCULAR_NAMES when circular) – shape (7,) for one series, (days, 7) for a set of days; all zero where no event occurs.

Examples

>>> mask = np.array([False, True, True, False, True])
>>> severity = np.array([0.0, 0.02, 0.05, 0.0, 0.01])
>>> events(mask, severity, step=1.0).round(3)
array([3.   , 2.   , 0.027, 0.05 , 0.08 , 2.333, 1.247])
>>> events(np.stack([mask, ~mask]), np.stack([severity, severity]), step=1.0).shape
(2, 7)
milia.features.events.band(values, low, high, step=0.5, circular=False)#

events() for excursions outside a band, severity being the distance outside.

For example a voltage trace in per unit against the 0.94-1.10 band.

Parameters:
  • values (ArrayLike) – The series.

  • low (float) – Lower bound of the band.

  • high (float) – Upper bound of the band.

  • step (float) – Hours per step.

  • circular (bool) – Encode the timing on the circle, see events().

Return type:

ndarray

Returns:

The seven events() features.

Examples

>>> band([1.0, 1.12, 1.15, 1.0], 0.94, 1.10, step=1.0)[:2]
array([2., 1.])
milia.features.events.exports(net, step=0.5, circular=False)#

events() for export (negative net load), severity being the export depth.

Parameters:
  • net (ArrayLike) – Net load per step, negative while exporting.

  • step (float) – Hours per step.

  • circular (bool) – Encode the timing on the circle, see events().

Return type:

ndarray

Returns:

The seven events() features.

Examples

>>> exports([1.0, -0.5, -1.0, 0.2], step=0.5)[[0, 3]]
array([2., 1.])
milia.features.events.runlength(mask)#

Longest run of consecutive True values along the last axis.

The run is accumulated over the last axis for every leading index at once, so a (meters, days, steps) mask is scored with one loop over the steps.

Parameters:

mask (ArrayLike) – Boolean series, or an array of them with the series on the last axis.

Return type:

int | ndarray

Returns:

The length of the longest run in steps – an int for a 1-D mask, otherwise an array of shape mask.shape[:-1].

Examples

>>> runlength([True, True, False, True])
2
>>> runlength(np.array([[1, 0, 0, 0, 1], [0, 0, 1, 1, 1]]) == 0).tolist()
[3, 2]
milia.features.events.REPRESENTATIONS: tuple[str, ...] = ('magnitude', 'events', 'eventscircular')#

its magnitude, or its events with clock or circular time.

Type:

Representations of a day compared for behaviour faults

milia.features.events.describe(days, kind, representation)#

Describe each day of a behaviour by its magnitude or by its events.

The magnitude is the day’s total, the energy of the behaviour (export or charging), the extremes and the spread. The events are the seven features of exports() (export) or events() (charging), with the hour on the clock or on the circle.

Parameters:
  • days (ArrayLike) – Readings, shape (days, steps): net load for "export" (negative while exporting), charging power for "charging".

  • kind (str) – "export" or "charging".

  • representation (str) – One of REPRESENTATIONS.

Return type:

ndarray

Returns:

One feature row per day.

Raises:

ValueError – If kind or representation is unknown.

Examples

>>> days = np.ones((2, 48))
>>> days[:, 20:28] = -2.0
>>> describe(days, "export", "eventscircular").shape, describe(days, "export", "magnitude").shape
((2, 7), (2, 5))
milia.features.events.daysummary(days, steps=None)#

Level, spread and extremes of each day, with the hours of its largest and smallest reading on the circle.

Parameters:
  • days (ArrayLike) – Readings, shape (days, steps).

  • steps (int | None) – Steps per day, for the hour of a step; the width of days by default.

Return type:

ndarray

Returns:

Shape (days, 8) – mean, standard deviation, maximum, minimum, and the sine and cosine of the hour of the maximum and of the minimum.

Examples

>>> daysummary(np.array([[0.0, 2.0, 1.0, 1.0]])).round(2).tolist()
[[1.0, 0.71, 2.0, 0.0, 1.0, 0.0, 0.0, 1.0]]
milia.features.events.eventset(days, high, export=-0.2, step=0.5)#

Events of export, high demand and flat readings, seven events() features each.

Parameters:
  • days (ArrayLike) – Net load, shape (days, steps), negative while exporting.

  • high (float) – Demand above which a reading is high.

  • export (float) – Net load below which a reading is export.

  • step (float) – Hours per step.

Return type:

ndarray

Returns:

Shape (days, 21) – the export, high-demand and flat-line features.

Examples

>>> eventset(np.array([[1.0, -1.0, -1.0, 3.0]]), high=2.0).shape
(1, 21)
milia.features.events.DAYREPRESENTATIONS: tuple[str, ...] = ('raw', 'shape', 'summary', 'events', 'eventsummary')#

the readings, their shape, summary statistics, events, and events with statistics.

Type:

Representations of a day compared when detecting faults of any kind

milia.features.events.represent(days, name, high=None)#

Represent each day by its readings, its shape, summary statistics, events, or events with statistics.

  • "raw": the readings.

  • "shape": the readings divided by the day’s largest absolute value, which discards the level.

  • "summary": daysummary().

  • "events": eventset(), which is invariant to the timing of normal variation.

  • "eventsummary": both, so that the level the events discard is restored.

Parameters:
  • days (ArrayLike) – Readings, shape (days, steps).

  • name (str) – One of DAYREPRESENTATIONS.

  • high (float | None) – Demand above which a reading is high, for the events; required for the two event representations.

Return type:

ndarray

Returns:

One feature row per day.

Raises:

ValueError – If name is unknown, or an event representation lacks high.

Examples

>>> days = np.array([[1.0, -2.0, 3.0, 1.0]])
>>> represent(days, "shape", None).round(2).tolist()
[[0.33, -0.67, 1.0, 0.33]]
>>> represent(days, "eventsummary", high=2.0).shape
(1, 29)
milia.features.events.eventmap(start, duration, depth, floor=0.0001, standardise=True)#

Describe each event by its duration, depth and the time of day and of year at which it happens.

The columns of MAP_NAMES are the logarithm of the duration, the logarithm of the depth (floor keeps it finite for an event that only grazes its limit), and the sine and cosine of the angle of the mid-point of the event on the 24-hour clock and of its day on the 365-day year. Time sits on a circle because 23:00 and 01:00 are close: the Euclidean distance between two points of the unit circle is the chord 2 sin(d / 2) of the angle d between them, which is small across midnight and across new year. With standardise only the two logarithms are scaled to zero mean and unit variance; the circular columns keep the geometry of the unit circle, which scaling each of them by its own spread would distort whenever the events cluster at one time of day.

Parameters:
  • start (ArrayLike) – Start of each event, as datetimes.

  • duration (ArrayLike) – Duration of each event, in hours, positive.

  • depth (ArrayLike) – Distance beyond the limit at the worst step, positive, in the unit of the limit.

  • floor (float) – Added to the depth before the logarithm.

  • standardise (bool) – Scale the two logarithmic columns to zero mean and unit variance.

Return type:

ndarray

Returns:

An array of shape (events, 6).

Raises:

ValueError – If a duration is not positive, a depth is negative, or the inputs differ in length.

Examples

>>> starts = pd.to_datetime(["2019-01-01 23:00", "2019-07-02 11:30"])
>>> eventmap(starts, [2.0, 1.0], [0.05, 0.02], standardise=False).round(2)[:, [2, 3]].tolist()
[[0.0, 1.0], [0.0, -1.0]]
milia.features.events.exposure(events, units, group='pattern', hours='hours', groups=None, extra=None)#

Hours of event in each group, and the number of events, of every unit.

A unit is any entity that events happen to, such as a transformer at one rate of adoption. Each unit is described by the total duration of its events in every group of events, by further totals for selected kinds of event, and by its number of events. A unit without events has zeros and is not missing, so the table covers every unit.

Parameters:
  • events (DataFrame) – One row per event, with the columns named by units (the level names of a MultiIndex), group and hours.

  • units (Index) – Every unit, as an index whose level names are columns of events.

  • group (str) – Column that gives the group of each event, such as its violation pattern.

  • hours (str) – Column that gives the duration of each event.

  • groups (Sequence[Hashable] | None) – Values of group that get a column, in order; those present in events when omitted.

  • extra (Mapping[str, tuple[str, Hashable]] | None) – Further columns, each named, as the total duration of the events whose column equals a value, for example {"undervoltage": ("kind", "undervoltage")}.

Return type:

DataFrame

Returns:

A frame indexed by units with one column for each value of group, one for each extra entry and a column events.

Examples

>>> table = pd.DataFrame(
...     {"unit": ["a", "a", "b"], "pattern": [1, 2, 1], "hours": [2.0, 1.0, 4.0], "kind": ["x", "y", "x"]}
... )
>>> exposure(table, pd.Index(["a", "b", "c"], name="unit"), extra={"y": ("kind", "y")}).to_dict("list")
{'pattern1': [2.0, 4.0, 0.0], 'pattern2': [1.0, 0.0, 0.0], 'y': [1.0, 0.0, 0.0], 'events': [2, 1, 0]}